Skip to content

Commit

Permalink
Merge branch 'main' into DR-471-v126-release-tasks
Browse files Browse the repository at this point in the history
  • Loading branch information
databyjp committed Jul 3, 2024
2 parents a5cccf2 + 70da319 commit d09a58b
Show file tree
Hide file tree
Showing 115 changed files with 1,558 additions and 79 deletions.
22 changes: 15 additions & 7 deletions _includes/clients/matrix-intro.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,33 @@
These charts show the Weaviate client releases associated with Weaviate core releases.
For links to the client releases associated with the latest Weaviate core releases, expand this section.

### Current Weaviate core minor releases
<details>
<summary>Current core and client releases</summary>

|Weaviate Version|Release Date|Python|TypeScript|Go|Java|
|:-|:-|:-:|:-:|:-:|:-:|
| [1.25.5](https://github.com/weaviate/weaviate/releases/tag/v1.25.5) | 2024-06-24 | 4.6.5 | 3.0.8 | 4.14.0 | 4.7.0 |
| [1.25.6](https://github.com/weaviate/weaviate/releases/tag/v1.25.5) | 2024-06-24 | 4.6.5 | 3.0.8 | 4.14.0 | 4.7.0 |
| [1.25.5](https://github.com/weaviate/weaviate/releases/tag/v1.25.5) | 2024-06-24 | '' | '' | '' | '' |
| [1.25.4](https://github.com/weaviate/weaviate/releases/tag/v1.25.4) | 2024-06-13 | 4.6.3 | 3.0.3 | '' | '' |
| [1.25.3](https://github.com/weaviate/weaviate/releases/tag/v1.25.3) | 2024-06-11 | '' | '' | '' | '' |
| [1.25.2](https://github.com/weaviate/weaviate/releases/tag/v1.25.2) | 2024-05-31 | '' | 2.2.0 | '' | '' |
| [1.25.1](https://github.com/weaviate/weaviate/releases/tag/v1.25.1) | 2024-05-17 | 4.6.0 | 2.1.1 | 4.13.1 | 4.6.0 |
| [1.25.0](https://github.com/weaviate/weaviate/releases/tag/v1.25.0) | 2024-05-10 | '' | '' | '' | '' |

### Weaviate core major releases
</details>

For links to previous releases, expand this section.

<details>
<summary>Previous core and client releases</summary>

|Weaviate Version|Release Date|Python|TypeScript|Go|Java|
|:-|:-|:-:|:-:|:-:|:-:|
| [1.25.0](https://github.com/weaviate/weaviate/releases/tag/v1.25.0) | 2024-05-10 | 4.6.0 | 2.1.1 | 4.13.1 | 4.6.0 |
| [1.24.0](https://github.com/weaviate/weaviate/releases/tag/v1.24.0) | 2024-02-27 | 4.5.0 | 2.1.0 | 4.12.0 | 4.5.1 |
| [1.23.0](https://github.com/weaviate/weaviate/releases/tag/v1.23.0) | 2023-12-18 | 3.26.0 | 2.0.0 | '' |4.4.2 |
| [1.22.0](https://github.com/weaviate/weaviate/releases/tag/v1.22.0) | 2023-10-27 | 3.25.0 | '' | 4.10.0 | 4.3.0 |
| [1.21.0](https://github.com/weaviate/weaviate/releases/tag/v1.21.0) | 2023-08-17 | 3.22.1 |1.4.0 | 4.9.0 | 4.2.1 |
| [1.20.0](https://github.com/weaviate/weaviate/releases/tag/v1.20.0) | 2023-07-06 | 3.22.0 | '' | '' | 4.2.0 |
| [1.19.0](https://github.com/weaviate/weaviate/releases/tag/v1.19.0) | 2023-05-04 | 3.17.0 | 1.1.0[^1] | 4.7.1 | 4.0.1 |
| [1.19.0](https://github.com/weaviate/weaviate/releases/tag/v1.19.0) | 2023-05-04 | 3.17.0 | 1.1.0] | 4.7.1 | 4.0.1 |
| [1.18.0](https://github.com/weaviate/weaviate/releases/tag/v1.18.0) | 2023-03-07 | 3.13.0 | 2.14.5 | 4.6.2 | 3.6.4 |
| [1.17.0](https://github.com/weaviate/weaviate/releases/tag/v1.17.0) | 2022-12-20 | 3.9.0 | 2.14.0 | 4.5.0 | 3.5.0 |
| [1.16.0](https://github.com/weaviate/weaviate/releases/tag/v1.16.0) | 2022-10-31 | 3.8.0 | 2.13.0 | 4.4.0 | 3.4.0 |
Expand All @@ -42,4 +48,6 @@ These charts show the Weaviate client releases associated with Weaviate core rel
| [1.1.0](https://github.com/weaviate/weaviate/releases/tag/v1.1.0) | 2021-02-10 | 2.1.0 | '' | '' | - |
| [1.0.0](https://github.com/weaviate/weaviate/releases/tag/v1.0.0) | 2021-01-14 | 2.0.0 | '' | '' | - |

[^1]: The TypeScript client replaced the JavaScript client on 2023-03-17.
Note: The TypeScript client replaced the JavaScript client on 2023-03-17.

</details>
14 changes: 9 additions & 5 deletions _includes/code/generative.openai.singleresult.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,20 @@ package main
import (
"context"
"fmt"
"os"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
"github.com/weaviate/weaviate-go-client/v4/weaviate/auth"
)

func main() {
cfg := weaviate.Config{
Host: "WEAVIATE_INSTANCE_URL", // Replace with your instance URL
Host: os.Getenv("WEAVIATE_INSTANCE_URL"),
Scheme: "https",
Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
AuthConfig: auth.ApiKey{Value: os.Getenv("WEAVIATE_API_KEY")},
Headers: map[string]string{"X-OpenAI-Api-Key": os.Getenv("OPENAI_API_KEY"),},
}

client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
Expand All @@ -104,22 +108,22 @@ func main() {

fields := []graphql.Field{
{Name: "title"},
{Name: "summary"},
{Name: "content"},
}

concepts := []string{"Italian food"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)

// highlight-next-line
gs := graphql.NewGenerativeSearch().SingleResult("\"Describe the following as a Facebook Ad: {summary}\"")
generativeSearchPrompt := graphql.NewGenerativeSearch().SingleResult("Describe the following as a Facebook Ad: {summary}")

result, err := client.GraphQL().Get().
WithClassName("Article").
WithFields(fields...).
WithNearText(nearText).
// highlight-next-line
withGenerativeSearch(generativeSearch).
WithGenerativeSearch(generativeSearchPrompt).
WithLimit(5).
Do(ctx)

Expand Down
4 changes: 0 additions & 4 deletions _includes/code/howto/manage-data.collections.py
Original file line number Diff line number Diff line change
Expand Up @@ -444,9 +444,7 @@
sharding_config=Configure.sharding(
virtual_per_physical=128,
desired_count=1,
actual_count=1,
desired_virtual_count=128,
actual_virtual_count=128,
)
# highlight-end
)
Expand All @@ -457,9 +455,7 @@
config = collection.config.get()
assert config.sharding_config.virtual_per_physical == 128
assert config.sharding_config.desired_count == 1
assert config.sharding_config.actual_count == 1
assert config.sharding_config.desired_virtual_count == 128
assert config.sharding_config.actual_virtual_count == 128

# =========================
# ===== MULTI-TENANCY =====
Expand Down
4 changes: 0 additions & 4 deletions _includes/wcs/create-cluster-managed.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
import V125EarlyAccess from '/_includes/wcs/version-1-25-early-access.mdx';

<V125EarlyAccess />

1. Select the "Managed cluster" tab.
1. Give your cluster a name.
1. Select a version.
Expand Down
3 changes: 0 additions & 3 deletions _includes/wcs/version-1-25-early-access.mdx

This file was deleted.

3 changes: 0 additions & 3 deletions _includes/wcs/version-1-25-upgrade.mdx

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
---
title: Create and Load a Weaviate Cluster
slug: apple-apps-part-5
authors: [randy, adam]
date: 2024-07-02
tags: [iOS, mobile app development, vector databases]
image: ./img/hero.png

---
![Set up a Weaviate Cluster on WCD and load in book data using a python data pipeline that can be searched through your iOS and other Apple Ecosystem Applications.](./img/hero.png)


## Overview

In this article we will setup a Weaviate Cluster and load the same books data used in the article [IOS Intro - Search with Benefits](/apple-and-weaviate/apple-apps-part-1).

You can download the resources used in this blog [here](https://github.com/weaviate/BookRecs/blob/main/blog-resources/article05.zip).

### Quckstart

How to create a Weaviate account and cluster

https://weaviate.io/developers/wcs/quickstart

### Running Python on your Mac

How to run Python applications on a Mac.

### External Builder (Xcode) for Python

Using Xcode to run Python applications


![external-builder-image](./img/external-builder-image.png)


## Create a Cluster

Follow account setup and cluster creation as explained in the **Quickstart Tutorial** and take note of the following.


1. REST Endpoint URL

![create-a-cluster-1](./img/create-a-cluster-1.png)

2. Weaviate API Key
![create-a-cluster-2](./img/create-a-cluster-2.png)

## Create XCode Project

Create an Xcode Project using External Builder as described in **External Builder (Xcode) for Python**.

![create-xcode-project](./img/create-xcode-project.png)

## Install Weaviate Client

From the root directory of your Xcode Project, install the Weaviate Client.

pip install -U weaviate-client

Refer to the section **Install Python Packages and Setup a Virtual Environment** in the article **External Builder (Xcode) for Python** for guidance





## OpenAI Embeddings Access

Create an OpenAI API Key and note the number created.

https://www.howtogeek.com/885918/how-to-get-an-openai-api-key/

OpenAI is needed to translate text to vector embeddings for book data.

As of this article, it will cost less than 10 cents to use this one-time (create vector input to be loaded into the cluster) OpenAI feature.

## Enter Run Arguments

As described in the article External Builder (Xcode) for Python enter the Run Arguments.

The Cohere key can remain blank.

![enter-run-arguments](./img/enter-run-arguments.png)

## Support Files

What is included: A python script **main.py** to define the cluster and load book data, and book data **7k-books-kaggle.csv**.

These files were copied from Adam’s (of Weaviate) GitHub project

The python script was modified slightly for clarity.

https://github.com/weaviate/BookRecs

Copy these files to your Xcode project.

![support-files](./img/support-files.png)


### Book Data

A CSV of approximately 7,000 books.

![book-data](./img/book-data.png)

### Python Script to Create and Load Cluster

The **main.py** python script will do the following

Define the schema of a Books Custer
Create vector embeddings of Book Data
Load embeddings into the Books Cluster


Each step is logged in the console.

The longest step to run is the “Load Data” step which may take a couple of minutes.

Don’t forget to make sure that the virtual environment is activated before running.

source venv/bin/activate

![python-execution-logs](./img/python-execution-logs.png)

### main.py: Startup

API Keys are specified.

![main-py-image-startup](./img/main-py-image-startup.png)


### main.py: Cluster Defined

Defines the Weaviate Books cluster.

The **text2vec-openai** parameter defines a vectorizer to convert text into numerical vectors.

The specific OpenAI model used for embedding is **ADA Version 2**.

https://openai.com/index/new-and-improved-embedding-model



![main-py-image-collection](./img/main-py-image-collection.png)



### main.py: Read Book Data

Read the file of Book Data.

Make sure to define your directory path.


![main-py-image-read-books](./img/main-py-image-read-books.png)


### main.py: Load Data

Properties are mapped from the data file to the Weaviate cluster.

For each book record, text is converted to a vector and loaded into the Weaviate cluster.

![main-py-image-load-data](./img/main-py-image-load-data.png)


### main.py: Clean-up

Close the book data file and end the script.

![main-py-image-close](./img/main-py-image-close.png)


import StayConnected from '/_includes/stay-connected.mdx'

<StayConnected />
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit d09a58b

Please sign in to comment.