Refesh (#2294)

* Webinars Update (#2272)

* Update

* Blog Update

* Minor fixes iosblog (#2275)

* fix dates, title, linkedIn and twitter links

* Webinars Update (#2272) (#2274)

* Update

* Blog Update

Co-authored-by: Shan-Weaviate <137914800+Shan-Weaviate@users.noreply.github.com>

---------

Co-authored-by: AJ Chan <adam@ajchan.io>
Co-authored-by: Shan-Weaviate <137914800+Shan-Weaviate@users.noreply.github.com>

* Update docusaurus.config.js (#2277)

* People Update (#2278)

* Episode Update (#2279)

* Add `Kagome` Korean tokenizer docs (#2281)

* Draft Kagome tokenizer docs

* Add 'USE_GSE'

* add tokenization examples

* Update kagome available version

* 20240705 update banner (#2284)

* Draft banner

* Update time

* add many shot ICL paper (#2285)

* CTA Update (#2286)

* DR-521 release py4.6.6 core1.25.7 (#2283)

* Correct version links (#2287)

* DR-522 align section names with semantic naming (#2288)

* DR-522 align section names with semantic naming

* DR-522 align section names with semantic naming

* staging

* staging

* redirects

* add axn paper (#2289)

* add axn paper

* Update index.mdx

* Title Update (#2290)

* Fix version include (#2291)

---------

Co-authored-by: Shan-Weaviate <137914800+Shan-Weaviate@users.noreply.github.com>
Co-authored-by: AJ Chan <achan99@gmail.com>
Co-authored-by: AJ Chan <adam@ajchan.io>
Co-authored-by: JP Hwang <61258750+databyjp@users.noreply.github.com>
Co-authored-by: Zain Hasan <zain_has@hotmail.com>

_includes/clients/matrix-intro.md

@@ -5,7 +5,8 @@ For links to the client releases associated with the latest Weaviate core releas
 
 |Weaviate Version|Release Date|Python|TypeScript|Go|Java|
 |:-|:-|:-:|:-:|:-:|:-:|
-| [1.25.6](https://github.com/weaviate/weaviate/releases/tag/v1.25.5) | 2024-06-24 | 4.6.5 | 3.0.8 | 4.14.0 | 4.7.0 |
+| [1.25.7](https://github.com/weaviate/weaviate/releases/tag/v1.25.7) | 2024-07-04 | 4.6.6 | 3.0.8 | 4.14.0 | 4.7.0 |
+| [1.25.6](https://github.com/weaviate/weaviate/releases/tag/v1.25.6) | 2024-06-28 | 4.6.5 |'' | '' | '' |
 | [1.25.5](https://github.com/weaviate/weaviate/releases/tag/v1.25.5) | 2024-06-24 | '' | '' | '' | '' |
 | [1.25.4](https://github.com/weaviate/weaviate/releases/tag/v1.25.4) | 2024-06-13 | 4.6.3 | 3.0.3 | '' | '' |
 | [1.25.3](https://github.com/weaviate/weaviate/releases/tag/v1.25.3) | 2024-06-11 | '' | '' | '' | '' |

_includes/tokenization.mdx

@@ -1,3 +1,3 @@
-Weaviate converts filter terms into tokens. The default tokenization is `word`. The `word` tokenizer keeps alphanumeric characters and splits on whitespace. It converts a string like "test_domain_weaviate" into "test", "domain", and "weaviate".
+Weaviate converts filter terms into tokens. The default tokenization is `word`. The `word` tokenizer keeps alphanumeric characters, lowercase them and splits on whitespace. It converts a string like "Test_domain_weaviate" into "test", "domain", and "weaviate".
 
-For details and additional tokenization methods, see [Tokenization](/developers/weaviate/config-refs/schema#tokenization).
+For details and additional tokenization methods, see [Tokenization](/developers/weaviate/config-refs/schema#tokenization).

_includes/tokenization_definition.mdx

@@ -5,4 +5,5 @@
 | `whitespace`        | Split the text on whitespace. Searches/filters become case-sensitive.        | `Hello,`, `(beautiful)`, `world` |
 | `field`             | Index the whole field after trimming whitespace characters.                  | `Hello, (beautiful) world`       |
 | `trigram`           | Split the property as rolling trigrams.                                      | `Hel`, `ell`, `llo`, `lo,`, ...   |
-| `gse`               | Use the `gse` tokenizer to split the property.                               | [See `gse` docs](https://pkg.go.dev/github.com/go-ego/gse#section-readme) |
+| `gse`               | Use the `gse` tokenizer to split the property.                               | [See `gse` docs](https://pkg.go.dev/github.com/go-ego/gse#section-readme) |
+| `kagome_kr`         | Use the `Kagome` tokenizer with a Korean dictionary to split the property.   | [See `kagome` docs](https://github.com/ikawaha/kagome) and the [Korean dictionary](https://github.com/ikawaha/kagome-dict-ko) |

apple-and-weaviate/2024-06-14-create-and-load-a-weaviate-cluster/index.mdx → apple-and-weaviate/2024-07-02-create-and-load-a-weaviate-cluster/index.mdx

@@ -18,17 +18,15 @@ You can download the resources used in this blog [here](https://github.com/weavi
 
 ### Quckstart
 
-    How to create a Weaviate account and cluster
-
-    https://weaviate.io/developers/wcs/quickstart
+How to [create a Weaviate account and cluster](https://weaviate.io/developers/wcs/quickstart)
 
 ### Running Python on your Mac
 
-    How to run Python applications on a Mac.
+How to run [Python applications on a Mac](/apple-and-weaviate/apple-apps-part-3).
 
 ### External Builder (Xcode) for Python
 
-    Using Xcode to run Python applications
+Using [Xcode to run Python](/apple-and-weaviate/apple-apps-part-4) applications
 
 
 ![external-builder-image](./img/external-builder-image.png)

apple-and-weaviate/2024-06-14-external-builder-xcode-for-python/index.mdx → apple-and-weaviate/2024-07-02-external-builder-xcode-for-python/index.mdx

@@ -55,12 +55,12 @@ The alias points to an executable file.
 	/opt/anaconda3/bin/python3.11
 
 
-Refer to the article **Running Python on your Mac**, for more details, if this sounds confusing.
+Refer to the article [Running Python on your Mac](/apple-and-weaviate/apple-apps-part-3), for more details, if this sounds confusing.
 
 
 ## Install Python Packages and Setup a Virtual Environment
 
-See the article **Running Python on your Mac** for instructions.
+See the article [Running Python on your Mac](/apple-and-weaviate/apple-apps-part-3) for instructions.
 
 Install Python packages into a `venv` directory within your project to create a virtual environment.
 

apple-and-weaviate/2024-06-14-ios-intro-search-with-benefits/index.mdx → apple-and-weaviate/2024-07-02-ios-intro-search-with-benefits/index.mdx

@@ -15,7 +15,7 @@ Convert your structured data to a Weaviate Vector Database and unlock powerful s
 
 In this article we will talk about some of these important features.
 
-- This article includes a ready-to-use Swift app and all the code.
+- This article includes a ready-to-use Swift app and all the code. Download it from the Git repo [here](https://github.com/weaviate/BookRecs/tree/main/blog-resources/Search-With-Benefits.swiftpm).
 - You can run the code in a Swift Playground, which is an interactive learning environment for Swift.
 - By running the code in the Playground, you can experiment with Swift and try out different ideas.
 

authors.yml

@@ -157,7 +157,7 @@ victoria:
   image_url: /img/people/icon/victoria.png
 adam:
   name: Adam Chan
-  title: Developer Relations Engineer
+  title: Developer Growth Engineer
   url: https://www.linkedin.com/in/itsajchan/
   image_url: /img/people/icon/adam.png
 asdine:

blog/2024-07-09-iOS-apps-and-weaviate/index.mdx → blog/2024-07-02-iOS-apps-and-weaviate/index.mdx

@@ -39,11 +39,10 @@ Each article is designed to be followed in the following sequence.
 1. [Running Python on your Mac](/apple-and-weaviate/apple-apps-part-3). The basics on running python on your Mac.
 1. [External Builder (Xcode) for Python](/apple-and-weaviate/apple-apps-part-4). Using Xcode External Builder to run python scripts.
 1. [Create and Load Cluster](/apple-and-weaviate/apple-apps-part-5). Create and load a Weaviate Cluster.
-Special Thanks
 
-These articles are based on Adam Chan’s Book Recommendation project.
+These articles are based on Adam Chan’s Book Recommendation project found at this Git repo [here](https://github.com/weaviate/BookRecs).
+
 
-https://github.com/weaviate/BookRecs
 
 Without his collaboration and support these articles would not have been possible.
 

developers/academy/py/tokenization/200_options.mdx

@@ -125,12 +125,14 @@ This means that after tokenization, any stop words in the text behave as if they
 
 The above tokenization methods work well for English, or other languages that use spaces to separate words.
 
-However, not all languages rely on spaces to define natural semantic boundaries. For languages like Japanese or Chinese, where words are not separated by spaces, you may need to use a different tokenization method.
+However, not all languages rely on spaces to define natural semantic boundaries. For languages like Japanese, Chinese or Korean, where words are not separated by spaces, you may need to use a different tokenization method.
 
-Starting with `v1.24` Weaviate provides `gse` and `trigram` tokenization methods for this reason.
+Weaviate provides `gse` and `trigram` (from `v1.24`) and `kagome_kr` (from `v1.25.7`) tokenization methods for this reason.
 
 `gse` implements the "Jieba" algorithm, which is a popular Chinese text segmentation algorithm. `trigram` splits text into all possible trigrams, which can be useful for languages like Japanese.
 
+`kagome_kr` uses the [`Kagome` tokenizer](https://github.com/ikawaha/kagome?tab=readme-ov-file) with a Korean MeCab ([mecab-ko-dic](https://bitbucket.org/eunjeon/mecab-ko-dic/src/master/)) dictionary to split the property text. This is useful for Korean text.
+
 ## Questions and feedback
 
 import DocsFeedback from '/_includes/docs-feedback.mdx';

developers/weaviate/client-libraries/python/index.md

@@ -1107,6 +1107,7 @@ For links to the Python Client releases, expand this section.
 
 | Client Version | Release Date |
 | :- | :- |
+| [4.6.6](https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.5) | 2024-07-02 |
 | [4.6.5](https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.5) | 2024-06-19 |
 | [4.6.4](https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.4) | 2024-06-19 |
 | [4.6.3](https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.3) | 2024-05-21 |

developers/weaviate/concepts/modules.md

@@ -24,7 +24,7 @@ This page explains what modules are, and what purpose they serve in Weaviate.
 
 ## Available module types
 
-This graphic displays the available modules for the latest Weaviate version (v||site.weaviate_version|). Modules are grouped into these categories:
+This graphic displays the available modules for the latest Weaviate version (||site.weaviate_version||) . Modules are grouped into these categories:
 
 - Vectorization modules
 - Vectorization and additional functionality modules

developers/weaviate/config-refs/env-vars.md

@@ -26,7 +26,9 @@ All other values are interpreted as `false`.
 | `DISK_USE_READONLY_PERCENTAGE` | If disk usage is higher than the given percentage all shards on the affected node will be marked as `READONLY`, meaning all future write requests will fail. See [Disk Pressure Warnings and Limits for details](/developers/weaviate/configuration/persistence.md#disk-pressure-warnings-and-limits). | `string - number` | `90` |
 | `DISK_USE_WARNING_PERCENTAGE` | If disk usage is higher than the given percentage a warning will be logged by all shards on the affected node's disk. See [Disk Pressure Warnings and Limits for details](/developers/weaviate/configuration/persistence.md#disk-pressure-warnings-and-limits). | `string - number` | `80` |
 | `DISABLE_GO_PROFILING` | If `true`, disables Go profiling. Default: `false`. | `boolean` | `false` |
-| `ENABLE_MODULES` | Which modules to enable in the setup? | comma-separated list of strings | `text2vec-openai,generative-openai` |
+| `ENABLE_MODULES` | Specify Weaviate modules to enable | `string - comma separated names` | `text2vec-openai,generative-openai` |
+| `ENABLE_TOKENIZER_GSE` | Enable the [`GSE` tokenizer](../config-refs/schema/index.md#gse-and-trigram-tokenization-methods) for use | `boolean` | `true` |
+| `ENABLE_TOKENIZER_KAGOME_KR` | Enable the [`Kagome` tokenizer for Korean](../config-refs/schema/index.md#kagome_kr-tokenization-method) for use (Experimental as of `v1.25.7`) | `boolean` | `true` |
 | `GODEBUG` | Controls debugging variables within the runtime. [See official Go docs](https://pkg.go.dev/runtime). | `string - comma-separated list of name=val pairs` | `gctrace=1` |
 | `GOMAXPROCS` | Set the maximum number of threads that can be executing simultaneously. If this value is set, it be respected by `LIMIT_RESOURCES`. | `string - number` | `NUMBER_OF_CPU_CORES` |
 | `GOMEMLIMIT` | Set the memory limit for the Go runtime. This should match your available memory, such as 10-20% of your total memory for Weaviate. The Go runtime tries to make sure that long-lived and temporary memory allocations do not exceed this value by making the garbage collector more aggressive as the memory usage approaches the limit. [Learn more about GOMEMLIMIT](/blog/gomemlimit-a-game-changer-for-high-memory-applications). | `string - memory limit in SI units` | `4096MiB` |
@@ -49,6 +51,7 @@ All other values are interpreted as `false`.
 | `REINDEX_SET_TO_ROARINGSET_AT_STARTUP` | Allow Weaviate to perform a one-off re-indexing to [use Roaring Bitmaps](../concepts/prefiltering.md#migration-to-roaring-bitmaps). <br/><br/>Available in versions `1.18` and higher. | `boolean` | `true` |
 | `TOMBSTONE_DELETION_MAX_PER_CYCLE` | Maximum number of tombstones to delete per cleanup cycle. Set this to limit cleanup cycles, as they are resource-intensive. As an example, set a maximum of 10000000 (10M) for a cluster with 300 million-object shards. (Default: none) | `string - int` (Available from `v1.24.15` / `v1.25.2`) | `10000000` |
 | `TOMBSTONE_DELETION_MIN_PER_CYCLE` | Minimum number of tombstones to delete per cleanup cycle. Set this to prevent triggering unnecessary cleanup cycles below a threshold. As an example, set a minimum of 1000000 (1M) for a cluster with 300 million-object shards. (Default: 0) (Available from `v1.24.15` / `v1.25.2`) | `string - int` | `100000` |
+| `USE_GSE` | Enable the [`GSE` tokenizer](../config-refs/schema/index.md#gse-and-trigram-tokenization-methods) for use. <br/> (The same as `ENABLE_TOKENIZER_GSE`. We recommend using `ENABLE_TOKENIZER_GSE` for consistency in naming with other optional tokenizers.) | `boolean` | `true` |
 
 ## Module-specific
 

developers/weaviate/config-refs/schema/index.md

@@ -558,6 +558,28 @@ So, a `string` property value `Hello, (beautiful) world` with `tokenization` set
 
 For Japanese and Chinese text, we recommend use of `gse` or `trigram` tokenization methods. These methods work better with these languages than the other methods as these languages are not easily able to be tokenized using whitespaces.
 
+The `gse` tokenizer is not loaded by default to save resources. To use it, set the environment variable `ENABLE_TOKENIZER_GSE` to `true` on the Weaviate instance.
+
+`gse` tokenization examples:
+
+- `"素早い茶色の狐が怠けた犬を飛び越えた"`: `["素早", "素早い", "早い", "茶色", "の", "狐", "が", "怠け", "けた", "犬", "を", "飛び", "飛び越え", "越え", "た", "素早い茶色の狐が怠けた犬を飛び越えた"]`
+- `"すばやいちゃいろのきつねがなまけたいぬをとびこえた"`: `["すばや", "すばやい", "やい", "いち", "ちゃ", "ちゃい", "ちゃいろ", "いろ", "のき", "きつ", "きつね", "つね", "ねが", "がな", "なま", "なまけ", "まけ", "けた", "けたい", "たい", "いぬ", "を", "とび", "とびこえ", "こえ", "た", "すばやいちゃいろのきつねがなまけたいぬをとびこえた"]`
+
+### `kagome_kr` tokenization method
+
+:::caution Experimental feature
+Available starting in `v1.25.7`. This is an experimental feature. Use with caution.
+:::
+
+For Korean text, we recommend use of the `kagome_kr` tokenization method. This uses the [`Kagome` tokenizer](https://github.com/ikawaha/kagome?tab=readme-ov-file) with a Korean MeCab ([mecab-ko-dic](https://bitbucket.org/eunjeon/mecab-ko-dic/src/master/)) dictionary to split the property text.
+
+The `kagome_kr` tokenizer is not loaded by default to save resources. To use it, set the environment variable `ENABLE_TOKENIZER_KAGOME_KR` to `true` on the Weaviate instance.
+
+`kagome_kr` tokenization examples:
+
+- `"아버지가방에들어가신다"`: `["아버지", "가", "방", "에", "들어가", "신다"]`
+- `"아버지가 방에 들어가신다"`: `["아버지", "가", "방", "에", "들어가", "신다"]`
+- `"결정하겠다"`: `["결정", "하", "겠", "다"]`
 
 ### `indexFilterable` and `indexSearchable`
 

developers/weaviate/manage-data/collections.mdx

@@ -392,7 +392,7 @@ Various vector index parameters are configurable at collection creation time, in
 
 ## Property-level settings
 
-Configure each property to choose whether to vectorize property name, include property in vectorization, and choose tokenization type.
+Configure each property to choose whether to vectorize the property and its name, and choose a [tokenization type](../config-refs/schema/index.md#tokenization).
 
 <Tabs groupId="languages">
   <TabItem value="py" label="Python Client v4">

developers/weaviate/release-notes/index.md

@@ -10,23 +10,30 @@ This page has links to the [developer release notes](https://github.com/weaviate
 
 For descriptive notes on recent major releases, follow the links in the table of contents at the left of this page. For descriptive notes on older releases, see the [blog pages at Weaviate.io](https://weaviate.io/blog/tags/release).
 
-### Current minor releases
+### Current core and client releases
+
+This chart shows the latest client release at the time of the Weaviate core release.
 
 | Weaviate Version | Release Date | Python | TypeScript<br/>JavaScript | Go | Java |
 | :- | :- | :-: | :-: | :-: | :-: |
-| [1.25.6][c1.25.6] | 2024-06-28 | [4.6.5][p4.6.5] | [3.0.8][ts3.0.8] | [4.14.0][g4.14.0] | [4.7.0][j4.7.0] |
+| [1.25.7][c1.25.7] | 2024-07-04 | [4.6.6][p4.6.6] | [3.0.8][ts3.0.8] | [4.14.0][g4.14.0] | [4.7.0][j4.7.0] |
+| [1.25.6][c1.25.6] | 2024-06-28 | [4.6.5][p4.6.5] | '' | '' | '' |
 | [1.25.5][c1.25.5] | 2024-06-24 | '' | '' | '' | '' |
 | [1.25.4][c1.25.4] | 2024-06-13 | [4.6.3][p4.6.3] | [3.0.3][ts3.0.3] | '' | '' |
 | [1.25.3][c1.25.3] | 2024-06-11 | '' | '' | '' | '' |
 | [1.25.2][c1.25.2] | 2024-05-31 | '' | [2.2.0][ts2.2.0] | ''| '' |
 | [1.25.1][c1.25.1] | 2024-05-17 | [4.6.2][p4.6.2] | '' | '' | '' |
 | [1.25.0][c1.25.0] | 2024-05-10 | [4.6.0][p4.6.0] | [2.1.1][ts2.1.1] | [4.13.1][g4.13.1] | [4.6.0][j4.6.0] |
 
-### Major releases
+### Previous core and client releases
+
+For links to the matrix of previous core and client releases, expand this section.
+
+<details>
+  <summary>Releases</summary>
 
 | Weaviate Version | Release Date | Python | TypeScript<br/>JavaScript | Go | Java | Up voted issues |
 | :- | :- | :-: | :-: | :-: | :-: | :-: |
-| [1.25.0][c1.25.0] | 2024-05-10 | [4.6.0][p4.6.0] | [2.1.1][ts2.1.1] | [4.13.1][g4.13.0] | [4.6.0][j4.6.0] | [Votes 1.25][v1.25] |
 | [1.24.0][c1.24.0] | 2024-02-27 | [4.5.1][p4.5.1] | [2.0.0][ts2.0.0] | [4.13.1][g4.13.0] | [4.6.0][j4.6.0] | [Votes 1.24][v1.24] |
 | [1.23.0][c1.23.0] | 2023-12-18 | [3.26.0][p3.26.0] | '' | '' | [4.4.2][j4.4.2] | [Votes 1.23][v1.23] |
 | [1.22.0][c1.22.0] | 2023-10-27 | [3.25.0][p3.25.0] | [1.5.0][ts1.5.0] | [4.10.0][g4.10.0] | [4.3.0][j4.3.0] | [ Votes 1.22][v1.22] |
@@ -53,7 +60,9 @@ For descriptive notes on recent major releases, follow the links in the table of
 | [1.1.0][c1.1.0] | 2021-02-10 | [2.1.0][p2.1.0] | '' | '' | - | - |
 | [1.0.0][c1.0.0] | 2021-01-14 | [2.0.0][p2.0.0] | '' | '' | - | - |
 
-<a name="typescriptChange"><sup>1</sup></a>The TypeScript client replaced the JavaScript client on 2023-03-17.
+Note: The TypeScript client replaced the JavaScript client on 2023-03-17.
+
+</details>
 
 ## Weaviate Core
 
@@ -64,6 +73,8 @@ For links to the Weaviate Core releases, expand this section.
 
   | Weaviate Version | Release Date |
   | :- | :- |
+  | [1.25.7][c1.25.7] | 2024-07-04 |
+  | [1.25.6][c1.25.6] | 2024-06-28 |
   | [1.25.5][c1.25.5] | 2024-06-24 |
   | [1.25.4][c1.25.4] | 2024-06-13 |
   | [1.25.3][c1.25.3] | 2024-06-11 |
@@ -226,6 +237,7 @@ For links to the Python Client releases, expand this section.
 
    | Client Version | Release Date |
    | :- | :- |
+   | [4.6.6][p4.6.6] | 2024-07-02 |
    | [4.6.5][p4.6.5] | 2024-06-19 |
    | [4.6.4][p4.6.4] | 2024-06-19 |
    | [4.6.3][p4.6.3] | 2024-05-21 |
@@ -512,6 +524,7 @@ For links to the Java Client releases, expand this section.
 
 
 [comment]: # ( core links )
+[c1.25.7]: https://github.com/weaviate/weaviate/releases/tag/v1.25.7
 [c1.25.6]: https://github.com/weaviate/weaviate/releases/tag/v1.25.6
 [c1.25.5]: https://github.com/weaviate/weaviate/releases/tag/v1.25.5
 [c1.25.4]: https://github.com/weaviate/weaviate/releases/tag/v1.25.4
@@ -673,6 +686,7 @@ For links to the Java Client releases, expand this section.
 
 [comment]: # ( python client links )
 
+[p4.6.6]: https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.6
 [p4.6.5]: https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.5
 [p4.6.4]: https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.4
 [p4.6.3]: https://github.com/weaviate/weaviate-python-client/releases/tag/v4.6.3