Changelog

2024-09-26

Vectorize GA

Vectorize is now generally available

2024-09-16

Vectorize is available on Workers Free plan

Developers with a Workers Free plan can:

• Query up to 30 million queried vector dimensions / month per account.
• Store up to 5 million stored vector dimensions per account.

2024-08-14

Vectorize v1 is deprecated

With the new Vectorize storage engine, which supports substantially larger indexes (up to 5 million vector dimensions) and reduced query latencies, we are deprecating the original "legacy" (v1) storage subsystem.

To continue interacting with legacy (v1) indexes in wrangler versions after 3.71.0, pass the --deprecated-v1 flag.

For example: 'wrangler vectorize --deprecated-v1' flag to create, get, list, delete and insert vectors into legacy Vectorize v1 indexes. There is no currently no ability to migrate existing indexes from v1 to v2. Existing Workers querying or clients to use the REST API against legacy Vectorize indexes will continue to function.

Vectorize v2 in public beta

Vectorize now has a new underlying storage subsystem (Vectorize v2) that supports significantly larger indexes, improved query latency, and changes to metadata filtering.

Specifically:

• Indexes can now support up to 5 million vector dimensions each, up from 200,000 per index.
• Metadata filtering now requires explicitly defining the metadata properties that will be filtered on.
• Reduced query latency: queries will now return faster and with lower-latency.
• You can now return up to 100 results (topK), up from the previous limit of 20.

2024-01-17

HTTP API query vectors request and response format change

Vectorize /query HTTP endpoint has the following changes:

• returnVectors request body property is deprecated in favor of returnValues and returnMetadata properties.
• Response format has changed to the below format to match [Workers API change]:(/workers/configuration/compatibility-dates/#vectorize-query-with-metadata-optionally-returned)

{
  "result": {
    "count": 1,
    "matches": [
      {
        "id": "4",
        "score": 0.789848214,
        "values": [ 75.0999984741211, 67.0999984741211, 29.899999618530273],
        "metadata": {
          "url": "/products/sku/418313",
          "streaming_platform": "netflix"
        }
      }
    ]
  },
  "errors": [],
  "messages": [],
  "success": true
}

2023-12-06

Metadata filtering

Vectorize now supports metadata filtering with equals ($eq) and not equals ($neq) operators. Metadata filtering limits query() results to only vectors that fulfill new filter property.

let metadataMatches = await env.YOUR_INDEX.query(queryVector,
  {
    topK: 3,
    filter: { streaming_platform: "netflix" },
    returnValues: true,
    returnMetadata: true
  })

Only new indexes created on or after 2023-12-06 support metadata filtering. Currently, there is no way to migrate previously created indexes to work with metadata filtering.

2023-11-08

Metadata API changes

Vectorize now supports distinct returnMetadata and returnValues arguments when querying an index, replacing the now-deprecated returnVectors argument. This allows you to return metadata without needing to return the vector values, reducing the amount of unnecessary data returned from a query. Both returnMetadata and returnValues default to false.

For example, to return only the metadata from a query, set returnMetadata: true.

let matches = await env.YOUR_INDEX.query(queryVector, { topK: 5, returnMetadata: true })

New Workers projects created on or after 2023-11-08 or that update the compatibility date for an existing project will use the new return type.

2023-10-03

Increased indexes per account limits

You can now create up to 100 Vectorize indexes per account. Read the limits documentation for details on other limits, many of which will increase during the beta period.

2023-09-27

Vectorize now in public beta

Vectorize, Cloudflare's vector database, is now in public beta. Vectorize allows you to store and efficiently query vector embeddings from AI/ML models from Workers AI, OpenAI, and other embeddings providers or machine-learning workflows.

To get started with Vectorize, see the guide.