OriginChain docs
schema · vector

Vector search.

Vector search answers "what is closest to this?" — semantic retrieval for RAG, recommendations, deduplication, and anything where the query is an embedding rather than a predicate. You hand the engine a query vector and it returns the k nearest ids with their similarity scores.

Reach for it when meaning matters more than wording. If you need exact keyword matching, full-text search is cheaper and more precise; if you know the predicate, plain SQL beats both.

Every operation below is shown in cURL, Python, TypeScript and Go. Where an SDK does not wrap an endpoint the tab says so and shows the raw call instead.

1

Before you start.

Every example on this page uses the shop.orders table from the quickstart, with an embedding of the order's notes field attached to each row's id.

vectors are not a column type

There is no vector column type, and vectors cannot be written through the rows endpoint. They live in their own keyspace, addressed by (tenant, table, id), and are written with the dedicated /vector/… endpoints below. Using the same id on both sides is what lets you take a vector hit and read the full row back with SQL.

The :table path segment is free-form — it does not have to name a registered schema. Registering one anyway is worth it: it gives you SQL access to the same rows, and the optional [vector] block turns a dimension mistake into a readable error.

schemas/orders.toml
# The row schema. Vectors do NOT live in a column - they sit in their
# own keyspace, addressed by the same id. Registering the table is what
# lets you read the row back with SQL after a vector hit gives you an id.

namespace   = "shop"
table       = "orders"
primary_key = ["id"]

[[columns]]
name = "id"
ty   = "str"
required = true

[[columns]]
name = "customer"
ty   = "str"

[[columns]]
name = "amount_cents"
ty   = "i64"

[[columns]]
name = "status"
ty   = "str"

[[columns]]
name = "notes"
ty   = "str"

[[columns]]
name = "placed_ms"
ty   = "u64"

# Optional. Declares the dimensionality the collection expects so a
# wrong-sized vector is refused with a readable message instead of a
# generic mismatch. `distance` accepts cosine | l2 | dot only.
[vector]
dim      = 768
distance = "cosine"
the [vector] block checks dim, not metric

dim is enforced on every write and query. distance is declarative only — it is never compared against the metric you send at runtime, so it documents intent rather than enforcing it. Note also that the schema block accepts only cosine, l2 and dot, while the runtime metric field additionally accepts manhattan.

2

Insert vectors.

id, embedding and dim are required. metadata is a free-form JSON object stored alongside the vector — it is what filtering reads later, so put anything you may want to narrow on in there at write time.

Writes are indexed eagerly and atomically: the embedding and the updated graph ship in one batch, so the index can never lag the data across a crash. There is no "build the index" step for the default index — a vector is queryable as soon as the call returns.

single insert
POST /v1/tenants/:tenant/vector/:table/put
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/put" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id":        "01JTRX9KQ3YH8K2WMX0F5JZAB7",
    "embedding": [0.0124, -0.0883, 0.0451],
    "dim":       3,
    "metric":    "cosine",
    "metadata":  { "status": "paid", "customer": "01JTRX1H4Q9P0N2WMX0F5JZ001" }
  }'
# → 201 Created, empty body
bulk insert

Bulk is the right shape for ingest: it builds the graph in one pass and writes one log frame for the whole batch. dim, metric, quantization and index live on the envelope; each item carries only id, embedding and metadata. The body-size limit is lifted on this route only.

POST /v1/tenants/:tenant/vector/:table/put_bulk
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/put_bulk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "dim":    768,
    "metric": "cosine",
    "vectors": [
      { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB7", "embedding": [/* 768 floats */],
        "metadata": { "status": "paid" } },
      { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB8", "embedding": [/* 768 floats */],
        "metadata": { "status": "refunded" } }
    ]
  }'
response
{
  "inserted":   2,
  "elapsed_ms": 14
}
  • At most 100,000 vectors per call — over that the engine answers 413, not 400.
  • An empty vectors array is a 200 with inserted: 0 and no write.
  • Duplicate ids inside one batch are last-writer-wins. The whole batch is one atomic write.
  • Single insert returns 201 Created with an empty body — don't try to parse it.
sparse vectors

Learned-sparse embeddings go to POST /vector/:table/put_sparse with { id, indices, values, dim, metadata? }, and are queried with POST /vector/:table/topk_sparse. indices and values must be the same length, every index must be inside dim, and non-finite values are refused. No SDK wraps the sparse endpoints today.

3

Query — top-k.

query, k and dim are required; everything else has a default. The field is k — there is no top_k alias.

POST /v1/tenants/:tenant/vector/:table/topk
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [0.0124, -0.0883, 0.0451],
    "k":      5,
    "dim":    3,
    "metric": "cosine"
  }'
response
[
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB7", "score": 0.9421 },
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB8", "score": 0.9187 },
  { "id": "01JTRX9KQ3YH8K2WMX0F5JZAB9", "score": 0.8804 }
]

A bare JSON array, not an envelope — no hits wrapper and no total count. Sorted by score descending, and larger always means closer: for l2 and manhattan the distance is returned negated so the ordering convention holds across every metric.

k: 0 returns [] without touching storage. The ceiling on k is 4096 by default; above it you get a 400.

4

Query modes — fast vs high_recall.

mode is the one recall/latency knob exposed on the API. It sets the search beam width and nothing else — the build-time graph parameters are unaffected.

mode Beam width Measured (100k vectors, 128-dim)
"high_recall" 1200 Default. recall@10 ≈ 0.96, p99 ≈ 109 ms
"fast" 300 recall@10 ≈ 0.69, p99 ≈ 37 ms
"mode": "fast"
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "mode":   "fast"
  }'
  • Omitting mode gives you high_recall. That is the safe default and the right one for first-pass retrieval.
  • The value is case-sensitive. An unknown value is a hard 400: unknown `mode` "FAST": expected one of "fast", "high_recall".
  • mode applies to the default graph index only. On ivf and ivf_pq queries it is accepted and then ignored — those paths have no beam width. Use nprobe there instead.
  • The recall figures above are the published measurements for one corpus shape. Treat them as a guide to the trade-off, not an SLA for your data.
seeing what a query actually did

POST /vector/:table/topk_explain takes the same body and returns { hits, config }, where config reports the resolved metric, dim, k, ef_search, m and beam_width. Useful for confirming a mode actually took effect. It always runs the graph index, whatever index you pass.

5

Metadata filtering.

filter is a flat map of strict equalities against the metadata you stored at write time. Multiple keys are ANDed. There are no operators — no ranges, no $in, no nesting, no negation.

filtered top-k
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      10,
    "dim":    768,
    "metric": "cosine",
    "filter": { "status": "paid" }
  }'

Values are compared with strict JSON equality, so 1 does not match "1" and true does not match "true". A record that is missing a key named in the filter is rejected, not treated as null.

a filtered query can return fewer than k hits

On the default graph index, filtering is a post-filter: the engine searches for k × 4 candidates and then drops the ones that don't match. If your filter is highly selective, most of the over-fetched set is discarded and you get back fewer than k results — even when more matching vectors exist. Ask for a larger k than you need when you filter narrowly.

filter is silently ignored on ivf_pq

A filter sent with "index": "ivf_pq" is not applied, and the request still returns 200 with unfiltered results. This is the sharpest edge on this page: filter and IVF-PQ do not compose today. Filter on the default index, or apply the predicate yourself after the hits come back. On "index": "ivf" the filter is honoured, and applied inside each cell scan before scoring.

6

Metrics and dimensions.

metric When to use it
"cosine" The default, and the right answer for almost every text embedding model. Angle only — magnitude is ignored. A zero-magnitude vector scores 0.
"dot" Inner product. Use when your model is trained for it and magnitude carries signal (some retrieval and recommender models).
"l2" Euclidean distance, returned negated. Common for image and audio embeddings. Note the accepted string is l2"euclidean" is not recognised.
"manhattan" L1 distance, returned negated. "l1" is accepted as an alias. Not available in the [vector] schema block.
an unrecognised metric falls back to cosine — silently

Metric parsing is case-insensitive and has no validation arm: "euclidean", "hamming" or a typo like "cosin" all resolve to cosine with a 200. You will get plausible-looking results scored under the wrong metric. Spell the four accepted values exactly.

Dimensions.

The only hard rule is dim > 0 — there is no upper bound on dimensionality in the engine. What matters is consistency: every vector in a collection, and every query against it, must agree.

A mismatch is a 400. With a [vector] block registered you get the friendly form:

vector has 512 dims but collection "shop.orders" expects 768.
Use the same model or re-index the collection.

Without one, the underlying error surfaces instead: vec: dimension mismatch: expected 768, got 512.

Picking a dimensionality.

  • Storage is dim × 4 bytes per raw vector, so 1536-dim costs exactly twice 768-dim before any index overhead.
  • Graph search cost scales with dim too — every distance computation touches every component.
  • Many modern embedding models support truncation (Matryoshka-style). Halving dimensions usually costs a little recall and saves a lot of memory; measure on your own data before committing, because the collection's dim is fixed once you have written vectors at it.
7

Index kinds.

index appears on both the write and the query body, and the two must agree — a vector written under one index kind is not visible to a query using another. Omitting it everywhere is the common and correct choice.

index What it is
"hnsw" The default. A navigable small-world graph, built incrementally on every insert. Build parameters are fixed: 16 neighbours per node (32 at the base layer) and a construction beam of 200. No setup, no training, no minimum corpus. This is what you want unless you have measured a reason otherwise.
"ivf" Inverted file: vectors are assigned to the nearest of K centroids, and a query scans only nprobe of those cells. Requires centroids to be installed or trained first. Filters are applied inside the cell scan.
"ivf_pq" Inverted file plus product quantization — each vector is stored as a short code instead of full floats. This is the memory-footprint option for large corpora, and the one the presets build.

Approximate, not exact.

All three index kinds are approximate nearest-neighbour structures: they trade a small amount of recall for a large amount of speed, and none of them guarantees that the true nearest neighbour is in the result set. That is the deal ANN makes, and it is almost always the right one — an exhaustive scan is exact but linear in corpus size, which stops being viable long before a million vectors. If exactness genuinely matters for a small collection, raise k and re-rank the candidates yourself with your own distance function.

8

Building an IVF-PQ index from a preset.

IVF-PQ has a lot of knobs — partition count, subspace count, code width. Rather than expose them, the engine ships exactly two named presets and derives every parameter from your corpus. preset is the only required field.

POST /v1/tenants/:tenant/vector/:table/create-ivf-pq-index
curl -X POST \
  "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/create-ivf-pq-index" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "preset": "compressed" }'
response — the config it chose
{
  "trained":              true,
  "installed":            true,
  "preset":               "compressed",
  "partitions":           1024,
  "pq_m":                 48,
  "pq_bits":              8,
  "keep_raw":             false,
  "dim":                  768,
  "training_corpus_size": 50000
}

The optional seed field (default 0) makes training deterministic. There are no other fields — you cannot override partitions, subspace count or code width.

build then query — no re-insert needed

This endpoint populates the index cells as part of the build, feeding your stored vectors back through the same write path an IVF-PQ insert would use. The index is queryable the moment the call returns — you do not have to re-insert your rows under index: "ivf_pq". This is specific to this preset endpoint: the lower-level centroid-install primitives deliberately do not write postings.

How each preset resolves.

Both presets pick the same partition count and the same 8-bit codes. They differ in how finely the vector is subdivided, and in whether the original vector is kept.

Parameter high_recall compressed
Target sub-vector width 8 16
pq_m (subspaces) The divisor of dim — capped at 64 — whose sub-vector width dim / m lands closest to the target above.
pq_bits 8 8
keep_raw true false
partitions 4 × √N rounded to the nearest power of two, clamped to [64, 65536]. Identical for both presets.

Because pq_m must divide dim and is capped at 64, the two presets converge at high dimensionality. Worked from the real formula:

dim high_recall pq_m compressed pq_m Code bytes / vector
128 16 8 16 B vs 8 B
768 64 48 64 B vs 48 B
1536 64 64 64 B vs 64 B

At 1536 dimensions the codes are identical — both presets hit the 64-subspace cap — and the only remaining difference is keep_raw. Since keep_raw stores an extra dim × 4 bytes per vector, at 1536 dimensions high_recall costs about 6.2 KB per vector against 64 bytes for compressed — roughly a 97× difference in footprint for the same search behaviour today.

what keep_raw does and does not buy you today

high_recall retains the full-precision vector so that a candidate set can later be re-scored exactly. The HTTP top-k path does not perform that re-rank today — an ivf_pq query scores against the quantized codes for both presets. So on the current API the two presets return comparable results, and high_recall's extra storage is buying future re-ranking rather than present accuracy. If footprint is why you are reaching for IVF-PQ at all, compressed is the honest choice.

Which to pick.

  • Neither, at small scale. Under a few hundred thousand vectors the default graph index is faster and more accurate, and needs no build step. IVF-PQ is a memory-footprint tool, not a speed tool.
  • compressed when the corpus no longer fits comfortably in memory and you want the smallest possible resident footprint.
  • high_recall when you want the raw vectors kept alongside the codes — for exact re-ranking you perform yourself, or to be ready for engine-side re-ranking without a rebuild.

Querying the built index.

Pass "index": "ivf_pq" and, optionally, nprobe — the number of cells to visit.

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/topk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query":  [/* 768 floats */],
    "k":      20,
    "dim":    768,
    "metric": "cosine",
    "index":  "ivf_pq",
    "nprobe": 16
  }'
  • nprobe defaults to min(8, partitions). Higher visits more cells: better recall, more work.
  • Valid range is 1 to 256. 0 is a 400 (nprobe must be >= 1); above 256 is a 400 naming the cap.
  • nprobe is ignored on the default graph index — it only means something for ivf and ivf_pq.
  • In the Python SDK nprobe is available on the typed db.vector.topk() namespace. No other SDK exposes it.

The minimum-corpus rule.

Training refuses to run on a corpus too small to populate its partitions: you need at least four vectors per partition. Because the partition count is floored at 64, that means a practical floor of 256 vectors — and far more once 4 × √N pushes the partition count up.

{
  "error": "not enough vectors to build a Compressed index: 1024 partitions
            need >=4*K = 4096 vectors, found 900"
}

Building on an empty table is a separate 400: no vectors stored for this table; put vectors before building an index. Note that the preset name is echoed in the error in its internal capitalised form (Compressed, HighRecall) rather than the wire form you sent.

the lower-level centroid endpoints

Plain ivf (without quantization) is driven by four separate endpoints, wrapped by the Python SDK only:

  • POST …/train-and-install-centroidsdb.vector.train_and_install_centroids(table, partitions=…). Same four-per-partition minimum.
  • POST …/install-centroidsdb.vector.install_centroids(table, centroids) for centroids you trained elsewhere.
  • GET …/centroidsdb.vector.centroids(table), a truncated preview.
  • GET …/ivf-rebalance-statusdb.vector.rebalance_status(table), reporting skew and whether a rebalance is none, recommended or required.

Unlike the preset endpoint, installing centroids does not write postings for existing rows — that path is "install once, then write". Querying an IVF index with no centroids installed returns 503, not 404, with the install URLs in the response body.

hybrid dense + sparse

POST /vector/:table/topk_hybrid runs a dense and a sparse query together and fuses the two rankings server-side with Reciprocal Rank Fusion. Fields: dense_query, dense_dim, sparse_query_indices, sparse_query_values, sparse_dim, k, plus optional dense_metric, dense_mode, rrf_k (default 60), candidates and filter. The returned score is a fused rank score, not a distance — it is not comparable to the scores from a single-mode query. No SDK wraps it.

9

Delete.

DELETE /vector/:table/:vec_id · POST /vector/:table/delete-bulk
# Single - idempotent, 200 with {"deleted": false} when the id is absent
curl -X DELETE \
  "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/01JTRX9KQ3YH8K2WMX0F5JZAB7" \
  -H "Authorization: Bearer $OC_TOKEN"

# Bulk - at most 10,000 ids per call
curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/vector/shop.orders/delete-bulk" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "ids": ["01JTRX...AB7", "01JTRX...AB8"] }'
  • Single delete is idempotent: deleting an id that isn't there is a 200 with { "deleted": false }, never a 404.
  • Bulk delete caps at 10,000 ids and returns { deleted_count, missing_count }.
  • Both accept an optional index selector (hnsw by default) — it must match the index the vector was written under.
  • Vector ids are capped at 1024 characters and may not be empty.
10

Limits and gotchas.

Limit Value
Max k per query4096
Max vectors per bulk insert100,000
Max ids per bulk delete10,000
Max nprobe256
Max IVF partitions65,536
Max PQ subspaces (pq_m)64
Vectors sampled for index training1,000,000
Max vector id length1024 chars
Dimensionality> 0, no upper bound
metric and index are per-request, not per-collection

Nothing stores "this collection is cosine". You can write with cosine and query with l2 and get no error — just meaningless rankings. There is no metric-mismatch error anywhere. Same for index: writing under the default graph index and then querying index: "ivf_pq" finds nothing at all, unless the preset build populated those cells. Pick one metric and one index kind per collection and hold them constant in your own code.

training and populate sample at most one million vectors

A preset build reads up to 1,000,000 vectors, both for training and for populating cells. On a collection larger than that, rows beyond the first million are not written into the IVF-PQ index by the build and will not be found by an ivf_pq query until they are written again under that index.

back-pressure and quota

Queries take a process-wide heavy-operation permit before touching storage; under memory pressure you get 429 with a Retry-After. Writes are checked against your vector quota and answer 402 when it is exhausted — a bulk insert is pre-checked for the whole batch, so it is all-or-nothing. Vector endpoints also require the vector add-on to be enabled on the instance; without it the call fails with a 402 naming the add-on.

quantization on the write path

Separately from IVF-PQ, a write can carry quantization: "none" (default), "scalar", "binary" or "pq". This shrinks the stored payload at some cost in precision. Note that under binary quantization cosine and dot become the same computation, and manhattan is evaluated on the l2 path — rank-equivalent, but the absolute scores differ from what you would expect.

11

Where this lives in the dashboard.

The query workbench has no vector mode. The LANG switcher offers SQL, Cypher, NL and Search — and that is the whole list. A nearest-neighbour query needs a query embedding, which is not something you can usefully type into a text editor.

What the dashboard does own is the index: on Data → Schema you can train and install a vector index over a table that already holds vectors. Running the search itself is always an API or SDK call.

Vector collections do appear in the workbench's schema rail under a vector tables heading with their vector count, so you can confirm what has been indexed without leaving the page. See Run queries from the dashboard for the rest.

Related.