reference · full-text

Full-text search

Full-text search finds rows by keywords in their text. Use it for "find products matching wireless headphones", search-as-you-type, log line filtering, and anywhere a user is typing words instead of structured filters. The ranking algorithm is BM25 - the same one Elasticsearch, Lucene, and most search engines use.

Full-text indexes live on their own runtime endpoint - they are not declared on the schema. You index a text under (table, field, doc_id), then search. The doc_id is what links search hits back to your rows - use the row's primary key.

:table and :field are opaque path segments - they are not validated against any schema. They must match exactly between the index call and the query call. Index under shop.products and query shop_products and you get a silent empty result, not an error. The default search mode is boolean - omit mode and you get AND-of-terms.

1. Index a text.

what this does

Tell OriginChain to make a piece of text searchable. Re-indexing the same doc_id replaces the old text in the same write - no stale matches.

POST /v1/tenants/:t/fts/:table/:field

curl -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "doc_id": "sku-9281",
    "text":   "Lightweight road runner with a carbon plate, designed for marathon pace."
  }'

db.fts.index(
    "shop.products",
    "description",
    doc_id="sku-9281",
    text="Lightweight road runner with a carbon plate, designed for marathon pace.",
)

await db.ftsIndex("shop.products", "description", {
  doc_id: "sku-9281",
  text:   "Lightweight road runner with a carbon plate, designed for marathon pace.",
});

err := db.FTSIndex(ctx, "shop.products", "description", originchain.FTSIndexRequest{
    DocID: "sku-9281",
    Text:  "Lightweight road runner with a carbon plate, designed for marathon pace.",
})

what you get back

HTTP/1.1 201 Created

(empty body)

A successful index returns 201 Created with no body. There is no token count or confirmation JSON - check the status code, not the body.

what each field means

Field	Where	What it is
:table	URL	An opaque label for the index - conventionally your row table, e.g. `shop.products`. Not checked against any schema; must match byte-for-byte at query time.
:field	URL	Which "logical column" you're indexing under. You can index multiple fields per table - `title`, `description`, etc.
doc_id	body	A unique ID for this document. Use the row's primary key so hits link back cleanly.
text	body	The text to search. No size limit on this endpoint, but very large documents are better split into multiple `doc_id`s.

common mistakes

Indexing only one of several fields. If you want users to search "Wireless headphones" and match products whose title or description contains those words, you need to either concatenate both fields into one text before indexing, or index each field separately and union the results.
Forgetting to re-index on updates. Editing a row's text does not automatically update the FTS index. Re-call this endpoint with the new text whenever you change the source.

2. BM25 - ranked search.

what this does

Return the top-k documents ranked by relevance to the query. This is what most users mean when they say "search". Rare query words count more than common ones; documents where the query words appear more often (relative to length) rank higher.

GET /v1/tenants/:t/fts/:table/:field?mode=bm25

curl "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description?q=carbon+marathon&mode=bm25&k=10" \
  -H "Authorization: Bearer $OC_TOKEN"

hits = db.fts.search(
    "shop.products",
    "description",
    q="carbon marathon",
    mode="bm25",
    k=10,
)
for hit in hits:
    print(hit.score, hit.doc_id)

const hits = await db.ftsSearch("shop.products", "description", {
  q:    "carbon marathon",
  mode: "bm25",
  k:    10,
});
for (const hit of hits) {
  if (typeof hit === "object") console.log(hit.score, hit.doc_id);
}

hits, err := db.FTSSearch(ctx, "shop.products", "description", originchain.FTSSearchRequest{
    Q:    "carbon marathon",
    Mode: "bm25",
    K:    10,
})
if err != nil { /* handle */ }
for _, h := range hits {
    fmt.Println(h.Score, h.DocID)
}

what you get back

[
  { "doc_id": "sku-9281", "score": 9.42 },
  { "doc_id": "sku-3140", "score": 4.18 }
]

Plain BM25 returns a bare array of { doc_id, score }, best first - no { "mode": ..., "hits": [...] } wrapper. The wrapper object only appears when you add highlight=true or facets= (then you get { "hits": [...], "facets": {...} }); explain=true returns a separate scoring-breakdown object.

query params — all BM25-only; ignored in boolean / phrase

Param	Required	Notes
q	yes	The query text. URL-encode spaces as `+` or `%20`.
mode	no	`bm25` \| `boolean` \| `phrase`. Defaults to `boolean` when omitted. Use `bm25` for this section.
k	no	Max results. Default 10. BM25 only.
fuzzy	no	Edit distance for typo tolerance. `fuzzy=1` matches one-character typos.
highlight	no	`highlight=true` returns matched-term snippets per hit.
facets	no	Comma-separated field names to aggregate as facet counts. Switches the response to the wrapper-object shape.
explain	no	`explain=true` returns a BM25 scoring-breakdown object instead of hits.

highlight=true requires the doc text to have been stored first via POST /fts/:t/:f/doc. All of k, fuzzy, highlight, facets, and explain are silently ignored in boolean and phrase modes.

3. Boolean AND - every word must match.

what this does

Return every document that contains all the query words, in any order, with no ranking. Fast token-presence check - use when you don't need relevance scoring.

GET /v1/tenants/:t/fts/:table/:field?mode=boolean

curl "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description?q=carbon+marathon&mode=boolean" \
  -H "Authorization: Bearer $OC_TOKEN"

hits = db.fts.search(
    "shop.products",
    "description",
    q="carbon marathon",
    mode="boolean",     # every word must appear
)
for hit in hits:
    print(hit.doc_id)

const hits = await db.ftsSearch("shop.products", "description", {
  q:    "carbon marathon",
  mode: "boolean",
});

hits, err := db.FTSSearch(ctx, "shop.products", "description", originchain.FTSSearchRequest{
    Q:    "carbon marathon",
    Mode: "boolean",
})

what you get back

["sku-9281", "sku-3140"]

A bare array of doc_id strings (sorted lexicographically) - no score field and no wrapper object. If you need ordering by relevance, use BM25.

4. Phrase - exact word order.

what this does

Match documents that contain the query words contiguously, in the exact order given. Use for branded phrases ("New York Times"), product model numbers, log message templates.

GET /v1/tenants/:t/fts/:table/:field?mode=phrase

curl "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description?q=carbon+plate&mode=phrase" \
  -H "Authorization: Bearer $OC_TOKEN"

hits = db.fts.search(
    "shop.products",
    "description",
    q="carbon plate",
    mode="phrase",      # exact phrase "carbon plate", in that order
)

const hits = await db.ftsSearch("shop.products", "description", {
  q:    "carbon plate",
  mode: "phrase",
});

hits, err := db.FTSSearch(ctx, "shop.products", "description", originchain.FTSSearchRequest{
    Q:    "carbon plate",
    Mode: "phrase",
})

what you get back

["sku-9281"]

Same shape as boolean - a bare array of doc_id strings. Phrase just narrows which docs qualify.

5. End-to-end - index, search, enrich.

A complete runnable sequence against a live tenant. Set $OC_HOST, $OC_TENANT, and $OC_TOKEN first. Every response shown below is the real shape the engine returns.

step 1 — index three docs

# 1) Index three docs. Each POST returns 201 with an EMPTY body.
curl -sS -o /dev/null -w "%{http_code}\n" \
  -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" -H "Content-Type: application/json" \
  -d '{ "doc_id": "p001", "text": "Wireless over-ear headphones with active noise cancellation" }'
# → 201

curl -sS -o /dev/null -w "%{http_code}\n" \
  -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" -H "Content-Type: application/json" \
  -d '{ "doc_id": "p002", "text": "Wired earbuds, no noise cancellation" }'
# → 201

curl -sS -o /dev/null -w "%{http_code}\n" \
  -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" -H "Content-Type: application/json" \
  -d '{ "doc_id": "p003", "text": "USB-C charging cable, 2 metres" }'
# → 201

step 2 — boolean query (default mode)

# 2) Boolean query (the DEFAULT mode). Bare array of doc_id strings.
curl -sS -G "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  --data-urlencode "q=wireless noise"
# → ["p001"]      (only p001 has BOTH "wireless" AND "noise")

step 3 — bm25 ranked query

# 3) BM25 ranked query. Bare array of { doc_id, score }, best first.
curl -sS -G "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  --data-urlencode "q=noise cancellation" \
  --data-urlencode "mode=bm25" \
  --data-urlencode "k=10"
# → [ { "doc_id": "p001", "score": 6.31 }, { "doc_id": "p002", "score": 2.04 } ]

step 4 — enrich with highlights

# 4) Enrich with highlights. First store the doc text (highlights read it),
#    THEN ask for highlight=true. Now the response is an OBJECT, not an array.
curl -sS -o /dev/null -w "%{http_code}\n" \
  -X POST "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description/doc" \
  -H "Authorization: Bearer $OC_TOKEN" -H "Content-Type: application/json" \
  -d '{ "doc_id": "p001", "text": "Wireless over-ear headphones with active noise cancellation" }'
# → 201

curl -sS -G "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  --data-urlencode "q=noise cancellation" \
  --data-urlencode "mode=bm25" \
  --data-urlencode "highlight=true"
# → {
#     "hits": [
#       { "doc_id": "p001", "score": 6.31,
#         "highlights": { "description": ["…active <em>noise</em> <em>cancellation</em>"] } }
#     ]
#   }

6. Analyzer + languages.

read this first

Stemming, lemmatization, diacritic folding, and stopword removal are implemented in the engine but not yet selectable through the HTTP API. There is no language or analyzer query parameter today. The analyzer the API actually uses is Unicode tokenize + lowercase, and nothing else. Concretely: q=runs will not match a document that says "running", and q=cafe will not match "café". Plan your indexing around exact tokens. The one transform that is live is synonyms (see below).

What runs today

Unicode tokenize. Text is split into words by the Unicode word-boundary rules (UAX #29), so it works across scripts.
Lowercase. Every token is lowercased, so Wireless and wireless match. This is the only normalisation applied.
Synonyms. If you install a per-(table, field) synonym map via POST /fts/:t/:f/synonyms, members of a class are treated as equivalent at both index time and BM25 query time. This is the one customer-controlled analyzer feature that is wired through. See FTS runtime calls.

Built in the engine, not yet exposed (roadmap)

The following analyzer stages exist in the engine but cannot be turned on from the API yet. They are listed so you know what is coming, not what you can call today.

Step	What it would do	Status
fold diacritics	"café" would match "cafe".	not API-exposed
stopwords	Drop common words ("the", "and", "of").	not API-exposed
stemming	Suffix-strip. "running" / "runs" → "run". Snowball-based.	not API-exposed
lemmatization	Dictionary lookup. "ran" → "run". More precise than stemming.	not API-exposed

languages the engine has stemmers for (not yet selectable)

ArabicDanishDutchEnglishFinnishFrenchGermanHungarianItalianNorwegianPortugueseRomanianRussianSpanishSwedishTamilTurkishHindi

There is no per-field analyzer knob you can set today. The runtime calls that are live - plain / JSON-aware index, doc store, synonyms, stopword override - are documented in FTS runtime calls.