Core concepts

Full-text search

Rank documents with BM25, tune fuzzy matching and match thresholds, boost fields, and highlight matches.

Table of Contents

query(indexName, params) runs every search. Full-text mode is the default: it scores with BM25 and returns hits shaped as { id, score, document, highlights? }, together with results.count for the total match count and results.elapsed for the query time in milliseconds. This page builds one file, search.ts, step by step. You start with a basic query and refine it. Each step highlights exactly what changed.

Run a basic query

Start search.ts with an engine, a products index, and a few documents to search over. The query passes a term and returns the ranked hits, ordered by BM25.

search.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
})
 
await narsil.insertBatch('products', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const results = await narsil.query('products', {
  term: 'mechanical keyboard',
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

The keyboard wins on a two-term match in the title. The mouse and the wireless keyboard each match one term, so they tie at the same score and the engine orders the tie deterministically. limit defaults to 10, and you set offset to page through the rest. Pass includeScoreComponents: true to receive per-term frequencies, field lengths, and IDF values when you need to debug a ranking.

Restrict fields and boost matches

fields narrows the search to named fields, and boost multiplies per-field scores so a title match ranks above a body match. Add both to the query so a hit in the title counts for twice as much as one in the description.

search.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
})
 
await narsil.insertBatch('products', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const results = await narsil.query('products', {
  term: 'mechanical keyboard',
  fields: ['title', 'description'], 
  boost: { title: 2.0 },
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

With the boost in place, Mechanical Keyboard ranks above Gaming Mouse, whose only match is in the description.

Allow for typos

tolerance sets the maximum Levenshtein edit distance between a query term and an indexed term, so a mistyped term still matches. It defaults to 0, which requires exact matches. Loosen the query to tolerate two edits, and a mistyped search such as mechanical keybaord still finds both keyboards.

search.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
})
 
await narsil.insertBatch('products', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const results = await narsil.query('products', {
  term: 'mechanical keyboard',
  fields: ['title', 'description'],
  boost: { title: 2.0 },
  tolerance: 2, 
  prefixLength: 3,
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

prefixLength limits fuzzy candidates to terms that share that many leading characters with the query term. It defaults to 2, and raising it makes fuzzy lookups faster and stricter. Set exact: true to turn fuzzy expansion off for the whole query.

Require stronger matches

termMatch sets how many query terms a document must match: 'any' (the default) accepts one term, 'all' requires every term, and a number requires at least that many. minScore then removes any hit scoring below a floor. Tighten the query to demand at least two matching terms and a minimum score.

search.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
})
 
await narsil.insertBatch('products', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const results = await narsil.query('products', {
  term: 'mechanical keyboard',
  fields: ['title', 'description'],
  boost: { title: 2.0 },
  tolerance: 2,
  prefixLength: 3,
  termMatch: 2, 
  minScore: 1.5,
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

Highlight the matches

highlight returns snippets with tags marking where query terms appear. Highlighting requires trackPositions: true in the index config, which is the default. Add a highlight block so each hit carries a marked-up snippet for the fields you name.

search.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
})
 
await narsil.insertBatch('products', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const results = await narsil.query('products', {
  term: 'mechanical keyboard',
  fields: ['title', 'description'],
  boost: { title: 2.0 },
  tolerance: 2,
  prefixLength: 3,
  termMatch: 2,
  minScore: 1.5,
  highlight: { 
    fields: ['title', 'description'],
    preTag: '<mark>',
    postTag: '</mark>',
    maxSnippetLength: 160,
  },
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

Each hit now carries a highlights map keyed by field. The first hit's highlights.title.snippet reads <mark>Mechanical</mark> <mark>Keyboard</mark>, ready to render.

Choose a scoring mode

Three scoring modes handle the statistics-skew problem that appears when an index spans partitions or instances:

  • 'local' scores each partition with its own statistics. It is the fastest mode and the default.
  • 'dfs' runs a two-phase query that first collects global term statistics, then scores with unified IDF values.
  • 'broadcast' shares statistics across instances through the invalidation adapter, so scoring uses pre-computed global values.

Set a per-index default with defaultScoring in the index config, or set scoring per query. This is a variation on the same query rather than a further step, so run it on its own against the same products index.

const results = await narsil.query('products', {
  term: 'mechanical keyboard',
  scoring: 'dfs',
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

This products index has a single partition, so dfs collects the same term statistics local already used and the scores and order match the query above. The two-phase pass changes rankings only when an index spans several partitions or instances. There each partition would otherwise score with its own statistics, and dfs unifies the IDF values so a term that is rare across the whole corpus keeps its weight in every partition.

Search as you type

prefix: true treats the last word of the query as an unfinished word, so it also matches indexed terms that complete it, and a search box can show results while the user is still typing. Earlier words must match in full. tolerance keeps applying to them but not to the unfinished word, which is completed rather than typo-corrected. Completions rank below full-word matches, each query expands to at most fifty completions, and exact: true turns the option off. Despite the similar name, prefixLength belongs to fuzzy matching and is a separate control. This is a variation on the same query rather than a further step, so run it on its own.

const results = await narsil.query('products', {
  term: 'wireless keyb',
  prefix: true,
})
 
console.log(JSON.stringify(results, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

keyb completes to keyboard, so both keyboards match, and Wireless Keyboard ranks first because it also matches the full word wireless.

Size a query and suggest terms

preflight returns the match count for a query without materialising, ranking, or paginating hits, letting you size a result set before an expensive query. suggest returns autocomplete candidates that complete a prefix, ranked by how many documents contain each term. limit defaults to 10 and caps at 100. Both read the same products index without changing the search script, so run them on their own.

const preflight = await narsil.preflight('products', { term: 'keyboard' })
 
console.log(JSON.stringify(preflight, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

preflight returns the match count and the query time with no hits to rank or paginate. Now ask for the autocomplete candidates.

const suggestions = await narsil.suggest('products', { prefix: 'mech', limit: 5 })
 
console.log(JSON.stringify(suggestions, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))

The suggested term is the stem mechan, because suggestions come from the stemmed terms the index stores. Set surfaceForms: true in the index config to suggest the original spellings from your documents. The index then records every spelling it has indexed, which costs some insert throughput, and the setting persists with the index. When several spellings share a stem, the suggestion shows the most frequent one. Because surfaceForms is set when the index is created, this example builds a second products-surface index with the same documents.

surface-forms.ts
import { createNarsil } from '@delali/narsil'
 
const narsil = await createNarsil()
 
await narsil.createIndex('products-surface', {
  schema: {
    title: 'string',
    description: 'string',
    price: 'number',
    inStock: 'boolean',
  },
  language: 'english',
  surfaceForms: true, 
})
 
await narsil.insertBatch('products-surface', [
  { id: 'mx-keeb', title: 'Mechanical Keyboard', description: 'Cherry MX Brown switches with PBT keycaps.', price: 129.99, inStock: true },
  { id: 'wl-keeb', title: 'Wireless Keyboard', description: 'Low-profile keys and Bluetooth for a tidy desk.', price: 89.99, inStock: true },
  { id: 'gm-mouse', title: 'Gaming Mouse', description: 'A lightweight mouse tuned for fast mechanical clicks.', price: 59.99, inStock: false },
])
 
const suggestions = await narsil.suggest('products-surface', { prefix: 'mech', limit: 5 })
 
console.log(JSON.stringify(suggestions, (k, v) => (typeof v === 'number' ? Number(v.toFixed(2)) : v), 2))