Narsil is a search engine that answers each query inside the process that sends it, so you provision no server. You build one file here, search.ts, in which you create a typed index, fill it with a bicycle shop's catalogue, and read a ranked answer. Each code block below holds the whole file, with green lines marking the lines that you add at each step.
Install the package
pnpm add -E @delali/narsilAlong with its own TypeScript types, the package declares a Node.js build and a browser build under one entry point in its package.json, so Node.js and a browser bundler each resolve the same import to their own build.
Create an engine and an index
createNarsil() returns a promise for the engine, which keeps every index in memory when you pass no argument. The schema that you declare when you create an index fixes the type of every field, the fields that a filter may name, and the way the engine tokenises your text.
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
inStock: 'boolean',
},
language: 'english',
})Field types cover string, string:sortable, number, boolean, enum, and geopoint, the array forms string[], number[], boolean[], and enum[], and vector[N] for an of N numbers. A query may sort on a text field only where the schema declares that field string:sortable, because ordering text needs far more memory for each document than ordering a number.
The language option, which defaults to english, picks the analyser that tokenises and every text field. That analyser is what makes a search for bicycles return a document that holds the word bicycle, since it stems both words to one term.
Fill the index with documents
insert stores one document and returns the id under which it stored that document, while insertBatch takes an array and reports on each document in it. The batch call returns { succeeded, failed }, where succeeded lists the ids that the engine stored and each entry in failed names one document, an error code, and a message. The engine therefore stores the rest of a batch around one rejected document. When a batch of three gives price the string 'nine hundred' in its middle document, for example, the engine stores the other two and reports DOC_VALIDATION_FAILED against that one.
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
inStock: 'boolean',
},
language: 'english',
})
await narsil.insertBatch('bicycles', [
{
id: 'urban-commuter',
name: 'Gomes Urban commuter',
summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes and mudguards fitted as standard.',
maker: 'Gomes Cycles',
category: 'city',
price: 749,
inStock: true,
},
{
id: 'folding-metro',
name: 'Obi Metro folding bicycle',
summary: 'A folding bicycle that fits under a train seat and unfolds in fifteen seconds.',
maker: 'Obi Mobility',
category: 'folding',
price: 899,
inStock: true,
},
{
id: 'folding-tourer',
name: 'Khoury Tourer folding bicycle',
summary: 'A folding bicycle with a rear rack and eight gears, sold with a travel bag.',
maker: 'Khoury Composites',
category: 'folding',
price: 1150,
inStock: false,
},
{
id: 'gravel-explorer',
name: 'Mori Gravel explorer',
summary: 'A steel gravel bicycle built for long unpaved rides, with clearance for 45 millimetre tyres.',
maker: 'Mori Works',
category: 'gravel',
price: 1299,
inStock: true,
},
{
id: 'cargo-hauler',
name: 'Larsson Cargo hauler',
summary: 'An electric cargo bicycle rated to carry two children and a week of shopping.',
maker: 'Larsson Bikes',
category: 'cargo',
price: 3499,
inStock: true,
},
])Ask the index for a match
query takes the term and returns the hits in ranked order. document.include names the fields that come back with each hit, while the engine stops at the limit of three hits, or at 10 where a query leaves limit out.
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
inStock: 'boolean',
},
language: 'english',
})
await narsil.insertBatch('bicycles', [
{
id: 'urban-commuter',
name: 'Gomes Urban commuter',
summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes and mudguards fitted as standard.',
maker: 'Gomes Cycles',
category: 'city',
price: 749,
inStock: true,
},
{
id: 'folding-metro',
name: 'Obi Metro folding bicycle',
summary: 'A folding bicycle that fits under a train seat and unfolds in fifteen seconds.',
maker: 'Obi Mobility',
category: 'folding',
price: 899,
inStock: true,
},
{
id: 'folding-tourer',
name: 'Khoury Tourer folding bicycle',
summary: 'A folding bicycle with a rear rack and eight gears, sold with a travel bag.',
maker: 'Khoury Composites',
category: 'folding',
price: 1150,
inStock: false,
},
{
id: 'gravel-explorer',
name: 'Mori Gravel explorer',
summary: 'A steel gravel bicycle built for long unpaved rides, with clearance for 45 millimetre tyres.',
maker: 'Mori Works',
category: 'gravel',
price: 1299,
inStock: true,
},
{
id: 'cargo-hauler',
name: 'Larsson Cargo hauler',
summary: 'An electric cargo bicycle rated to carry two children and a week of shopping.',
maker: 'Larsson Bikes',
category: 'cargo',
price: 3499,
inStock: true,
},
])
const results = await narsil.query('bicycles', {
term: 'folding bicycle',
document: { include: ['name', 'maker', 'price'] },
limit: 3,
})
console.log(
JSON.stringify(
results,
(key, value) => (key === 'elapsed' ? undefined : typeof value === 'number' ? Number(value.toFixed(2)) : value),
2
)
)
await narsil.shutdown(){
"hits": [
{
"id": "folding-metro",
"score": 1.92,
"document": {
"name": "Obi Metro folding bicycle",
"maker": "Obi Mobility",
"price": 899
}
},
{
"id": "folding-tourer",
"score": 1.88,
"document": {
"name": "Khoury Tourer folding bicycle",
"maker": "Khoury Composites",
"price": 1150
}
},
{
"id": "cargo-hauler",
"score": 0.09,
"document": {
"name": "Larsson Cargo hauler",
"maker": "Larsson Bikes",
"price": 3499
}
}
],
"count": 5,
"countExact": true,
"cursor": "eyJ2Ijo0LCJhIjoiY2FyZ28taGF1bGVyIiwicyI6MC4wODg1NTI5MTcxMzMxNDY3NCwiZCI6MywicSI6ImEwY2I4OTcyIn0=",
"coverage": {
"totalPartitions": 1,
"queriedPartitions": 1,
"timedOutPartitions": 0,
"failedPartitions": 0
}
}All five bicycles match, because the engine admits a document that holds either word until you ask for both, and every summary here holds the word bicycle. The two folding bicycles score above 1.8 while the cargo bicycle scores 0.09, since counts a term for more where fewer documents hold it, and two of these five hold the word folding.
Next to the hits, count is the total that matched across the index, countExact is true where the engine counted every one of them, and coverage holds how many partitions the index has, how many answered, and how many timed out or failed, which here comes to the one partition that this index opens with. Two more documents match beyond these three hits, so the engine returns a cursor, which a later query passes as searchAfter to reach them. The engine also returns elapsed, the milliseconds that it spent, which the replacer above drops so that this page prints the same panel on every run.
Narrow the search and mark the matched words
Four options change that answer. With termMatch: 'all' the engine keeps a document only where that document holds every term, with filters.fields it tests one field against a value, with boost it multiplies what one field contributes to the score, and with highlight it wraps the matched words in a tag, which is <mark> until you set preTag and postTag.
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
inStock: 'boolean',
},
language: 'english',
})
await narsil.insertBatch('bicycles', [
{
id: 'urban-commuter',
name: 'Gomes Urban commuter',
summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes and mudguards fitted as standard.',
maker: 'Gomes Cycles',
category: 'city',
price: 749,
inStock: true,
},
{
id: 'folding-metro',
name: 'Obi Metro folding bicycle',
summary: 'A folding bicycle that fits under a train seat and unfolds in fifteen seconds.',
maker: 'Obi Mobility',
category: 'folding',
price: 899,
inStock: true,
},
{
id: 'folding-tourer',
name: 'Khoury Tourer folding bicycle',
summary: 'A folding bicycle with a rear rack and eight gears, sold with a travel bag.',
maker: 'Khoury Composites',
category: 'folding',
price: 1150,
inStock: false,
},
{
id: 'gravel-explorer',
name: 'Mori Gravel explorer',
summary: 'A steel gravel bicycle built for long unpaved rides, with clearance for 45 millimetre tyres.',
maker: 'Mori Works',
category: 'gravel',
price: 1299,
inStock: true,
},
{
id: 'cargo-hauler',
name: 'Larsson Cargo hauler',
summary: 'An electric cargo bicycle rated to carry two children and a week of shopping.',
maker: 'Larsson Bikes',
category: 'cargo',
price: 3499,
inStock: true,
},
])
const results = await narsil.query('bicycles', {
term: 'folding bicycle',
termMatch: 'all',
filters: { fields: { inStock: { eq: true }, price: { lte: 1500 } } },
boost: { name: 3 },
highlight: { fields: ['name'] },
document: { include: ['name', 'maker', 'price'] },
limit: 3,
})
console.log(
JSON.stringify(
results,
(key, value) => (key === 'elapsed' ? undefined : typeof value === 'number' ? Number(value.toFixed(2)) : value),
2
)
)
await narsil.shutdown(){
"hits": [
{
"id": "folding-metro",
"score": 3.72,
"document": {
"name": "Obi Metro folding bicycle",
"maker": "Obi Mobility",
"price": 899
},
"highlights": {
"name": {
"snippet": "Obi Metro <mark>folding</mark> <mark>bicycle</mark>",
"positions": [
{
"start": 10,
"end": 17
},
{
"start": 18,
"end": 25
}
]
}
}
}
],
"count": 1,
"countExact": true,
"coverage": {
"totalPartitions": 1,
"queriedPartitions": 1,
"timedOutPartitions": 0,
"failedPartitions": 0
}
}Both folding bicycles hold the two words, while the Khoury tourer fails the inStock test, so one bicycle comes back. The filters and termMatch change which documents match and leave every score alone, so the boost of 3 on name is what moves this document from 1.92 to 3.72. highlights.name.snippet is ready to render as it stands, while positions holds the start and end offset of each match in the stored text, so you can mark those spans in your own markup.
The search box on this site queries Narsil. One route builds an index over every article, project, study, and documentation page, then answers each query in the same process that serves the page.
Upgrade from Narsil 0.2
Narsil 0.3 saves indexes in a new format, so it can't open the indexes that version 0.2 saved. If your application saved its indexes to a directory under 0.2, createNarsil fails with PERSISTENCE_LOAD_FAILED when you start 0.3 on that directory. restore also refuses a snapshot that you took under 0.2, failing with ENVELOPE_INVALID_MAGIC.
To upgrade, start 0.3 on a new, empty directory and insert your documents again from the database or the files that they came from. Once the new indexes return the results that you expect, you can delete the old directory. If your code imports @delali/narsil/vector/hnsw-build-worker, delete that import, because the 0.3 package leaves that export out.
Where to go next
- Indexes and documents covers the schema in full, nested fields, and the calls that read, update, and remove a document.
- Full-text search covers the ranking, the typo tolerance, the prefix completion, and the suggestions.
- Vector search covers the search by meaning, which answers a question that shares no word with the article that holds its answer.