With a filter, the engine keeps only the documents whose fields pass a test. With a facet, it counts how many of the matching documents hold each value of a field, which is the number that a shop's filter sidebar shows beside each option. You build one file here, search.ts, on a catalogue of twelve bicycles, and change its query step by step. Each code block below holds the whole file, with green lines marking the lines that you add at each step and struck lines marking the lines that you remove.
Search the whole catalogue
Every summary in the catalogue holds the word bicycle, so a search for it matches all twelve. The program prints each hit's price and tags, so that you can see which hits each filter below removes.
search.ts
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
rating: 'number',
inStock: 'boolean',
tags: 'string[]',
},
})
await narsil.insertBatch('bicycles', [
{ id: 'urban-commuter', name: 'Gomes Urban commuter', summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes.', maker: 'Gomes Cycles', category: 'city', price: 749, rating: 4.5, inStock: true, tags: ['commuter', 'aluminium'] },
{ id: 'city-step', name: 'Gomes Step-through city bicycle', summary: 'A step-through city bicycle with a chain guard and a hub dynamo.', maker: 'Gomes Cycles', category: 'city', price: 549, rating: 4.0, inStock: true, tags: ['commuter', 'steel'] },
{ id: 'gravel-explorer', name: 'Mori Gravel explorer', summary: 'A steel gravel bicycle built for long unpaved rides.', maker: 'Mori Works', category: 'gravel', price: 1299, rating: 4.8, inStock: true, tags: ['gravel', 'steel'] },
{ id: 'gravel-allroad', name: 'Mori Allroad gravel bicycle', summary: 'An aluminium gravel bicycle with mounts for three bottles.', maker: 'Mori Works', category: 'gravel', price: 999, rating: 4.4, inStock: false, tags: ['gravel', 'aluminium'] },
{ id: 'folding-metro', name: 'Obi Metro folding bicycle', summary: 'A folding bicycle that fits under a train seat.', maker: 'Obi Mobility', category: 'folding', price: 899, rating: 4.2, inStock: true, tags: ['folding', 'commuter'] },
{ id: 'folding-tourer', name: 'Obi Tourer folding bicycle', summary: 'A folding bicycle with a rear rack and eight gears.', maker: 'Obi Mobility', category: 'folding', price: 1150, rating: 4.1, inStock: true, tags: ['folding', 'aluminium'] },
{ id: 'cargo-hauler', name: 'Larsson Cargo hauler', summary: 'An electric cargo bicycle rated to carry two children.', maker: 'Larsson Bikes', category: 'cargo', price: 3499, rating: 4.6, inStock: true, tags: ['cargo', 'electric'] },
{ id: 'cargo-trike', name: 'Larsson Cargo trike', summary: 'A three-wheeled cargo bicycle with a lockable box.', maker: 'Larsson Bikes', category: 'cargo', price: 4195, rating: 4.3, inStock: false, tags: ['cargo', 'electric'] },
{ id: 'road-racer', name: 'Khoury Road racer', summary: 'A carbon road bicycle with electronic shifting.', maker: 'Khoury Composites', category: 'road', price: 4250, rating: 4.9, inStock: true, tags: ['road', 'carbon'] },
{ id: 'road-endurance', name: 'Khoury Endurance road bicycle', summary: 'A carbon road bicycle tuned for comfort over long distances.', maker: 'Khoury Composites', category: 'road', price: 2799, rating: 4.7, inStock: true, tags: ['road', 'carbon'] },
{ id: 'kids-first', name: 'Adams First ride bicycle', summary: 'A first bicycle for a child, with stabilisers that come off in one minute.', maker: 'Adams Assembly', category: 'kids', price: 189, rating: 4.3, inStock: true, tags: ['kids'] },
{ id: 'kids-junior', name: 'Adams Junior mountain bicycle', summary: 'A junior mountain bicycle with a suspension fork and six gears.', maker: 'Adams Assembly', category: 'kids', price: 329, rating: 4.1, inStock: true, tags: ['kids', 'aluminium'] },
])
const results = await narsil.query('bicycles', {
term: 'bicycle',
document: { include: ['price', 'tags'] },
})
console.log(JSON.stringify({ count: results.count, hits: results.hits.map((hit) => ({ id: hit.id, ...hit.document })) }, null, 2))
await narsil.shutdown(){
"count": 12,
"hits": [
{
"id": "folding-metro",
"price": 899,
"tags": [
"folding",
"commuter"
]
},
{
"id": "folding-tourer",
"price": 1150,
"tags": [
"folding",
"aluminium"
]
},
{
"id": "gravel-allroad",
"price": 999,
"tags": [
"gravel",
"aluminium"
]
},
{
"id": "city-step",
"price": 549,
"tags": [
"commuter",
"steel"
]
},
{
"id": "kids-first",
"price": 189,
"tags": [
"kids"
]
},
{
"id": "kids-junior",
"price": 329,
"tags": [
"kids",
"aluminium"
]
},
{
"id": "road-endurance",
"price": 2799,
"tags": [
"road",
"carbon"
]
},
{
"id": "road-racer",
"price": 4250,
"tags": [
"road",
"carbon"
]
},
{
"id": "cargo-trike",
"price": 4195,
"tags": [
"cargo",
"electric"
]
},
{
"id": "cargo-hauler",
"price": 3499,
"tags": [
"cargo",
"electric"
]
}
]
}count reports all twelve matches while the engine returns ten hits, because the engine returns 10 hits for a query that leaves limit out.
Keep the documents whose fields pass a test
filters.fields maps each field to its tests, and a document has to pass every test to stay in the results. Every field takes eq and ne, and a number also takes gt, gte, lt, lte, and between. A string field takes in, nin, startsWith, and endsWith, and an enum field takes in and nin as well. An array field takes matchesAny, containsAll, and size. Any field takes exists and notExists, while isEmpty and isNotEmpty test a string or an array for content.
search.ts
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
rating: 'number',
inStock: 'boolean',
tags: 'string[]',
},
})
await narsil.insertBatch('bicycles', [
{ id: 'urban-commuter', name: 'Gomes Urban commuter', summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes.', maker: 'Gomes Cycles', category: 'city', price: 749, rating: 4.5, inStock: true, tags: ['commuter', 'aluminium'] },
{ id: 'city-step', name: 'Gomes Step-through city bicycle', summary: 'A step-through city bicycle with a chain guard and a hub dynamo.', maker: 'Gomes Cycles', category: 'city', price: 549, rating: 4.0, inStock: true, tags: ['commuter', 'steel'] },
{ id: 'gravel-explorer', name: 'Mori Gravel explorer', summary: 'A steel gravel bicycle built for long unpaved rides.', maker: 'Mori Works', category: 'gravel', price: 1299, rating: 4.8, inStock: true, tags: ['gravel', 'steel'] },
{ id: 'gravel-allroad', name: 'Mori Allroad gravel bicycle', summary: 'An aluminium gravel bicycle with mounts for three bottles.', maker: 'Mori Works', category: 'gravel', price: 999, rating: 4.4, inStock: false, tags: ['gravel', 'aluminium'] },
{ id: 'folding-metro', name: 'Obi Metro folding bicycle', summary: 'A folding bicycle that fits under a train seat.', maker: 'Obi Mobility', category: 'folding', price: 899, rating: 4.2, inStock: true, tags: ['folding', 'commuter'] },
{ id: 'folding-tourer', name: 'Obi Tourer folding bicycle', summary: 'A folding bicycle with a rear rack and eight gears.', maker: 'Obi Mobility', category: 'folding', price: 1150, rating: 4.1, inStock: true, tags: ['folding', 'aluminium'] },
{ id: 'cargo-hauler', name: 'Larsson Cargo hauler', summary: 'An electric cargo bicycle rated to carry two children.', maker: 'Larsson Bikes', category: 'cargo', price: 3499, rating: 4.6, inStock: true, tags: ['cargo', 'electric'] },
{ id: 'cargo-trike', name: 'Larsson Cargo trike', summary: 'A three-wheeled cargo bicycle with a lockable box.', maker: 'Larsson Bikes', category: 'cargo', price: 4195, rating: 4.3, inStock: false, tags: ['cargo', 'electric'] },
{ id: 'road-racer', name: 'Khoury Road racer', summary: 'A carbon road bicycle with electronic shifting.', maker: 'Khoury Composites', category: 'road', price: 4250, rating: 4.9, inStock: true, tags: ['road', 'carbon'] },
{ id: 'road-endurance', name: 'Khoury Endurance road bicycle', summary: 'A carbon road bicycle tuned for comfort over long distances.', maker: 'Khoury Composites', category: 'road', price: 2799, rating: 4.7, inStock: true, tags: ['road', 'carbon'] },
{ id: 'kids-first', name: 'Adams First ride bicycle', summary: 'A first bicycle for a child, with stabilisers that come off in one minute.', maker: 'Adams Assembly', category: 'kids', price: 189, rating: 4.3, inStock: true, tags: ['kids'] },
{ id: 'kids-junior', name: 'Adams Junior mountain bicycle', summary: 'A junior mountain bicycle with a suspension fork and six gears.', maker: 'Adams Assembly', category: 'kids', price: 329, rating: 4.1, inStock: true, tags: ['kids', 'aluminium'] },
])
const results = await narsil.query('bicycles', {
term: 'bicycle',
filters: {
fields: {
inStock: { eq: true },
price: { between: [500, 1500] },
tags: { matchesAny: ['commuter', 'gravel'] },
},
},
document: { include: ['price', 'tags'] },
})
console.log(JSON.stringify({ count: results.count, hits: results.hits.map((hit) => ({ id: hit.id, ...hit.document })) }, null, 2))
await narsil.shutdown(){
"count": 4,
"hits": [
{
"id": "folding-metro",
"price": 899,
"tags": [
"folding",
"commuter"
]
},
{
"id": "city-step",
"price": 549,
"tags": [
"commuter",
"steel"
]
},
{
"id": "gravel-explorer",
"price": 1299,
"tags": [
"gravel",
"steel"
]
},
{
"id": "urban-commuter",
"price": 749,
"tags": [
"commuter",
"aluminium"
]
}
]
}Four bicycles pass all three tests. The allroad gravel bicycle fails inStock, and the folding tourer holds neither commuter nor gravel in its tags. Every other bicycle is priced outside the range. startsWith treats upper and lower case as different letters, so 'Gomes' matches the two Gomes bicycles while 'gomes' matches none. A nested field takes a dotted path such as 'maker.country'.
The engine fails with SEARCH_INVALID_FILTER for an operator that it does not know, for a string operator such as startsWith on a number field, and for a between with one bound. For a test on a field outside the schema, the engine checks the values that your documents store under that name, so a misspelt field name matches nothing and raises no error. On an index created with strict: true, however, the engine throws SEARCH_INVALID_FIELD for that test, as the errors page describes.
Combine tests with and, or, and not
and, or, and not each take whole filter expressions, so one expression can hold another. Here the engine keeps the bicycles in stock that are gravel or road bicycles or that hold a rating of 4.5 or more, and then drops every bicycle that Khoury makes.
search.ts
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
rating: 'number',
inStock: 'boolean',
tags: 'string[]',
},
})
await narsil.insertBatch('bicycles', [
{ id: 'urban-commuter', name: 'Gomes Urban commuter', summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes.', maker: 'Gomes Cycles', category: 'city', price: 749, rating: 4.5, inStock: true, tags: ['commuter', 'aluminium'] },
{ id: 'city-step', name: 'Gomes Step-through city bicycle', summary: 'A step-through city bicycle with a chain guard and a hub dynamo.', maker: 'Gomes Cycles', category: 'city', price: 549, rating: 4.0, inStock: true, tags: ['commuter', 'steel'] },
{ id: 'gravel-explorer', name: 'Mori Gravel explorer', summary: 'A steel gravel bicycle built for long unpaved rides.', maker: 'Mori Works', category: 'gravel', price: 1299, rating: 4.8, inStock: true, tags: ['gravel', 'steel'] },
{ id: 'gravel-allroad', name: 'Mori Allroad gravel bicycle', summary: 'An aluminium gravel bicycle with mounts for three bottles.', maker: 'Mori Works', category: 'gravel', price: 999, rating: 4.4, inStock: false, tags: ['gravel', 'aluminium'] },
{ id: 'folding-metro', name: 'Obi Metro folding bicycle', summary: 'A folding bicycle that fits under a train seat.', maker: 'Obi Mobility', category: 'folding', price: 899, rating: 4.2, inStock: true, tags: ['folding', 'commuter'] },
{ id: 'folding-tourer', name: 'Obi Tourer folding bicycle', summary: 'A folding bicycle with a rear rack and eight gears.', maker: 'Obi Mobility', category: 'folding', price: 1150, rating: 4.1, inStock: true, tags: ['folding', 'aluminium'] },
{ id: 'cargo-hauler', name: 'Larsson Cargo hauler', summary: 'An electric cargo bicycle rated to carry two children.', maker: 'Larsson Bikes', category: 'cargo', price: 3499, rating: 4.6, inStock: true, tags: ['cargo', 'electric'] },
{ id: 'cargo-trike', name: 'Larsson Cargo trike', summary: 'A three-wheeled cargo bicycle with a lockable box.', maker: 'Larsson Bikes', category: 'cargo', price: 4195, rating: 4.3, inStock: false, tags: ['cargo', 'electric'] },
{ id: 'road-racer', name: 'Khoury Road racer', summary: 'A carbon road bicycle with electronic shifting.', maker: 'Khoury Composites', category: 'road', price: 4250, rating: 4.9, inStock: true, tags: ['road', 'carbon'] },
{ id: 'road-endurance', name: 'Khoury Endurance road bicycle', summary: 'A carbon road bicycle tuned for comfort over long distances.', maker: 'Khoury Composites', category: 'road', price: 2799, rating: 4.7, inStock: true, tags: ['road', 'carbon'] },
{ id: 'kids-first', name: 'Adams First ride bicycle', summary: 'A first bicycle for a child, with stabilisers that come off in one minute.', maker: 'Adams Assembly', category: 'kids', price: 189, rating: 4.3, inStock: true, tags: ['kids'] },
{ id: 'kids-junior', name: 'Adams Junior mountain bicycle', summary: 'A junior mountain bicycle with a suspension fork and six gears.', maker: 'Adams Assembly', category: 'kids', price: 329, rating: 4.1, inStock: true, tags: ['kids', 'aluminium'] },
])
const results = await narsil.query('bicycles', {
term: 'bicycle',
filters: {
fields: {
inStock: { eq: true },
price: { between: [500, 1500] },
tags: { matchesAny: ['commuter', 'gravel'] },
},
and: [
{ fields: { inStock: { eq: true } } },
{ or: [{ fields: { category: { in: ['gravel', 'road'] } } }, { fields: { rating: { gte: 4.5 } } }] },
{ not: { fields: { maker: { startsWith: 'Khoury' } } } },
],
},
document: { include: ['price', 'tags'] },
document: { include: ['maker', 'category', 'rating'] },
})
console.log(JSON.stringify({ count: results.count, hits: results.hits.map((hit) => ({ id: hit.id, ...hit.document })) }, null, 2))
await narsil.shutdown(){
"count": 3,
"hits": [
{
"id": "cargo-hauler",
"maker": "Larsson Bikes",
"category": "cargo",
"rating": 4.6
},
{
"id": "gravel-explorer",
"maker": "Mori Works",
"category": "gravel",
"rating": 4.8
},
{
"id": "urban-commuter",
"maker": "Gomes Cycles",
"category": "city",
"rating": 4.5
}
]
}Three bicycles pass. The gravel explorer passes on its category, while the cargo hauler and the urban commuter pass on their ratings of 4.6 and 4.5. Both Khoury road bicycles pass the or on their category, and the engine then removes them under the not.
Count each value for a filter sidebar
facets names the fields to count. The engine counts every document that matches the term and the filters, whatever limit you set, so a query with limit: 0 returns the counts alone. A number field takes ranges in place of single values, and a text or enum field takes a limit and a sort.
search.ts
import { createNarsil } from '@delali/narsil'
const narsil = await createNarsil()
await narsil.createIndex('bicycles', {
schema: {
name: 'string',
summary: 'string',
maker: 'string',
category: 'enum',
price: 'number',
rating: 'number',
inStock: 'boolean',
tags: 'string[]',
},
})
await narsil.insertBatch('bicycles', [
{ id: 'urban-commuter', name: 'Gomes Urban commuter', summary: 'A lightweight aluminium commuter bicycle with hydraulic disc brakes.', maker: 'Gomes Cycles', category: 'city', price: 749, rating: 4.5, inStock: true, tags: ['commuter', 'aluminium'] },
{ id: 'city-step', name: 'Gomes Step-through city bicycle', summary: 'A step-through city bicycle with a chain guard and a hub dynamo.', maker: 'Gomes Cycles', category: 'city', price: 549, rating: 4.0, inStock: true, tags: ['commuter', 'steel'] },
{ id: 'gravel-explorer', name: 'Mori Gravel explorer', summary: 'A steel gravel bicycle built for long unpaved rides.', maker: 'Mori Works', category: 'gravel', price: 1299, rating: 4.8, inStock: true, tags: ['gravel', 'steel'] },
{ id: 'gravel-allroad', name: 'Mori Allroad gravel bicycle', summary: 'An aluminium gravel bicycle with mounts for three bottles.', maker: 'Mori Works', category: 'gravel', price: 999, rating: 4.4, inStock: false, tags: ['gravel', 'aluminium'] },
{ id: 'folding-metro', name: 'Obi Metro folding bicycle', summary: 'A folding bicycle that fits under a train seat.', maker: 'Obi Mobility', category: 'folding', price: 899, rating: 4.2, inStock: true, tags: ['folding', 'commuter'] },
{ id: 'folding-tourer', name: 'Obi Tourer folding bicycle', summary: 'A folding bicycle with a rear rack and eight gears.', maker: 'Obi Mobility', category: 'folding', price: 1150, rating: 4.1, inStock: true, tags: ['folding', 'aluminium'] },
{ id: 'cargo-hauler', name: 'Larsson Cargo hauler', summary: 'An electric cargo bicycle rated to carry two children.', maker: 'Larsson Bikes', category: 'cargo', price: 3499, rating: 4.6, inStock: true, tags: ['cargo', 'electric'] },
{ id: 'cargo-trike', name: 'Larsson Cargo trike', summary: 'A three-wheeled cargo bicycle with a lockable box.', maker: 'Larsson Bikes', category: 'cargo', price: 4195, rating: 4.3, inStock: false, tags: ['cargo', 'electric'] },
{ id: 'road-racer', name: 'Khoury Road racer', summary: 'A carbon road bicycle with electronic shifting.', maker: 'Khoury Composites', category: 'road', price: 4250, rating: 4.9, inStock: true, tags: ['road', 'carbon'] },
{ id: 'road-endurance', name: 'Khoury Endurance road bicycle', summary: 'A carbon road bicycle tuned for comfort over long distances.', maker: 'Khoury Composites', category: 'road', price: 2799, rating: 4.7, inStock: true, tags: ['road', 'carbon'] },
{ id: 'kids-first', name: 'Adams First ride bicycle', summary: 'A first bicycle for a child, with stabilisers that come off in one minute.', maker: 'Adams Assembly', category: 'kids', price: 189, rating: 4.3, inStock: true, tags: ['kids'] },
{ id: 'kids-junior', name: 'Adams Junior mountain bicycle', summary: 'A junior mountain bicycle with a suspension fork and six gears.', maker: 'Adams Assembly', category: 'kids', price: 329, rating: 4.1, inStock: true, tags: ['kids', 'aluminium'] },
])
const results = await narsil.query('bicycles', {
term: 'bicycle',
filters: {
and: [
{ fields: { inStock: { eq: true } } },
{ or: [{ fields: { category: { in: ['gravel', 'road'] } } }, { fields: { rating: { gte: 4.5 } } }] },
{ not: { fields: { maker: { startsWith: 'Khoury' } } } },
],
filters: { fields: { inStock: { eq: true } } },
facets: {
category: {},
maker: { limit: 3 },
tags: {},
price: { ranges: [{ from: 0, to: 500 }, { from: 500, to: 1500 }, { from: 1500, to: 5000 }] },
},
document: { include: ['maker', 'category', 'rating'] },
limit: 0,
})
console.log(JSON.stringify({ count: results.count, hits: results.hits.map((hit) => ({ id: hit.id, ...hit.document })) }, null, 2))
console.log(JSON.stringify(results.facets, null, 2))
await narsil.shutdown(){
"category": {
"values": {
"city": 2,
"folding": 2,
"kids": 2,
"road": 2,
"cargo": 1,
"gravel": 1
},
"count": 6,
"errorBound": 0
},
"maker": {
"values": {
"Adams Assembly": 2,
"Gomes Cycles": 2,
"Khoury Composites": 2
},
"count": 3,
"errorBound": 2
},
"tags": {
"values": {
"aluminium": 3,
"commuter": 3,
"carbon": 2,
"folding": 2,
"kids": 2,
"road": 2,
"steel": 2,
"cargo": 1,
"electric": 1,
"gravel": 1
},
"count": 10,
"errorBound": 0
},
"price": {
"values": {
"500-1500": 5,
"1500-5000": 3,
"0-500": 2
},
"count": 3,
"errorBound": 0
}
}The engine counts the ten bicycles in stock for each facet, most frequent value first. It counts each entry of the tags array on its own and each whole value of maker, a plain string field, never its single words, while it keys a boolean facet's counts under the strings true and false. Four makers hold two bicycles each, so under maker: { limit: 3 } the engine keeps the first three of them in alphabetical order, and the errorBound of 2 means that no maker left out holds more than 2. count inside each facet is the number of values that it returned.
The counts stay exact on an index that you split into partitions, since on 400 documents from 8 makers at 50 each over four partitions the engine returned three makers at 50 each under limit: 3, with an errorBound of 50. Where worker copies or cluster nodes split the counting, however, each of them returns only its own top values, so a value that is common overall but rare on one of them can come up short by as much as errorBound.
Where to go next
- Sorting, grouping, and pagination covers ordering the filtered hits by a field, collapsing them into groups, and paging through them.
- Indexes and documents covers the listing that pages through an index without a search term.
- Vector search covers a search by meaning, which takes the same filter expression.