Narsil includes language modules for tokenisation, Snowball stemming, and stop word removal. Each module is a separate entry point, so your bundle includes only the languages you use. This page builds one file, articles.ts, step by step. You load a language, create an index that names it, add a second index in another language, and override the stop words and tokeniser for one index.
Load a language
A language module exports a language object you pass to registerLanguage. Register it once and its name becomes valid for any index. English is always available, so an English index needs no import or registration. Start articles.ts with a French module and an index that names French.
import { french } from '@delali/narsil/languages/french'
import { createNarsil, registerLanguage } from '@delali/narsil'
const narsil = await createNarsil()
registerLanguage(french)
await narsil.createIndex('articles', {
schema: { title: 'string', body: 'string' },
language: 'french',
})Naming a language that was never registered fails with LANGUAGE_NOT_SUPPORTED. The language option selects the analyser Narsil uses to tokenise and stem every text field on the index, so a French index stems chercher, cherche, and cherchons to the same root and drops French stop words such as le and des.
Index documents in the chosen language
With the module loaded and the index created, add two French articles. Narsil tokenises and stems each text field on the way in, using the French analyser the index selected.
import { french } from '@delali/narsil/languages/french'
import { createNarsil, registerLanguage } from '@delali/narsil'
const narsil = await createNarsil()
registerLanguage(french)
await narsil.createIndex('articles', {
schema: { title: 'string', body: 'string' },
language: 'french',
})
await narsil.insert('articles', {
id: 'recherche-vectorielle',
title: 'La recherche vectorielle expliquée',
body: 'Les vecteurs denses capturent le sens au-delà des mots exacts.',
})
await narsil.insert('articles', {
id: 'partitions',
title: 'Partitionner un index',
body: 'Le partitionnement répartit un index sur plusieurs processus.',
})A French query for vecteurs matches the singular vecteur and the verb form vectorielle, because the stemmer reduces them to the same root before ranking.
Add an index in another language
Each index names its own analyser, so a Swahili index works alongside the French one. A created index keeps the language it was made with, so to add Swahili you register the module and create a second index that names it. Register the Swahili module, create articles-sw, and insert a Swahili document.
import { french } from '@delali/narsil/languages/french'
import { swahili } from '@delali/narsil/languages/swahili'
import { createNarsil, registerLanguage } from '@delali/narsil'
const narsil = await createNarsil()
registerLanguage(french)
registerLanguage(swahili)
await narsil.createIndex('articles', {
schema: { title: 'string', body: 'string' },
language: 'french',
})
await narsil.insert('articles', {
id: 'recherche-vectorielle',
title: 'La recherche vectorielle expliquée',
body: 'Les vecteurs denses capturent le sens au-delà des mots exacts.',
})
await narsil.insert('articles', {
id: 'partitions',
title: 'Partitionner un index',
body: 'Le partitionnement répartit un index sur plusieurs processus.',
})
await narsil.createIndex('articles-sw', {
schema: { title: 'string', body: 'string' },
language: 'swahili',
})
await narsil.insert('articles-sw', {
id: 'utafutaji-wa-vekta',
title: 'Utafutaji wa vekta umefafanuliwa',
body: 'Vekta hunasa maana zaidi ya maneno kamili.',
})Swahili has a full stemmer, so the Swahili index stems and filters stop words the same way the French one does. Each index keeps its own analyser, and the schema, inserts, and queries share the same shape whichever language a field uses.
Coverage
Full support, meaning tokeniser, stemmer, and stop words, covers: Arabic, Armenian, Bulgarian, Danish, Dutch, English, Finnish, French, German, Greek, Hindi, Hungarian, Indonesian, Irish, Italian, Nepali, Norwegian, Portuguese, Romanian, Russian, Sanskrit, Serbian, Slovenian, Spanish, Swahili, Swedish, Tamil, Turkish, and Ukrainian.
Chinese (Mandarin) and Japanese use a character n-gram tokeniser with stop words.
Several African languages are supported: Swahili has a full stemmer, and Dagbani, Ewe, Ga, Hausa, Igbo, Twi (Akan), Yoruba, and Zulu have tokenisation and stop word support. Load any of them the same way, for example import { twi } from '@delali/narsil/languages/twi' and registerLanguage(twi) before creating a Twi index.
Override stop words and the tokeniser per index
A language module sets the defaults for its name, but you can override them on a single index without touching the module. The stopWords field takes the default set and returns a new Set, and the tokenizer field swaps in your own tokenize(text) that returns { token, position } entries. Both live in the index config alongside language. Add them to the articles-sw index you created above.
import { french } from '@delali/narsil/languages/french'
import { swahili } from '@delali/narsil/languages/swahili'
import { createNarsil, registerLanguage } from '@delali/narsil'
const narsil = await createNarsil()
registerLanguage(french)
registerLanguage(swahili)
await narsil.createIndex('articles', {
schema: { title: 'string', body: 'string' },
language: 'french',
})
await narsil.insert('articles', {
id: 'recherche-vectorielle',
title: 'La recherche vectorielle expliquée',
body: 'Les vecteurs denses capturent le sens au-delà des mots exacts.',
})
await narsil.insert('articles', {
id: 'partitions',
title: 'Partitionner un index',
body: 'Le partitionnement répartit un index sur plusieurs processus.',
})
await narsil.createIndex('articles-sw', {
schema: { title: 'string', body: 'string' },
language: 'swahili',
stopWords: (defaults) => new Set([...defaults, 'faharasa']),
tokenizer: {
tokenize: (text) => text.toLowerCase().split(/\s+/).map((token, position) => ({ token, position })),
},
})
await narsil.insert('articles-sw', {
id: 'utafutaji-wa-vekta',
title: 'Utafutaji wa vekta umefafanuliwa',
body: 'Vekta hunasa maana zaidi ya maneno kamili.',
})The override applies to this index alone, so other Swahili indexes keep the module defaults. Use stopWords to add domain terms that carry no signal in your corpus, and use tokenizer when a field needs splitting rules the language module does not provide.
Bring your own language
Call registerLanguage(module) to add your own language. A module provides a name, a tokeniser, a stemmer, and a stop word set. Any bundled module serves as a reference implementation. Once registered, name it in the language option exactly as you would a bundled module.