> Complete content index: https://narsil.sondelali.com/llms.txt

# Search by wording and by meaning

> Add the customer's own words to the catalogue and fuse the keyword and vector rankings, since neither ranking alone puts the right product first for this customer.

URL: https://narsil.sondelali.com/docs/agent-hybrid-retrieval
Section: AI agents
Version: 0.3 (@delali/narsil@0.3.0, latest)

In the example below, a customer at a brewing merchant tells the counter AI agent that their beer tastes of wet cardboard. The catalogue holds the cure, which is an antioxidant that keeps oxygen out of the bottle. A keyword search for that sentence matches nothing at all, because no product description contains the words beer, tastes, wet, or cardboard.

The reverse problem appears in search over a company's own documents. The authors of a 2026 paper on a deployed document search system report that dense vector retrieval alone performs poorly on queries that mix technical terminology with vendor-specific acronyms, which is why their system fuses a vector ranking, a keyword ranking, and a graph expansion through reciprocal rank fusion ([arXiv 2609.01617](https://arxiv.org/abs/2609.01617)). The keyword ranking matches exact terms, while the vector ranking matches paraphrases.

The mock model on this page repeats the top product that the tool returns, which means each answer below depends on the ranking alone. With a model string in its place, the wording changes and the ranking stays the same. Each code block below holds one whole file.

## A question with none of the catalogue's words in it

`createTransformersEmbedding` loads `Xenova/all-MiniLM-L6-v2` into this process and produces 384 numbers for each text, with no API key and no network call after the first download. `embedding.fields` maps the vector field to the text fields that the engine embeds into it. Each insert contains a product name and a description, and the engine fills `embedding` from both.

The tool searches by keyword in this step, which means the vector field is filled but unused. Install the adapter and its peer alongside Narsil.

```bash pm
pnpm add -E @delali/narsil-embeddings-transformers @huggingface/transformers
```

<CodeGroup defaultTab="counter.ts">

```ts title="counter.ts"
import { createNarsil } from '@delali/narsil'
import { createTransformersEmbedding } from '@delali/narsil-embeddings-transformers'
import { ToolLoopAgent, tool } from 'ai'
import { z } from 'zod'
import { type CatalogueHit, counterModel, QUESTION } from './mock-model.ts'

const narsil = await createNarsil({
  embedding: createTransformersEmbedding({ dimensions: 384 }),
})

await narsil.createIndex('brewing_catalogue', {
  schema: { product: 'string', code: 'string', description: 'string', embedding: 'vector[384]' },
  embedding: { fields: { embedding: ['product', 'description'] } },
})

await narsil.insertBatch('brewing_catalogue', [
  {
    id: 'sodium-metabisulphite',
    product: 'Sodium metabisulphite',
    code: 'BR-114',
    description:
      'An antioxidant powder that removes dissolved oxygen from the wort and from the headspace of a filled bottle, which protects the finished drink from staling on the shelf.',
  },
  {
    id: 'irish-moss',
    product: 'Irish moss finings',
    code: 'BR-207',
    description:
      'A seaweed extract added in the last ten minutes of the boil, which settles the protein haze and leaves a brighter glass.',
  },
  {
    id: 'three-piece-airlock',
    product: 'Three-piece airlock',
    code: 'BR-031',
    description:
      'A fermentation lock that lets carbon dioxide leave the vessel and keeps air out of the headspace above the brew.',
  },
  {
    id: 'bench-crown-capper',
    product: 'Bench crown capper',
    code: 'BR-402',
    description: 'A bench press that seats a 26 millimetre crown cap on a returnable bottle.',
  },
  {
    id: 'campden-tablets',
    product: 'Campden tablets',
    code: 'BR-118',
    description: 'Tablets that remove chlorine from the mash water, which prevents a plastic or medicinal off-flavour.',
  },
])

const counterTools = {
  searchCatalogue: tool({
    description: 'Search the brewing catalogue by what its text says.',
    inputSchema: z.object({ text: z.string() }),
    execute: async ({ text }: { text: string }): Promise<CatalogueHit[]> => {
      const found = await narsil.query<CatalogueHit>('brewing_catalogue', {
        term: text,
        document: { include: ['product', 'code', 'description'] },
        limit: 3,
      })
      return found.hits.map((hit) => hit.document)
    },
  }),
}

const askCounterAgent = async (): Promise<string> => {
  const agent = new ToolLoopAgent({
    model: counterModel(),
    instructions: 'Answer the customer from the brewing catalogue.',
    tools: counterTools,
  })
  const reply = await agent.generate({ prompt: QUESTION })
  return reply.text
}

console.log(JSON.stringify({ question: QUESTION, counterAnswered: await askCounterAgent() }, null, 2))

await narsil.shutdown()
```

```json result open
{
  "question": "My beer tastes of wet cardboard. What should I order?",
  "counterAnswered": "The catalogue holds nothing for that."
}
```

```ts title="mock-model.ts"
import { MockLanguageModelV4 } from 'ai/test'

const NO_USAGE = {
  inputTokens: { total: 0, noCache: 0, cacheRead: undefined, cacheWrite: undefined },
  outputTokens: { total: 0, text: 0, reasoning: undefined },
}

export const QUESTION = 'My beer tastes of wet cardboard. What should I order?'

export const SEARCH_TEXT = 'my beer tastes of wet cardboard'

export interface CatalogueHit {
  product: string
  code: string
  description: string
}

interface ToolResultPart {
  toolName?: string
  output?: { value?: unknown }
}

interface PromptMessage {
  role: string
  content: ToolResultPart[] | string
}

const everythingReturnedBy = (prompt: unknown, toolName: string): unknown[] => {
  const outputs: unknown[] = []
  for (const message of prompt as PromptMessage[]) {
    if (message.role !== 'tool' || typeof message.content === 'string') continue
    for (const part of message.content) {
      if (part.toolName === toolName) outputs.push(part.output?.value)
    }
  }
  return outputs
}

const callTool = (toolName: string, input: Record<string, unknown>) => ({
  content: [{ type: 'tool-call' as const, toolCallId: `${toolName}-1`, toolName, input: JSON.stringify(input) }],
  finishReason: { unified: 'tool-calls' as const, raw: undefined },
  usage: NO_USAGE,
  warnings: [],
})

const say = (text: string) => ({
  content: [{ type: 'text' as const, text }],
  finishReason: { unified: 'stop' as const, raw: undefined },
  usage: NO_USAGE,
  warnings: [],
})

export const counterModel = () =>
  new MockLanguageModelV4({
    doGenerate: async ({ prompt }) => {
      const searches = everythingReturnedBy(prompt, 'searchCatalogue') as CatalogueHit[][]
      if (searches.length === 0) return callTool('searchCatalogue', { text: SEARCH_TEXT })
      const found = searches[0]
      if (found.length === 0) return say('The catalogue holds nothing for that.')
      const top = found[0]
      return say(`Order ${top.product}, ${top.code}. ${top.description}`)
    },
  })
```

</CodeGroup>

A keyword search scores a document by the query terms that it contains, and no document contains any term from the customer's sentence. The merchant loses a sale that a person behind the counter would make at once, because to a brewer, wet cardboard means oxidation.

## A similarity search that returns the wrong tub

With `mode: 'vector'`, the engine compares the question's vector with each stored vector, and `vector.text` is the text that it embeds for that comparison. Every query now returns hits, because the engine applies no similarity threshold until you set one.

<CodeGroup defaultTab="counter.ts">

```ts title="counter.ts"
import { createNarsil } from '@delali/narsil'
import { createTransformersEmbedding } from '@delali/narsil-embeddings-transformers'
import { ToolLoopAgent, tool } from 'ai'
import { z } from 'zod'
import { type CatalogueHit, counterModel, QUESTION } from './mock-model.ts'

const narsil = await createNarsil({
  embedding: createTransformersEmbedding({ dimensions: 384 }),
})

await narsil.createIndex('brewing_catalogue', {
  schema: { product: 'string', code: 'string', description: 'string', embedding: 'vector[384]' },
  embedding: { fields: { embedding: ['product', 'description'] } },
})

await narsil.insertBatch('brewing_catalogue', [
  {
    id: 'sodium-metabisulphite',
    product: 'Sodium metabisulphite',
    code: 'BR-114',
    description:
      'An antioxidant powder that removes dissolved oxygen from the wort and from the headspace of a filled bottle, which protects the finished drink from staling on the shelf.',
  },
  {
    id: 'irish-moss',
    product: 'Irish moss finings',
    code: 'BR-207',
    description:
      'A seaweed extract added in the last ten minutes of the boil, which settles the protein haze and leaves a brighter glass.',
  },
  {
    id: 'three-piece-airlock',
    product: 'Three-piece airlock',
    code: 'BR-031',
    description:
      'A fermentation lock that lets carbon dioxide leave the vessel and keeps air out of the headspace above the brew.',
  },
  {
    id: 'bench-crown-capper',
    product: 'Bench crown capper',
    code: 'BR-402',
    description: 'A bench press that seats a 26 millimetre crown cap on a returnable bottle.',
  },
  {
    id: 'campden-tablets',
    product: 'Campden tablets',
    code: 'BR-118',
    description: 'Tablets that remove chlorine from the mash water, which prevents a plastic or medicinal off-flavour.',
  },
])

const counterTools = {
  searchCatalogue: tool({
    description: 'Search the brewing catalogue by what its text says.',
    inputSchema: z.object({ text: z.string() }),
    execute: async ({ text }: { text: string }): Promise<CatalogueHit[]> => {
      const found = await narsil.query<CatalogueHit>('brewing_catalogue', {
        term: text, // [!code --]
        mode: 'vector', // [!code ++:2]
        vector: { field: 'embedding', text },
        document: { include: ['product', 'code', 'description'] },
        limit: 3,
      })
      return found.hits.map((hit) => hit.document)
    },
  }),
}

const askCounterAgent = async (): Promise<string> => {
  const agent = new ToolLoopAgent({
    model: counterModel(),
    instructions: 'Answer the customer from the brewing catalogue.',
    tools: counterTools,
  })
  const reply = await agent.generate({ prompt: QUESTION })
  return reply.text
}

console.log(JSON.stringify({ question: QUESTION, counterAnswered: await askCounterAgent() }, null, 2))

await narsil.shutdown()
```

```json result open
{
  "question": "My beer tastes of wet cardboard. What should I order?",
  "counterAnswered": "Order Irish moss finings, BR-207. A seaweed extract added in the last ten minutes of the boil, which settles the protein haze and leaves a brighter glass."
}
```

</CodeGroup>

The customer leaves with finings for a haze that their beer does not have. Irish moss scores 0.2906 and the airlock 0.2378, while the antioxidant is outside the top three. A sentence-embedding model of this size places a paraphrase near its source; however, it has no brewing vocabulary, so the vectors for 'wet cardboard' and 'staling on the shelf' are far apart.

Solve this in the catalogue before you solve it in the query. A `symptoms` field holds the words that customers use for each fault, which gives the keyword search an exact match and also brings the antioxidant's vector closer to the question.

## Fuse the two rankings over a catalogue that holds both vocabularies

With `mode: 'hybrid'`, the engine takes `term` for the keyword ranking and `vector` for the similarity ranking, and merges the two rankings by reciprocal rank fusion. A fused score is therefore much lower than a BM25 score, and you can compare it only with other scores in the same result set. Print four decimals to tell two fused hits apart.

Add `symptoms` to the schema, to the embedding fields, and to every document. The keyword search then matches the word cardboard in exactly one product, with a BM25 score of 2.4399, while the vector search still returns all five products in its own order.

<CodeGroup defaultTab="counter.ts">

```ts title="counter.ts"
import { createNarsil } from '@delali/narsil'
import { createTransformersEmbedding } from '@delali/narsil-embeddings-transformers'
import { ToolLoopAgent, tool } from 'ai'
import { z } from 'zod'
import { type CatalogueHit, counterModel, QUESTION } from './mock-model.ts'

const narsil = await createNarsil({
  embedding: createTransformersEmbedding({ dimensions: 384 }),
})

await narsil.createIndex('brewing_catalogue', {
  schema: { product: 'string', code: 'string', description: 'string', embedding: 'vector[384]' }, // [!code --]
  embedding: { fields: { embedding: ['product', 'description'] } }, // [!code --]
  schema: { product: 'string', code: 'string', description: 'string', symptoms: 'string', embedding: 'vector[384]' }, // [!code ++:2]
  embedding: { fields: { embedding: ['product', 'description', 'symptoms'] } },
})

await narsil.insertBatch('brewing_catalogue', [
  {
    id: 'sodium-metabisulphite',
    product: 'Sodium metabisulphite',
    code: 'BR-114',
    description:
      'An antioxidant powder that removes dissolved oxygen from the wort and from the headspace of a filled bottle, which protects the finished drink from staling on the shelf.',
    symptoms: 'Wet cardboard, sherry, or papery flavours in a bottle that has stood too long.', // [!code ++]
  },
  {
    id: 'irish-moss',
    product: 'Irish moss finings',
    code: 'BR-207',
    description:
      'A seaweed extract added in the last ten minutes of the boil, which settles the protein haze and leaves a brighter glass.',
    symptoms: 'A cloudy pour that stays hazy after a week in the cold.', // [!code ++]
  },
  {
    id: 'three-piece-airlock',
    product: 'Three-piece airlock',
    code: 'BR-031',
    description:
      'A fermentation lock that lets carbon dioxide leave the vessel and keeps air out of the headspace above the brew.',
    symptoms: 'A vessel that draws air back in as it cools.', // [!code ++]
  },
  {
    id: 'bench-crown-capper',
    product: 'Bench crown capper',
    code: 'BR-402',
    description: 'A bench press that seats a 26 millimetre crown cap on a returnable bottle.',
    symptoms: 'Caps that leak or crimp unevenly by hand.', // [!code ++]
  },
  {
    id: 'campden-tablets',
    product: 'Campden tablets',
    code: 'BR-118',
    description: 'Tablets that remove chlorine from the mash water, which prevents a plastic or medicinal off-flavour.',
    symptoms: 'A plastic or TCP note that arrives with the first sip.', // [!code ++]
  },
])

const counterTools = {
  searchCatalogue: tool({
    description: 'Search the brewing catalogue by what its text says.',
    inputSchema: z.object({ text: z.string() }),
    execute: async ({ text }: { text: string }): Promise<CatalogueHit[]> => {
      const found = await narsil.query<CatalogueHit>('brewing_catalogue', {
        mode: 'vector', // [!code --]
        mode: 'hybrid', // [!code ++:2]
        term: text,
        vector: { field: 'embedding', text },
        document: { include: ['product', 'code', 'description'] },
        limit: 3,
      })
      return found.hits.map((hit) => hit.document)
    },
  }),
}

const askCounterAgent = async (): Promise<string> => {
  const agent = new ToolLoopAgent({
    model: counterModel(),
    instructions: 'Answer the customer from the brewing catalogue.',
    tools: counterTools,
  })
  const reply = await agent.generate({ prompt: QUESTION })
  return reply.text
}

console.log(JSON.stringify({ question: QUESTION, counterAnswered: await askCounterAgent() }, null, 2))

await narsil.shutdown()
```

```json result open
{
  "question": "My beer tastes of wet cardboard. What should I order?",
  "counterAnswered": "Order Sodium metabisulphite, BR-114. An antioxidant powder that removes dissolved oxygen from the wort and from the headspace of a filled bottle, which protects the finished drink from staling on the shelf."
}
```

</CodeGroup>

The customer leaves with the antioxidant. `document.include` lists three fields and leaves out the vector field. That matters for each hit as well as for the prompt, because the engine returns a stored vector as a `Float32Array`, which `JSON.stringify` turns into an object keyed by index.

<RankMovementDiagram
  highlightProduct="Sodium metabisulphite"
  rows={[
    { product: 'Sodium metabisulphite', keywordRank: 1, vectorRank: 2, fusedRank: 1, fusedScore: 0.0325 },
    { product: 'Irish moss finings', keywordRank: null, vectorRank: 1, fusedRank: 2, fusedScore: 0.0164 },
    { product: 'Three-piece airlock', keywordRank: null, vectorRank: 3, fusedRank: 3, fusedScore: 0.0159 },
  ]}
  caption="The keyword ranking contains the antioxidant alone, matched on the word cardboard, while it is second in the vector ranking. Fusion adds a score for each position, so the only product in both rankings has the highest fused score."
/>

When one ranking matters more than the other, `hybrid` accepts a linear blend with a weight, and [Hybrid search](/docs/hybrid-search) has the details of that blend. An embedding provider that you call over a network adds its own latency and its own failure modes to every query. Choose a provider, and batch a bulk load at a lower price, with the guidance in [Embedding adapters](/docs/embedding-adapters).

## Where to go next

- Read [Hybrid search](/docs/hybrid-search) for the fusion, the linear blend, and the two rankings behind a fused order.
- See [Vector search](/docs/vector-search) for the similarity floor, the graph, and the quantisation modes, which change how much memory a vector field uses.
- When neither search returns anything worth sending, return the status from [When the index has no good match](/docs/agent-weak-matches).