Embedding adapters

Embedding adapters turn text into vectors automatically, on insert and at query time. You configure an adapter once, map each vector field to the text fields it embeds, and Narsil handles the rest: inserts embed the source text before indexing, and vector queries embed the search text the same way. This page builds one file, embeddings.ts, step by step. You start with a default adapter for the whole engine, then move to a named adapter that survives durability recovery. Each step highlights exactly what changed.

Auto-embed an index

Pass an embedding adapter to createNarsil and it becomes the default for every index. The OpenAI adapter talks to any OpenAI-compatible endpoint through fetch. Start embeddings.ts with an engine and an articles index whose embedding field is built from the title and body text. Multiple source fields concatenate before embedding.

import { createNarsil } from '@delali/narsil'
import { createOpenAIEmbedding } from '@delali/narsil/embeddings/openai'
 
const narsil = await createNarsil({
  embedding: createOpenAIEmbedding({
    baseUrl: 'https://api.openai.com/v1',
    apiKey: process.env.OPENAI_API_KEY ?? '',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  }),
})
 
await narsil.createIndex('articles', {
  schema: {
    title: 'string',
    body: 'string',
    embedding: 'vector[1536]',
  },
  embedding: {
    fields: {
      embedding: ['title', 'body'],
    },
  },
})

The dimensions value on the adapter must match the vector[1536] field size. A mismatch fails at index creation with EMBEDDING_DIMENSION_MISMATCH.

Insert without touching a vector

With the index in place, add two articles to embeddings.ts. Neither insert carries an embedding value: Narsil reads title and body, embeds them through the default adapter, and stores the result in the embedding field for you.

import { createNarsil } from '@delali/narsil'
import { createOpenAIEmbedding } from '@delali/narsil/embeddings/openai'
 
const narsil = await createNarsil({
  embedding: createOpenAIEmbedding({
    baseUrl: 'https://api.openai.com/v1',
    apiKey: process.env.OPENAI_API_KEY ?? '',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  }),
})
 
await narsil.createIndex('articles', {
  schema: {
    title: 'string',
    body: 'string',
    embedding: 'vector[1536]',
  },
  embedding: {
    fields: {
      embedding: ['title', 'body'],
    },
  },
})
 
await narsil.insert('articles', { 
  id: 'scaling-search',
  title: 'Scaling search with partitions',
  body: 'Partitioning spreads one index across many workers so throughput rises with hardware.',
})
 
await narsil.insert('articles', { 
  id: 'vector-basics',
  title: 'Vector search basics',
  body: 'Dense vectors capture meaning beyond the exact keywords a reader typed.',
})

Query in plain text

A vector query passes text instead of a raw Float32Array, and the same default adapter embeds it before the search runs. Add the query to the end of embeddings.ts and read the ranked hits.

import { createNarsil } from '@delali/narsil'
import { createOpenAIEmbedding } from '@delali/narsil/embeddings/openai'
 
const narsil = await createNarsil({
  embedding: createOpenAIEmbedding({
    baseUrl: 'https://api.openai.com/v1',
    apiKey: process.env.OPENAI_API_KEY ?? '',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  }),
})
 
await narsil.createIndex('articles', {
  schema: {
    title: 'string',
    body: 'string',
    embedding: 'vector[1536]',
  },
  embedding: {
    fields: {
      embedding: ['title', 'body'],
    },
  },
})
 
await narsil.insert('articles', {
  id: 'scaling-search',
  title: 'Scaling search with partitions',
  body: 'Partitioning spreads one index across many workers so throughput rises with hardware.',
})
 
await narsil.insert('articles', {
  id: 'vector-basics',
  title: 'Vector search basics',
  body: 'Dense vectors capture meaning beyond the exact keywords a reader typed.',
})
 
const results = await narsil.query('articles', { 
  mode: 'vector',
  vector: { field: 'embedding', text: 'how do search engines scale?' },
})

The query embeds how do search engines scale? through the default adapter and ranks the two articles by cosine similarity. The scaling article comes back first, because its meaning is closest to the query, even though neither document repeats the exact phrase. Each hit has the same shape as any other search mode, { id, score, document }, and the exact scores depend on the embedding model you configured, so this page describes the ranking rather than pinning a number the model would set. The insert embedded title and body into the embedding field without you constructing a vector, and the query embedded its text the same way. The vector remains in the index and is never returned on hit.document, so each hit shows only the title and body you inserted.

Name the adapter so it survives recovery

An adapter instance is a function, so it cannot be serialised to disk. When you turn on durability, an index that points at a default adapter has nothing to rebind to after a restart. Give the adapter a name instead: move it into an embeddingAdapters map, reference that name from the index with adapter:, and turn on durability. The engine persists the name in index metadata and rebinds the live adapter on the next start.

import { createNarsil } from '@delali/narsil'
import { createOpenAIEmbedding } from '@delali/narsil/embeddings/openai'
 
const narsil = await createNarsil({
  embedding: createOpenAIEmbedding({ 
    baseUrl: 'https://api.openai.com/v1',
    apiKey: process.env.OPENAI_API_KEY ?? '',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  }),
  embeddingAdapters: { 
    'openai-small': createOpenAIEmbedding({
      baseUrl: 'https://api.openai.com/v1',
      apiKey: process.env.OPENAI_API_KEY ?? '',
      model: 'text-embedding-3-small',
      dimensions: 1536,
    }),
  },
  durability: { directory: './narsil-data' }, 
})
 
await narsil.createIndex('articles', {
  schema: {
    title: 'string',
    body: 'string',
    embedding: 'vector[1536]',
  },
  embedding: {
    adapter: 'openai-small', 
    fields: {
      embedding: ['title', 'body'],
    },
  },
})

The inserts and the query above keep working unchanged, because the index now resolves its adapter by name.

Rebind an adapter at runtime

registerEmbeddingAdapter(name, adapter) replaces the live adapter behind a name and rebinds every index that references it. Call it to rotate an API key or swap the model without recreating any index. This call is separate from the running script, so run it on its own with a fresh adapter under the same openai-small name.

narsil.registerEmbeddingAdapter(
  'openai-small',
  createOpenAIEmbedding({
    baseUrl: 'https://api.openai.com/v1',
    apiKey: process.env.OPENAI_API_KEY_ROTATED ?? '',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  }),
)

Bundled adapters

Write your own

Any object that satisfies the EmbeddingAdapter interface works in place of the bundled adapters. This is a reference shape, so it is separate from the running file above.

interface EmbeddingAdapter {
  embed(input: string, purpose: 'document' | 'query', signal?: AbortSignal): Promise<Float32Array>
  embedBatch?(inputs: string[], purpose: 'document' | 'query', signal?: AbortSignal): Promise<Float32Array[]>
  readonly dimensions: number
  shutdown?(): Promise<void>
}

The purpose argument marks whether the adapter is embedding a document at insert time or a query at search time, which asymmetric models such as E5 and BGE use to prefix the input differently. When embedBatch is missing, Narsil falls back to parallel embed() calls with a concurrency limit of 8. The dimensions value must match the vector field's declared size; a mismatch fails at index creation with EMBEDDING_DIMENSION_MISMATCH.