This is a viewer only at the moment see the article on how this works.
To update the preview hit Ctrl-Alt-R (or ⌘-Alt-R on Mac) or Enter to refresh. The Save icon lets you save the markdown file to disk
This is a preview from the server running through my markdig pipeline
RAG for Implementers: CPU-Friendly Semantic Search with ONNX and Qdrant
Thursday, 05 November 2026
You search for “deploy an application in a container”. The article you need is called “Getting Started with Docker”. A useful search system should connect those ideas, even when the words don't match exactly.
NOTE: Updated 5 November 2026. Rebuilt for .NET 10 with correct tokenisation and pooling, passage-level indexing, source context, filtered retrieval and safer index updates. A new section shows how EmbeddingGemma 2 extends the architecture to images, audio and video.
This article builds that retrieval foundation in C#: run a small embedding model locally with ONNX Runtime, store its vectors in Qdrant, and retrieve related posts. It needs no language-generation service and no GPU. It still consumes CPU, memory, storage and operational effort; self-hosting removes the per-request embedding API bill, not the cost of running software.
This is Part 4a of the RAG series:
- Part 1: RAG Origins and Fundamentals
- Part 2: RAG Architecture and Internals
- Part 3: RAG in Practice
- Part 4a: ONNX and Qdrant implementation — this article
- Part 4b: Semantic Search in Action
- Part 5: Hybrid Search and Auto-Indexing
- Part 6: GraphRAG
You should be comfortable with C#, async methods and dependency injection. I'll explain the model and database terminology as we encounter it.
What we are building
An embedding is an array of numbers produced by a trained model. We compare those arrays to retrieve text that the model places near a query. It is a learned representation of useful relationships, rather than a proof that two passages mean the same thing.
ONNX is a format for representing a machine-learning model. ONNX Runtime executes that model. Qdrant stores vectors and finds nearby candidates, alongside ordinary fields that let us identify and filter the results.
The indexing path starts with source material, before anything reaches the model:
flowchart LR
D[Source document] --> E[Extract body and structure]
E --> S[Passages with headings and source ranges]
S --> T[MiniLM tokenizer and ONNX encoder]
T --> P[Pool and normalize each passage]
P --> V[Qdrant passage vectors and provenance]
Q[Search query] --> T
V --> F[Filter and rank passages]
F --> R[Best passage per post]
R --> B[Load current post details]
Indexing stores a vector for each passage. Searching embeds the query with the same model pipeline, filters eligible passages and returns the best match for each post. Related-post retrieval uses a separate summary vector, formed from that post's passage vectors.
RAG means retrieval-augmented generation: retrieve evidence, then provide it to a model generating an answer. We implement retrieval here. A search result is useful on its own, and nothing in this walkthrough generates an answer or establishes that an answer is factually correct.
Choose the representation before choosing the database
We use sentence-transformers/all-MiniLM-L6-v2, an English sentence-embedding model that produces 384-dimensional vectors. Its small size makes it a practical CPU baseline. Other embedding models may perform better on your languages, code or domain-specific vocabulary.
The important contract is larger than the model filename:
| Component | What must agree between indexing and searching |
|---|---|
| Model | The same weights and export |
| Tokenizer | Vocabulary, case handling, subword rules and special tokens |
| Input preparation | Which text is embedded and how long inputs are split |
| Pooling | How per-token outputs become one vector |
| Normalisation | How vector magnitude is handled |
| Database schema | Dimension, distance metric and pipeline version |
Two models can both produce 384 floats while placing those floats in incompatible spaces. A dimension check alone cannot detect that mistake.
The source lives in Mostlylucid.SemanticSearch. The walkthrough uses its public interfaces; the shorter internal excerpts below explain how it works without duplicating entire service classes.
Extraction and segmentation decide what can be found
Suppose a page contains a deployment guide, a navigation menu, three related-post panels and a cookie notice. Embedding the whole HTML response makes all of those things candidates for retrieval. Repeating the same navigation on every page can make unrelated pages look artificially similar.
Extraction selects the useful source content. Segmentation divides that content into retrievable units. Context extraction retains the information needed to interpret each unit: its title, section heading, source location and, where relevant, surrounding structure. These are separate jobs.
StyloExtract tackles HTML extraction through parsing, block segmentation, classification and rendering. It preserves headings, links, lists, tables and code while identifying page furniture. Its template induction article explains how learned site structure can support subsequent extraction. Put that work before this embedding pipeline: cleaner evidence gives the model a better input.
For this library, BlogPostDocument.Content should already contain extracted text or clean Markdown. The library doesn't crawl pages or remove HTML navigation. If headings matter, retain Markdown headings rather than stripping all structure to plain text.
The included ContentSegmenter is a small baseline: up to 800 characters per passage, 100 characters of overlap, paragraph boundaries where practical, and the preceding Markdown heading. It records exact start and length in the supplied content. These are UTF-16 string offsets, not byte positions in the original HTML; retain a source mapping if you need to highlight the original page.
Each passage is embedded with its post title and heading. Those context fields influence the vector but aren't spliced into the quoted source text. A paragraph saying “set this to false” becomes more useful when the model also sees “Docker deployment / HTTPS behind a proxy”. The result still points to what the document actually said.
Character segmentation is a readability heuristic, not a token budget. The encoder handles the actual token limit separately. This baseline can split a code fence or table; a production extractor should emit intact code blocks, table rows with headers, and section paths where those structures matter. DocSummarizer's RAG pipeline develops richer segment extraction and metadata for that next step. Its API is an alternative ingestion layer, rather than an extra dependency required to run this example.
Extraction quality is measurable. Keep examples with repeated navigation, nested headings, long code blocks and tables. Check that the relevant source survives, that its context is retained, and that a retrieved passage can be traced back to its source before judging the embedding model.
Run a small example first
Start Qdrant locally
From the repository root:
docker compose -f semantic-search-docker-compose.yml up -d
curl --fail http://localhost:6333/readyz
The standalone Compose file uses a persistent volume and binds to loopback:
services:
qdrant:
image: qdrant/qdrant:v1.20.0
restart: unless-stopped
ports:
- "127.0.0.1:6333:6333"
- "127.0.0.1:6334:6334"
volumes:
- qdrant_storage:/qdrant/storage
volumes:
qdrant_storage:
6333 is HTTP; 6334 is gRPC. The Qdrant .NET client uses gRPC. Configure http://localhost:6334, even though the readiness request and dashboard use 6333. A web browser showing a healthy dashboard doesn't prove your application's gRPC configuration is correct.
The image is pinned for reproducibility. Review and test an explicit upgrade rather than treating a moving latest tag as a deployment strategy. The local example has no authentication; it is bound to your machine. For remote deployments, configure private networking, authentication and transport protection using Qdrant's security guidance.
Download the model and matching vocabulary
The repository includes Bash and PowerShell scripts:
bash Mostlylucid.SemanticSearch/download-models.sh
./Mostlylucid.SemanticSearch/download-models.ps1
Both put all-MiniLM-L6-v2.onnx and vocab.txt in Mostlylucid/mlmodels. They fetch the same pinned model revision as the service. Files are downloaded to temporary siblings and renamed only after a successful download, so an interrupted transfer doesn't leave a partial model masquerading as a completed one.
The service can also fetch missing assets when it first initialises. For deployment, downloading them during preparation gives you a chance to verify and retain the assets before serving traffic. Keep model files in a persistent directory outside the web root. For stronger supply-chain control, retain checksums with the release and verify them before deployment.
Changing the configured revision doesn't replace a file already present at the same path. Stage a changed model and vocabulary in a new directory, update the paths and revision together, and rebuild the index.
Create a console host
From the repository root, create a small host that references the implementation:
dotnet new console -n SemanticSearchDemo -f net10.0
dotnet add SemanticSearchDemo/SemanticSearchDemo.csproj reference Mostlylucid.SemanticSearch/Mostlylucid.SemanticSearch.csproj
Replace its Program.cs with:
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Mostlylucid.SemanticSearch.Extensions;
using Mostlylucid.SemanticSearch.Models;
using Mostlylucid.SemanticSearch.Services;
var configuration = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string?>
{
["SemanticSearch:Enabled"] = "true",
["SemanticSearch:QdrantUrl"] =
Environment.GetEnvironmentVariable("QDRANT_DEMO_URL") ?? "http://localhost:6334",
["SemanticSearch:ReadApiKey"] = Environment.GetEnvironmentVariable("QDRANT_DEMO_READ_KEY"),
["SemanticSearch:WriteApiKey"] = Environment.GetEnvironmentVariable("QDRANT_DEMO_WRITE_KEY"),
["SemanticSearch:CollectionName"] = "blog_posts_demo_v1",
["SemanticSearch:EmbeddingModelPath"] =
"Mostlylucid/mlmodels/all-MiniLM-L6-v2.onnx",
["SemanticSearch:VocabPath"] = "Mostlylucid/mlmodels/vocab.txt",
["SemanticSearch:MinimumSimilarityScore"] = "-1"
})
.Build();
var services = new ServiceCollection();
services.AddLogging(logging => logging.AddConsole());
services.AddSemanticSearch(configuration);
using var provider = services.BuildServiceProvider();
var search = provider.GetRequiredService<ISemanticSearchService>();
await search.InitializeAsync();
await search.IndexPostsAsync(new[]
{
new BlogPostDocument
{
Id = "docker-deployment",
Slug = "docker-deployment",
Title = "Deploying with Docker",
Content = "Package a web application in a container image and run it with Docker Compose.",
Languages = new[] { "en" }
},
new BlogPostDocument
{
Id = "chocolate-cake",
Slug = "chocolate-cake",
Title = "Baking a chocolate cake",
Content = "Mix flour, butter and chocolate, then bake the cake in an oven.",
Languages = new[] { "en" }
}
});
var results = await search.SearchAsync(
"How do I deploy an application in a container?", limit: 2);
foreach (var result in results)
Console.WriteLine($"{result.Score:F3} {result.Slug}");
Run from the repository root so the relative model paths resolve consistently:
dotnet run --project SemanticSearchDemo/SemanticSearchDemo.csproj
The Docker post should rank above the cake post. The example deliberately uses a threshold of -1 to show both scores, including weak matches. That is a diagnostic setting for this tiny corpus, not a relevance policy to copy into production.
Running the indexing code again updates the same points. An unchanged document is skipped before inference. No new random ID is generated on each run.
Getting the embedding pipeline right
Tokens must match the trained model
A token is a model input unit. It may be a whole word, a subword or punctuation. MiniLM uses a BERT-style WordPiece tokenizer: a word absent from the whole-word vocabulary may still be represented by several known pieces.
A regex split followed by a vocabulary lookup is insufficient. Hashing an unknown word into a vocabulary index is worse: it assigns the word the meaning of an unrelated trained token.
The implementation uses BertTokenizer from Microsoft.ML.Tokenizers:
var tokenizer = BertTokenizer.Create(vocabPath, new BertOptions
{
LowerCaseBeforeTokenization = true,
ApplyBasicTokenization = true
});
var ids = tokenizer.EncodeToIds(text, addSpecialTokens: false);
We add the classification token [CLS] and separator [SEP] around each model window afterwards. They occupy two of the 256 available positions, leaving 254 content tokens per window.
This vocabulary-based adapter is specific to this model family. A different model may require a different tokenizer, input names, query prefix or pooling rule. Replacing the .onnx file alone is not a model migration.
Passages and model windows solve different problems
A passage is the unit we want to retrieve. A model window is the unit the encoder can process in one call. They needn't be identical.
Two thousand characters is not a token limit. Code, URLs, punctuation and different languages can produce different token counts for the same number of characters. Even an 800-character passage can need multiple MiniLM windows once its title and heading are included.
The encoder tokenises the complete supplied input, then walks through windows with a configurable overlap of 32 content tokens. Every window includes its own special tokens and stays within 256 positions. The final window includes the end of the input.
flowchart TB
A[Passage plus title and heading] --> B[WordPiece token IDs]
B --> C[First window]
B --> D[Overlapping windows through the tail]
C --> E[Masked pooling and normalization]
D --> E
E --> F[Average window vectors and normalize]
F --> G[One vector for this passage]
G --> H[Store source range and context]
Averaging multiple window vectors still dilutes fine detail and gives a short final window equal influence. Keeping passages reasonably focused reduces that problem. It also means a search can return the paragraph about TLS without requiring that detail to dominate an entire article's vector.
The separate document summary averages passage vectors for broad related-post suggestions. That summary is useful for discovery; the passage text and source range provide the evidence for a question-answering system.
ONNX outputs token vectors first
The model consumes three integer tensors:
| Input | Shape for one window | Meaning |
|---|---|---|
input_ids |
[1, tokenCount] |
Vocabulary IDs |
attention_mask |
[1, tokenCount] |
1 for real tokens, 0 for padding |
token_type_ids |
[1, tokenCount] |
All zero for this single-text input |
A tensor is a multidimensional array. Here the batch dimension is one. The service uses the actual window length; it doesn't pad every short query to 256 positions.
The selected export returns last_hidden_state, shaped [1, tokenCount, 384]: one vector per token. Flattening that tensor produces tokenCount × 384 numbers, not one sentence embedding.
We validate input and output names when loading the model, select the output by name, and reject unexpected shapes. This prevents a different export from quietly changing what gets indexed.
Pool, then normalise
Mean pooling averages the token vectors. The attention mask excludes padding from both the sum and the count:
windowVector[d] = sum(tokenVector[t, d] where mask[t] = 1)
/ number of included tokens
Special tokens remain included, matching the masked mean-pooling approach used by this pipeline. For the implementation and its padding regression test, see OnnxEmbeddingService. The Sentence Transformers ONNX guidance also explains that a transformer export can require pooling and normalisation outside the model.
L2 normalisation divides the vector by its Euclidean length. A nonzero result then has length one. We normalise each window vector and the final mean. Zero-magnitude and non-finite vectors are errors, not valid substitutes for a failed model invocation.
Similarity is a ranking signal
Cosine similarity compares the directions of two vectors:
cosine(a, b) = dot(a, b) / (length(a) × length(b))
For unit-length vectors, this simplifies to their dot product. Scores can range from -1 to 1. A result scoring 0.72 does not mean it has a 72% probability of being relevant.
Qdrant's cosine metric also normalises vectors on upload, as described in its vector documentation. We normalise in the embedding service too so its outputs behave consistently in tests and comparisons outside Qdrant.
A threshold that works for related-post suggestions may suppress useful results for short search queries. Tune it on examples from your own corpus. The model, document preparation and corpus all affect the score distribution.
Store a versioned retrieval index
A Qdrant collection contains searchable points. A point has an ID, vector and a payload, meaning ordinary fields stored alongside the vector.
The default MiniLM collection uses 384 dimensions with cosine distance. Initialisation checks an existing collection's vector schema instead of assuming a matching name implies compatibility.
Its payload contains:
| Field | Purpose |
|---|---|
slug |
Find the canonical post in the application database |
content_hash |
Detect changed indexing inputs |
embedding_version |
Keep incompatible vectors out of the same search |
published_date |
Unix timestamp available for date filtering |
languages |
Available translations of the post |
categories |
Fields available for filtering |
record_type |
Distinguish a document summary from a passage |
source_text, source_start, source_length |
Recover the matched source passage |
heading, source_uri |
Explain its context and link to its source |
modality |
Text, image, audio or video eligibility |
page, start_seconds, end_seconds |
Optional media locations supplied by ingestion |
Display titles, publication details and rendered posts are loaded from PostgreSQL. Qdrant also retains extracted passage text for evidence retrieval, so apply the same retention and access rules to this index as to its sources. This keeps Qdrant a rebuildable retrieval index rather than a second independent publishing database.
The service creates keyword indexes on slug, version, languages, categories, record type, modality and content hash, plus an integer index on publication date. Payload indexes make corresponding filters usable efficiently; adding a payload field alone doesn't create an index.
Publish a complete passage generation
A document summary has a deterministic unsigned 64-bit ID derived from the document ID using xxHash64. Passage IDs also incorporate the pipeline version, content fingerprint and segment ID. Repeated indexing is idempotent; changing content creates a new generation of passages. These hashes aren't a cryptographic identity scheme. Use a persisted ID mapping or suitable UUID scheme if collision risk is unacceptable.
The update order matters:
- Upload the new generation's passages in bounded batches, waiting for each write to complete.
- Commit the document summary, whose fingerprint acts as the manifest identifying the completed generation.
- Delete obsolete passages for that post.
Readers compare a candidate passage's fingerprint with its parent manifest. A cancelled upload cannot expose a half-written generation. Until the manifest changes, readers can use the previous generation. Failed cleanup can leave unused points taking space, but they don't become eligible results; periodic reconciliation should remove them.
This is an application publication protocol, not a transaction across Qdrant and PostgreSQL or a snapshot spanning several search requests. The writer lock covers one service instance. Coordinate multiple writers for the same post, or route indexing through a single worker. For large migrations, building another collection remains simpler to reason about.
Passage vectors are currently retained for one document at a time before upload. Write batches are bounded; the largest individual document still determines peak indexing memory. For very large books or recordings, stream segments through a staging generation instead.
Retrieve passages without filling the page with one post
Ten highly ranked chunks from one article shouldn't consume ten search results. The passage path keeps the best eligible passage per slug and fetches additional candidates while it needs distinct posts. It inspects at most MaxSearchCandidates candidates, default 4,096. A highly repetitive corpus can therefore return fewer than the requested number of posts; grouping and the candidate budget trade completeness for bounded work.
Use the richer method when you need the evidence:
var hits = await search.SearchPassagesAsync(
"configure HTTPS behind a reverse proxy",
new RetrievalFilter(Language: "en", Modality: "text"),
limit: 5);
foreach (var hit in hits)
Console.WriteLine($"{hit.Slug}: {hit.Segment.Heading} — {hit.Segment.Text}");
Language, category, modality and optional publication-date filters are applied inside the Qdrant query, before ranking and grouping. They are eligibility rules, not post-processing decorations. The simpler SearchAsync used earlier returns only slugs and scores while using this passage retrieval path underneath.
Fingerprint every input that matters
The indexing fingerprint includes the ID, slug, title, full content, publication date, sorted distinct languages and categories, and effective pipeline version. Title-only or metadata-only edits therefore count as changes too.
The effective version includes the configured algorithm label, pinned model revision, token overlap and passage segmentation settings. Change the algorithm label when document preparation, tokenisation or pooling changes. Matching dimensions are insufficient evidence that two versions are comparable.
Searches and related-post lookups filter on this version. Old points are not silently treated as new embeddings, and an old-version point doesn't satisfy the hash lookup that decides whether reindexing is needed.
For a live migration, create a new collection, rebuild it, verify relevance and completeness, and switch readers after it is ready. Keeping the previous collection temporarily provides a rollback path. Rebuilding in place can leave fewer searchable posts during the transition.
Related posts need a vector to compare
Retrieving payload fields doesn't necessarily retrieve vectors. The related-post path explicitly requests the source vector, validates it, then queries other points in the same pipeline version while excluding the source point by ID.
The relevant query is:
var related = await readClient.QueryAsync(
collectionName: collectionName,
query: sourceVector,
filter: filterWithVersionAndSourceExclusion,
limit: (ulong)limit,
scoreThreshold: minimumScore,
payloadSelector: true,
cancellationToken: cancellationToken);
Here filterWithVersionAndSourceExclusion is the filter constructed in the vector-store service, and sourceVector is the fetched vector. The complete method is in QdrantVectorStoreService.
Connect it to ASP.NET Core
Register the services once through the extension:
builder.Services.AddSemanticSearch(builder.Configuration);
The embedding service, vector-store service and orchestrator are singletons. The model session is expensive to construct and is reused. Model downloads use a named HTTP client, and the host disposes the services when it shuts down.
The configuration section is:
{
"SemanticSearch": {
"Enabled": true,
"QdrantUrl": "http://localhost:6334",
"CollectionName": "blog_posts",
"EmbeddingModelPath": "mlmodels/all-MiniLM-L6-v2.onnx",
"VocabPath": "mlmodels/vocab.txt",
"VectorSize": 384,
"EmbeddingVersion": "minilm-segments-v2",
"IntraOpNumThreads": 2,
"MaxConcurrentEmbeddings": 1,
"ChunkOverlapTokens": 32,
"SegmentMaxCharacters": 800,
"SegmentOverlapCharacters": 100,
"MaxSearchCandidates": 4096,
"IndexBatchSize": 16,
"MinimumSimilarityScore": 0.5,
"SearchResultsCount": 10,
"RelatedPostsCount": 5
}
}
ModelRevision defaults to the commit pinned in SemanticSearchConfig. The library validates settings such as dimensions, positive limits and overlap smaller than the content window.
Initialise during startup or a managed warm-up process. If search is optional, keep the rest of the site available when initialisation fails, while reporting search unavailability honestly. An HTTP endpoint should distinguish a successful search with no matches from a failed search service.
The dedicated search endpoint now validates query length and result limits, passes the request cancellation token, and returns 503 for operational failures. The library lets exceptions and cancellation reach the caller; it doesn't turn a failed embedding into zeros or claim that a failed write was indexed successfully.
Related-post caching varies by slug, requested language, result limit and HTMX response format. When integrating elsewhere, cache keys must include every input that changes the response, plus the index version if your cache can outlive a model deployment.
What language support means here
The blog indexes canonical source content and records its available translations on the summary and passage points. A languages payload entry means a translation exists; it doesn't mean the embedding model understands queries equally well in that language.
The UI loads current post details for the requested language. The related-post UI can still return fewer posts after language-specific display filtering. Passage search supports a language prefilter through RetrievalFilter; wire that trusted application preference into your search endpoint when required. A language flag describes availability, not the language of each stored passage.
Evaluate a multilingual model or a documented query-translation path for multilingual retrieval. Either choice changes the pipeline and requires an index migration.
Security and operational boundaries
Both ReadApiKey and WriteApiKey belong on the server. A public search endpoint does not make a database key public. Configure actual credentials through environment variables or your deployment secret store.
The vector store constructs distinct clients. When both keys are configured, searches use the reader and mutations use the writer. A service configured with only a read key rejects writes. If only a write key is configured, reads necessarily use it too; separate settings alone don't magically create separate privileges.
A combined web process still has access to its indexing credentials. For stronger isolation, provision collections separately and run indexing in a worker with write access, leaving the query service read-only. A process boundary and database-enforced permissions provide the separation.
For private or multi-tenant content, authorisation must also constrain retrieval. A relevant result isn't necessarily a permitted result. Apply access filters using trusted application context and check permission again before returning content. A payload index is not an authorisation policy.
Do not log complete queries or document fragments by default: those can contain sensitive information. Monitor operation names, counts, durations and error categories instead.
What happens when indexing fails
Qdrant and the application database don't share a transaction. If you publish a post and then Qdrant is unavailable, publication can succeed while the retrieval index remains stale.
The library now propagates failures so a caller can retry. A robust publishing workflow should enqueue an indexing job after the database commit, retain failed work and retry it idempotently. A periodic reconciliation can compare the canonical corpus with the index, repair missing points and delete points for removed posts. Related-post caches need invalidation after successful changes too.
The blog's startup indexing and watcher integrations are useful entry points, but they aren't a durable outbox. If every update must eventually be searchable, persist the work to be done.
Language updates deserve similar care. The library serialises concurrent AddLanguageAsync calls within one service instance, but that doesn't lock other processes. In a deployment with several writers, write the authoritative complete language list from the database through a coordinated indexing job.
CPU performance without invented promises
ONNX Runtime can execute concurrent runs on a session for supported configurations; the runtime's session API documents thread-safe runs. A semaphore is therefore a resource-control decision here, a choice about CPU capacity and admission.
MaxConcurrentEmbeddings limits admitted embedding operations. IntraOpNumThreads limits CPU parallelism within an operation. Raising both can make a small VPS slower by exhausting its CPU allocation. The ONNX Runtime threading guide explains the controls; measure with the actual CPU limits of your host or container.
An async method doesn't make CPU inference free. This implementation schedules the work on the thread pool and checks cancellation between model windows. A request cancelled during a native inference run may still consume CPU until that run returns. Under sustained traffic, add bounded admission and reject or defer excess work rather than allowing an unlimited waiting queue.
The plural embedding method currently processes documents sequentially; it is not tensor batching. Qdrant upload batches and ONNX inference batches are different optimisations. True inference batching would combine several inputs in one model call and pad them to a common length.
Memory and storage
One 384-dimensional float32 vector occupies:
384 × 4 bytes = 1,536 bytes = 1.5 KiB
100,000 raw vectors ≈ 146.5 MiB
That is the vector data alone. Add payloads, the search index, allocator overhead, model weights and inference buffers. Passage-level indexing multiplies the number of vectors by the average number of passages per document. Model download size isn't the service's resident memory usage.
Qdrant commonly uses HNSW, an approximate nearest-neighbour index, to search larger collections efficiently. Approximate means the result set can trade recall for speed; small collections or particular filters may use other search strategies. Choosing Qdrant doesn't establish a universal latency or capacity figure for your workload.
Extend the same design to multimodal retrieval
MiniLM embeds text. An image caption can enter that text pipeline, but it represents what the caption says, not everything visible in the image. Native multimodal embeddings let a text query retrieve an image, audio segment or video clip in a compatible learned space.
EmbeddingGemma 2 is a concrete upgrade path: Google's model card describes a shared 768-dimensional space for text, code, images, audio and video, with an 8,192-token context. It also supports shortened 512-, 256- and 128-dimensional representations through Matryoshka Representation Learning. A shorter representation needs renormalisation and evaluation on your own material. These are properties of EmbeddingGemma 2; don't assume similarly named models have the same inputs or output contract.
Replace the encoder contract deliberately
For exploration, Google's developer guide uses Sentence Transformers 6.1 or later. Text queries and documents use different task prompts; media inputs use modality dictionaries without those text prompts:
pip install -U "sentence-transformers[image,audio,video]>=6.1.0" transformers
import torch
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"google/embeddinggemma-2",
device="cpu",
model_kwargs={"torch_dtype": torch.float32},
)
query = model.encode(
"a diagram showing a reverse proxy in front of a web application",
prompt_name="SearchQuery",
normalize_embeddings=True,
)
passage = model.encode(
"A reverse proxy forwards incoming requests to the application server.",
prompt_name="Document",
normalize_embeddings=True,
)
diagram = model.encode(
{"image": "reverse-proxy.png"}, normalize_embeddings=True,
)
clip = model.encode(
{"video": "deployment-demo.mp4"}, normalize_embeddings=True,
)
assert query.shape == passage.shape == diagram.shape == clip.shape == (768,)
This is an extension example using local media files, rather than a replacement Program.cs for the MiniLM demo. Pin the Python dependencies and model revision when turning the experiment into a deployed worker. The precision guidance specifies float32 or bfloat16 and explicitly excludes float16, which can produce invalid or degraded vectors. The example chooses float32 on CPU; it makes no latency or memory promise for your machine.
The multimodal guide explains selective loading: disable unused vision or audio encoders through config_kwargs. Loading only the encoders needed by a query worker can reduce its footprint while ingestion workers handle the richer media. All modalities share a context budget, so a larger context doesn't remove the need to segment long sources.
Carry provenance across the boundary
The storage layer already accepts precomputed vectors through ISegmentedVectorStore.IndexSegmentsAsync. EmbeddedSegment pairs a float[] with a ContentSegment; the latter holds a source URI, modality, optional page and optional start/end times as well as source text. That is the boundary between a media-processing worker and Qdrant.
For example, ingestion can produce these records:
| Source unit | Vector input | Evidence retained with the point |
|---|---|---|
| Article section | Title, heading and passage | Extracted text and character range |
| Architecture diagram | Image or image with surrounding explanation | Original asset URI, caption and parent section |
| Scanned manual page | Rendered page image | Page number, document URI and separately extracted OCR |
| Demonstration clip | A bounded video clip | Original recording URI and start/end times |
| Spoken explanation | A bounded audio clip | Recording URI, timestamps and any separately produced transcript |
An embedding doesn't produce an OCR transcript or verify a caption. Store extracted observations with their origin. Image intelligence explores keeping OCR and visual signals distinct; AudioSummarizer shows why timestamps and recoverable audio evidence matter. Captions, OCR and transcripts can also feed lexical retrieval, complementing a native media vector.
To connect this example to C#, create a new 768-dimensional cosine collection and configure a distinct pipeline identity containing the actual model revision, preprocessing and prompt rules. Send the worker's float arrays and source metadata to IndexSegmentsAsync; send queries produced by the matching model to SearchSegmentsAsync. The store validates generic vector dimensions, and its integration test exercises 768-dimensional media records and modality filters.
The default OnnxEmbeddingService remains MiniLM-specific and rejects a non-384 configuration. A live EmbeddingGemma query path therefore needs a separate IEmbeddingService adapter, or a worker/client pair that exposes document and query encoding explicitly. Don't drop different weights into the BERT adapter or mix MiniLM queries with EmbeddingGemma points. The Python example follows the published API; the full EmbeddingGemma model isn't part of this article's exercised ONNX demo.
A shared vector space makes cross-modal ranking possible, not automatically well calibrated. Evaluate text-to-image, text-to-audio and text-to-video queries separately, including distractors. If you use unrelated specialist encoders, retain separate collections or named vector spaces and fuse their result rankings instead of comparing incompatible raw scores.
Test retrieval quality as well as correctness
The semantic-search test project covers tokenisation, pooling, window boundaries, invalid vectors, cancellation, atomic downloads, index fingerprints, source ranges, filtered passage retrieval, generation visibility and bounded uploads:
dotnet test Mostlylucid.SemanticSearch.Test/Mostlylucid.SemanticSearch.Test.csproj
The live tests are opt-in. Point them at an isolated Qdrant instance and downloaded assets:
MINILM_TEST_MODEL=/absolute/path/all-MiniLM-L6-v2.onnx \
MINILM_TEST_VOCAB=/absolute/path/vocab.txt \
QDRANT_TEST_URL=http://localhost:6334 \
dotnet test Mostlylucid.SemanticSearch.Test/Mostlylucid.SemanticSearch.Test.csproj
For an authenticated test instance, also set QDRANT_TEST_READ_KEY and QDRANT_TEST_WRITE_KEY. The tests create and delete their own uniquely named collection. Never point integration tests at a collection containing your production data.
A successful smoke test should demonstrate more than “no exception”:
- Embeddings have 384 finite values and unit length.
- Related text ranks above a clearly unrelated example.
- Content beyond the first model window affects its embedding.
- Passage source ranges reproduce the supplied text exactly.
- Filters select eligible candidates before document grouping.
- Uncommitted passage generations remain invisible, and reindexing removes obsolete chunks.
- A completed write is immediately visible to the following read.
- Related results exclude their source post.
- Old-pipeline vectors are excluded even when they have the same dimension.
- Changing a title or relevant metadata changes the indexing fingerprint.
- A failed dependency is distinguishable from a genuine empty result.
Then assemble representative queries with posts you judge relevant. Include synonyms, exact identifiers, error messages, short queries, ambiguous queries and queries with no useful answer. Record how often relevant posts appear in the first ten results and how highly the first useful result ranks. Measure p50 and p95 latency separately for embedding, database search and display-data lookup, including cold start and sustained load.
For approximate search, compare a sample against exact search as well. That separates problems in the embedding representation from candidates missed by the search index.
Where semantic search needs help
Dense vectors are useful for paraphrases. Exact error codes, rare package names and version strings often benefit from lexical search too. Hybrid search retrieves both kinds of candidates and combines their rankings. Reciprocal rank fusion combines rank positions without assuming that a lexical score and a cosine score mean the same thing.
A reranker can then examine each query–candidate pair to order a small candidate set more carefully. It adds latency and cannot recover a passage that the initial retrieval never found.
Part 4b covers the UI and hybrid-search path; Part 5 follows the integration into publishing and indexing. For a deeper look at Qdrant's storage and deployment choices, see Self-Hosted Vector Databases with Qdrant.
A useful retrieval system preserves the evidence it found: the passage, the context that makes it meaningful, and the source a reader can check. Model upgrades and richer media can then improve what it finds without making that evidence harder to inspect.