ask-node

A RAG assistant grounded in the Node.js docs

A command-line assistant that answers Node.js questions from the official API docs instead of from memory: structure-aware chunking, a Chroma vector store, retrieve-then-rerank, and answers that cite the file they came from.

Year
2026
Category
AI, Retrieval
Stack
PythonChromaDBOpenRoutertext-embedding-3-smallCohere RerankGemini Flash LiteOpenAI SDK
Note
Coursework project. The code lives in a private course repo.
ask-node

ask-node is a small CLI that knows the current Node.js API better than the model behind it does. Ask it how to read a file as a stream, and it pulls the relevant sections of the official docs, hands only those to an LLM, and answers with the source file cited, or says it doesn't know.

Why not just ask the model

LLMs have a knowledge cutoff, and API details are exactly the kind of thing they get confidently wrong: a renamed option, a method that moved modules, a flag that was experimental two versions ago. Retrieval Augmented Generation flips the job around: the model stops being the source of truth and becomes a reader of documents you hand it. Getting the right documents into that hand is the whole problem.

First, search that understands meaning

The groundwork was a semantic product search over the DummyJSON catalog. Each product is serialized into one labeled line (title, category, description, tags, brand) and the whole catalog is embedded in a single batched API call, then saved to disk so searching never re-embeds it. A query is embedded the same way and scored against every product with a dot product.

The useful lesson was the threshold. Running a query that should match ("nice smelling scent") beside one that shouldn't ("a textbook on quantum physics") showed a clear gap between real and accidental matches, and that gap set a 0.2 minimum score. Below it, the search admits it has nothing like that in stock instead of recommending a perfume to someone looking for physics.

Chunking that respects the docs

Splitting documentation every N characters cuts methods in half and separates examples from the text that explains them. The chunker splits on Markdown headings instead and keeps a stack of the headings above each section, so every chunk carries a breadcrumb like fs > Promises API > filehandle.read(), prepended to the text before it's embedded, so a short section about an option still knows which method it belongs to.

  • Headings inside fenced code blocks are ignored, so a # comment in a shell example never starts a new chunk.
  • HTML comments, which the Node docs use for version metadata, are stripped before chunking.
  • Sections under 50 characters are dropped: a heading with nothing under it is noise in a vector store.
  • Each chunk keeps its source file and breadcrumb as metadata, which is what makes citations possible later.
70
Node.js API docs indexed
5,411
chunks in the vector store
25 → 5
retrieved, then reranked
0.2
similarity floor for a real match
ask-node indexing and answeringIndexing · runs onceNode.js docs70 Markdown filesChunkersplit on headingsEmbeddingstext-embedding-3-smallChroma5,411 chunksAnswering · every questionQuestionfrom the command lineVector searchtop 25 by cosineRerankCohere · top 5Answercites the source filesame collection
Indexing happens once, offline. Every question then searches the same collection, keeps 25, and lets the reranker pick the 5 the model reads.

Retrieve wide, rerank narrow

Chroma stores the chunks with cosine distance and pulls the 25 nearest to a question. Embedding similarity is fast but blunt, so those 25 go through a Cohere reranker, which reads the question and each chunk together and scores how well one actually answers the other. Only the top 5 reach the model. It's the same retrieve-then-rerank pattern as the product search: cheap recall first, expensive precision second.

Answering, with receipts

The five chunks are wrapped in tagged blocks carrying their source file and breadcrumb, and the system prompt sets three rules: answer only from that context, say "I don't have enough information" when the context doesn't cover it, and cite the file. The sources and their rerank scores go to stderr and the answer to stdout, so the tool pipes cleanly into anything else while you can still see what it read.

What I took from it

Most of the quality in a RAG system is decided before the model is ever called. The breadcrumbs, the threshold, the rerank step: none of them are clever on their own, but each one removes a way for the wrong text to end up in the prompt. The generation step is the easy part once retrieval is honest.