ask-node is a small CLI that knows the current Node.js API better than the model behind it does. Ask it how to read a file as a stream, and it pulls the relevant sections of the official docs, hands only those to an LLM, and answers with the source file cited, or says it doesn't know.
Why not just ask the model
LLMs have a knowledge cutoff, and API details are exactly the kind of thing they get confidently wrong: a renamed option, a method that moved modules, a flag that was experimental two versions ago. Retrieval Augmented Generation flips the job around: the model stops being the source of truth and becomes a reader of documents you hand it. Getting the right documents into that hand is the whole problem.
First, search that understands meaning
The groundwork was a semantic product search over the DummyJSON catalog. Each product is serialized into one labeled line (title, category, description, tags, brand) and the whole catalog is embedded in a single batched API call, then saved to disk so searching never re-embeds it. A query is embedded the same way and scored against every product with a dot product.
The useful lesson was the threshold. Running a query that should match ("nice smelling scent") beside one that shouldn't ("a textbook on quantum physics") showed a clear gap between real and accidental matches, and that gap set a 0.2 minimum score. Below it, the search admits it has nothing like that in stock instead of recommending a perfume to someone looking for physics.
Chunking that respects the docs
Splitting documentation every N characters cuts methods in half and separates examples from the text that explains them. The chunker splits on Markdown headings instead and keeps a stack of the headings above each section, so every chunk carries a breadcrumb like fs > Promises API > filehandle.read(), prepended to the text before it's embedded, so a short section about an option still knows which method it belongs to.
- Headings inside fenced code blocks are ignored, so a # comment in a shell example never starts a new chunk.
- HTML comments, which the Node docs use for version metadata, are stripped before chunking.
- Sections under 50 characters are dropped: a heading with nothing under it is noise in a vector store.
- Each chunk keeps its source file and breadcrumb as metadata, which is what makes citations possible later.
Retrieve wide, rerank narrow
Chroma stores the chunks with cosine distance and pulls the 25 nearest to a question. Embedding similarity is fast but blunt, so those 25 go through a Cohere reranker, which reads the question and each chunk together and scores how well one actually answers the other. Only the top 5 reach the model. It's the same retrieve-then-rerank pattern as the product search: cheap recall first, expensive precision second.
Answering, with receipts
The five chunks are wrapped in tagged blocks carrying their source file and breadcrumb, and the system prompt sets three rules: answer only from that context, say "I don't have enough information" when the context doesn't cover it, and cite the file. The sources and their rerank scores go to stderr and the answer to stdout, so the tool pipes cleanly into anything else while you can still see what it read.
What I took from it
Most of the quality in a RAG system is decided before the model is ever called. The breadcrumbs, the threshold, the rerank step: none of them are clever on their own, but each one removes a way for the wrong text to end up in the prompt. The generation step is the easy part once retrieval is honest.





