I recently took on a coding challenge to build an end-to-end documentation assistant using RAG.

Instead of just making something that "works", the challenge pushed me to think about what happens underneath a production-ready RAG system.

I built a documentation assistant that:

- ingests and chunks documentation

- generates embeddings and stores them in Redis

- retrieves relevant context for each question

- generates grounded answers using an LLM

- uses semantic caching to avoid unnecessary LLM calls

- maintains session memory

- streams responses using SSE

The most interesting part for me wasn't getting the RAG pipeline working. It was thinking about the problems around it:

How similar does a query need to be before we can reuse a cached answer?

What information should actually be stored as memory?

How do you keep retrieval relevant as the amount of documentation grows?

And how do you design the system so that you're not blindly sending every request to an LLM?

It was a great exercise in going beyond "LLM + vector database" and thinking about the system as a whole.

I ended up building it with React, vite, nest, postgres, redis, and ts.

Still plenty I'd improve, but I'm happy with where it ended up.

https://github.com/Ramzi-Abidi/RTFM

If you're working with RAG or ai applications, I'd be interested to hear how you'd approach the caching and memory parts.