Docs-Reference: A CL RAG System for Open-Weight Models

[u/Technical-Cap5561]

Hello!

First post in this community, and I wanted to share my first completed project in Common Lisp here:

https://github.com/skarnati20/docs-reference

This is a very basic Local LLM Chat + RAG implementation, but it's more fully-featured than what I could find online. And it leverages Lisp's REPL effectively.

I built this because I found Claude/GPT overkill for lots of programming questions I had, and I needed something that could work offline. It indexes a URL you provide and lets you ask questions to a local LLM (Ollama) with that information.

Some Features

• Multiple Retrieval Methods — (BM25, Dense Vector Search, ColBERT). You can choose whichever combination of these you want.
• Agentic RAG — An Agent determines when there is enough context to answer a question. You can turn it off if you want speed.
• Fully Local Chat — Uses Ollama so you can run any model you want, so long as you serve it locally.
• Asynchronous Chat — Allows you to ask multiple questions at once which helps with multi-tasking and getting around local models' slow completion times.

Note: Local models will take time for chat completion, so it isn't as fast as Claude or ChatGPT. They might also hallucinate because they aren't as powerful either.

Acknowledgement: This project was an extension/combination of some projects in Mark Watson's Loving Common Lisp, or the Savvy Programmer's Secret Weapon. Definitely check it out since it gave me a lot of ideas for fun side projects.