AICodeKing connects the open-source Qwen Audio Agent to Claude Code so he can keep a voice conversation going, interrupt answers and change requirements while a VoiceDesk app is built in the background.
Build a local RAG system from scratch in 50 lines of Python
Tim builds a retrieval-augmented Q&A bot over a markdown handbook using only Python, numpy, Ollama, Nomic embeddings and Qwen3 4B, all running free on his own laptop.
Tech With Tim · YouTube · published Oct 30, 2026 · Software & DevOps
▶ Watch on YouTubePrompt
Write a plain-Python RAG script (only requests and numpy) that uses Ollama's nomic-embed-text for embeddings and qwen3:4b-instruct for answers: split every .md file in docs/ into paragraph chunks, embed them, retrieve the top 3 by cosine similarity for each question, and answer using only that context.
Visual breakdown
Tim builds RAG with no framework and no vector database: split a handbook into paragraph chunks, embed them with a small local model, find the closest chunks to a question by cosine similarity, and paste them into the prompt for a local chat model. It runs free on your own machine in under 50 lines.
The problem: A normal AI model can't answer questions about private documents (like a cafe's alarm code) because they were never in its training data, so it refuses or makes something up.
How the RAG loop answers a question
- Handbook markdownA fake cafe's employee handbook in docs/handbook.md, one topic per paragraph
- ChunkSplit on blank lines: 10 paragraphs plus the title give 11 chunks
- Embed chunksNomic embed text (274MB) turns each chunk into 768 numbers
- Cosine searchEmbed the question, compare to every chunk with numpy, keep the top k (3)Dropped: All the less similar chunks
- Build promptAnswer using only the context below, then the chunks, then the question
- Qwen3 4BLocal chat model reads the prompt and writes the answer
- Grounded answerFor example: dogs are only allowed on the patio
Your documentsPlain PythonLocal modelResult
The three steps of RAG
- Retrieval: search the documents for pieces related to the question
- Augmentation: add those pieces to the prompt
- Generation: the model reads the prompt and writes the answer
By the numbers
< 50lines of Python
274MBembedding model
~2.5GBchat model
768numbers per embedding
Build it yourself
Everything runs locally and free, with just requests and numpy installed through uv.
- Install Ollama, then run: ollama pull nomic-embed-text and ollama pull for the Qwen3 4B instruct model
- Run uv init in a new folder and add requests and numpy
- Write embed() and ask_llm() functions that call Ollama's local server on localhost:11434
- Put your markdown in docs/ and split each file on blank lines into chunks
- Embed all chunks, then write retrieve(question, k) using cosine similarity
- Build the prompt from the top chunks and the question, and loop asking questions
Cloud models from OpenAI or Anthropic also work if your computer can't run the local ones.
What to remember
- The model learns nothing new; you hand it the right notes just before it answers, like an open-book test.
- Small chunks keep matches precise and the prompt short, so the model is less likely to make mistakes.
- Your chunking strategy affects how accurate the answers are.
- Production systems add a vector database (PG vector or Chroma), smarter overlapping chunks, reranking and evals.
Jump to the part you need
Distilled from the video’s captions
How to try it
- Open Ollama.
- Copy the prompt and fill in the [placeholders].
- Watch Tech With Tim's video for the full walkthrough and setup.
Related use cases
Leon van Zyl logs and labels GitHub issues with dependencies for every piece of work, then tells Claude Code to read the outstanding issues and implement them one by one.
Igor from The AI Advantage points ChatGPT (or Codex/Claude) at the open-source Reverse Engineer Anything repo, inside a project with folder access, and asks it to make a version of an app he paid for. He recreated Screen Brush as AIA Ink and a Slido clone.
Income Stream Surfers shares a free Claude Code prompt that audits a repo, mocks up a cleaner UI for approval, regroups menus into 4-7 areas, and ships it behind a feature flag.