# Local RAG Chatbot with Ollama

This guide demonstrates how to build a **Retrieval-Augmented Generation (RAG)** chatbot that runs entirely locally. It loads a directory of Markdown (`.md`) files, creates a searchable vector index using local embeddings, and uses a local LLM (via Ollama) to answer questions based on your data.

**Modules Used:**
*   `langchain`: The framework for building LLM applications.
*   `langchain-community`: Community integrations (includes Ollama support).
*   `langchain-chroma`: Integration for the Chroma vector database.
*   `ollama`: The local LLM runner.

## Prerequisites

1.  **Install Ollama**: Download and install Ollama from ollama.com.
2.  **Pull Models**: Open your terminal and pull the LLM and the embedding model:
    ```bash
    ollama pull llama3
    ollama pull nomic-embed-text
    ```
3.  **Data**: A folder containing `.md` files (e.g., `docs/` or `notes/`).

## Installation

```bash
pip install langchain langchain-community langchain-chroma
```

## The Code

Save this as `local_doc_bot.py`.

```python
import argparse
import sys
import os

# LangChain Imports
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.chat_models import ChatOllama
from langchain_chroma import Chroma
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

def create_knowledge_base(directory):
    print(f"Loading .md files from '{directory}'...")
    
    if not os.path.exists(directory):
        print(f"Error: Directory '{directory}' not found.")
        return None

    # Load documents
    loader = DirectoryLoader(directory, glob="**/*.md", loader_cls=TextLoader)
    docs = loader.load()
    
    if not docs:
        print("No markdown files found.")
        return None

    print(f"Loaded {len(docs)} documents. Splitting text...")
    
    # Split documents into chunks for embedding
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    splits = text_splitter.split_documents(docs)
    
    print(f"Created {len(splits)} chunks. Building vector store (this may take a moment)...")
    
    # Create Vector Store using local embeddings (nomic-embed-text is efficient)
    vectorstore = Chroma.from_documents(documents=splits, embedding=OllamaEmbeddings(model="nomic-embed-text"))
    return vectorstore

def start_chat(vectorstore):
    # Initialize Local LLM
    llm = ChatOllama(model="llama3")
    
    # Create Retrieval Chain
    prompt = ChatPromptTemplate.from_template("""
    Answer the following question based only on the provided context:

    <context>
    {context}
    </context>

    Question: {input}
    """)

    document_chain = create_stuff_documents_chain(llm, prompt)
    retriever = vectorstore.as_retriever()
    retrieval_chain = create_retrieval_chain(retriever, document_chain)

    print("\n--- Local Doc Bot Ready (Type 'quit' to exit) ---")
    while True:
        query = input("\nQuestion: ")
        if query.lower() in ["quit", "exit"]:
            break
        
        print("Thinking...", end="", flush=True)
        response = retrieval_chain.invoke({"input": query})
        # Clear "Thinking..." line
        print("\r" + " " * 20 + "\r", end="")
        print(f"Answer: {response['answer']}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Local RAG Chatbot with Ollama")
    parser.add_argument("directory", help="Directory containing .md files")
    
    args = parser.parse_args()
    
    vs = create_knowledge_base(args.directory)
    if vs:
        start_chat(vs)
```

## Usage

1.  **Prepare Data**: Ensure you have a folder (e.g., `my_docs`) with some Markdown files inside.
2.  **Run the Bot**:
    ```bash
    python local_doc_bot.py ./my_docs
    ```
3.  **Ask Questions**: The bot will answer based *only* on the information found in your Markdown files, running completely offline.

[[programming/python/python]]