README.md

---
title: Professional RAG Assistant
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.44.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Professional RAG system for document Q&A and search
---

# Professional RAG Document Assistant

A comprehensive, production-ready Retrieval-Augmented Generation (RAG) system built with Gradio for document question-answering. This application combines vector similarity search, keyword search (BM25), and cross-encoder re-ranking to provide highly relevant answers from your documents.

## 🌟 Features

### Document Processing
- **Multi-format Support**: PDF, DOCX, and TXT files
- **Smart Chunking**: Intelligent text segmentation with overlapping chunks
- **Metadata Extraction**: Automatic extraction of document metadata
- **Progress Tracking**: Real-time processing status updates

### Advanced Search
- **Hybrid Search**: Combines vector similarity and BM25 keyword search
- **Re-ranking**: Cross-encoder models for improved result relevance
- **Configurable**: Adjustable search parameters and weights
- **Metadata Filtering**: Filter results by document properties

### User Interface
- **Professional Design**: Clean, modern Gradio interface
- **Multi-tab Layout**: Organized workflow from upload to search
- **Real-time Updates**: Live progress and status indicators
- **Analytics Dashboard**: Usage metrics and system performance

### Performance & Scalability
- **Caching System**: Multi-level caching for embeddings and queries
- **Memory Efficient**: Optimized for resource-constrained environments
- **Error Handling**: Comprehensive error management and recovery
- **Analytics**: Built-in usage tracking and performance monitoring

## 🚀 Quick Start

### Running Locally

1. **Clone the repository**:
   ```bash
   git clone <repository-url>
   cd professional-rag-assistant
   ```

2. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

3. **Run the application**:
   ```bash
   python app.py
   ```

4. **Open your browser** to `http://localhost:7860`

### Configuration

The system uses YAML configuration files:
- `config.yaml`: Production configuration
- `config-local.yaml`: Local development overrides

Key configuration sections:
```yaml
models:
  embedding:
    name: "sentence-transformers/all-MiniLM-L6-v2"
    batch_size: 32
  reranker:
    name: "cross-encoder/ms-marco-MiniLM-L-6-v2"
    enabled: true

search:
  default_k: 10
  vector_weight: 0.7
  bm25_weight: 0.3
```

## 📖 Usage Guide

### 1. Document Upload
- Navigate to the "📁 Document Upload" tab
- Select PDF, DOCX, or TXT files (up to 50MB each)
- Click "Process Documents" to index your files
- Monitor progress and view processing results

### 2. Searching Documents
- Go to the "🔍 Search" tab
- Enter your question or search query
- Configure search options:
  - **Search Mode**: hybrid, vector, or bm25
  - **Number of Results**: 1-20
  - **Re-ranking**: Enable for better relevance
- View results with source attribution and relevance scores

### 3. Document Management
- Use the "📚 Documents" tab to view indexed documents
- See document statistics (chunks, file size, type)
- Remove individual documents or clear all

### 4. Analytics
- Monitor system performance in the "📊 Analytics" tab
- View query patterns and usage statistics
- Track system health and resource usage

## 🏗️ Architecture

The system follows a modular architecture:

```
src/
├── rag_system.py          # Main orchestrator
├── document_processor.py  # Multi-format parsing
├── embedding_manager.py   # HF embeddings with caching
├── vector_store.py        # In-memory vector storage
├── search_engine.py       # Hybrid search implementation
├── reranker.py           # Cross-encoder re-ranking
├── cache_manager.py      # Multi-level caching
├── error_handler.py      # Error management
└── analytics.py          # Usage tracking

ui/
├── main_interface.py     # Primary Gradio interface
├── components.py         # Reusable UI components
├── themes.py            # Custom styling
└── utils.py             # UI utilities
```

## 🧪 Testing

Run the test suite:
```bash
# Install test dependencies
pip install -r requirements-dev.txt

# Run all tests
pytest

# Run with coverage
pytest --cov=src --cov-report=html
```

## 🛠️ Development

### Setup Development Environment

1. **Install development dependencies**:
   ```bash
   pip install -r requirements-dev.txt
   ```

2. **Run in development mode**:
   ```bash
   python app.py --debug --config config-local.yaml
   ```

## 📄 License

This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.