README.md

metadata

title: Professional RAG Assistant
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.44.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Professional RAG system for document Q&A and search

Professional RAG Document Assistant

A comprehensive, production-ready Retrieval-Augmented Generation (RAG) system built with Gradio for document question-answering. This application combines vector similarity search, keyword search (BM25), and cross-encoder re-ranking to provide highly relevant answers from your documents.

🌟 Features

Document Processing

• Multi-format Support: PDF, DOCX, and TXT files
• Smart Chunking: Intelligent text segmentation with overlapping chunks
• Metadata Extraction: Automatic extraction of document metadata
• Progress Tracking: Real-time processing status updates

Advanced Search

• Hybrid Search: Combines vector similarity and BM25 keyword search
• Re-ranking: Cross-encoder models for improved result relevance
• Configurable: Adjustable search parameters and weights
• Metadata Filtering: Filter results by document properties

User Interface

• Professional Design: Clean, modern Gradio interface
• Multi-tab Layout: Organized workflow from upload to search
• Real-time Updates: Live progress and status indicators
• Analytics Dashboard: Usage metrics and system performance

Performance & Scalability

• Caching System: Multi-level caching for embeddings and queries
• Memory Efficient: Optimized for resource-constrained environments
• Error Handling: Comprehensive error management and recovery
• Analytics: Built-in usage tracking and performance monitoring

🚀 Quick Start

Running Locally

1. Clone the repository:

   git clone <repository-url>
   cd professional-rag-assistant
   

2. Install dependencies:

   pip install -r requirements.txt
   

3. Run the application:

   python app.py
   

4. Open your browser to http://localhost:7860

Configuration

The system uses YAML configuration files:

• config.yaml: Production configuration
• config-local.yaml: Local development overrides

Key configuration sections:

models:
  embedding:
    name: "sentence-transformers/all-MiniLM-L6-v2"
    batch_size: 32
  reranker:
    name: "cross-encoder/ms-marco-MiniLM-L-6-v2"
    enabled: true

search:
  default_k: 10
  vector_weight: 0.7
  bm25_weight: 0.3

📖 Usage Guide

1. Document Upload

• Navigate to the "📁 Document Upload" tab
• Select PDF, DOCX, or TXT files (up to 50MB each)
• Click "Process Documents" to index your files
• Monitor progress and view processing results

2. Searching Documents

• Go to the "🔍 Search" tab
• Enter your question or search query
• Configure search options:
  • Search Mode: hybrid, vector, or bm25
  • Number of Results: 1-20
  • Re-ranking: Enable for better relevance
• View results with source attribution and relevance scores

3. Document Management

• Use the "📚 Documents" tab to view indexed documents
• See document statistics (chunks, file size, type)
• Remove individual documents or clear all

4. Analytics

• Monitor system performance in the "📊 Analytics" tab
• View query patterns and usage statistics
• Track system health and resource usage

🏗️ Architecture

The system follows a modular architecture:

src/
├── rag_system.py          # Main orchestrator
├── document_processor.py  # Multi-format parsing
├── embedding_manager.py   # HF embeddings with caching
├── vector_store.py        # In-memory vector storage
├── search_engine.py       # Hybrid search implementation
├── reranker.py           # Cross-encoder re-ranking
├── cache_manager.py      # Multi-level caching
├── error_handler.py      # Error management
└── analytics.py          # Usage tracking

ui/
├── main_interface.py     # Primary Gradio interface
├── components.py         # Reusable UI components
├── themes.py            # Custom styling
└── utils.py             # UI utilities

🧪 Testing

Run the test suite:

# Install test dependencies
pip install -r requirements-dev.txt

# Run all tests
pytest

# Run with coverage
pytest --cov=src --cov-report=html

🛠️ Development

Setup Development Environment

1. Install development dependencies:

   pip install -r requirements-dev.txt
   

2. Run in development mode:

   python app.py --debug --config config-local.yaml
   

📄 License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.