|
|
--- |
|
|
title: Voice-Activated RAG System |
|
|
emoji: 🗣️ |
|
|
colorFrom: indigo |
|
|
colorTo: blue |
|
|
sdk: gradio |
|
|
sdk_version: 5.34.2 |
|
|
app_file: main.py |
|
|
pinned: false |
|
|
--- |
|
|
# Voice to LLM & Sentiment Analyzer with RAG |
|
|
|
|
|
## Overview |
|
|
This project is a voice-driven application that integrates various machine learning models and APIs to provide sentiment analysis and context retrieval from uploaded PDF documents. It utilizes voice input to query the system, processes the audio using Whisper for transcription, retrieves relevant context from the uploaded PDFs using a Retrieval-Augmented Generation (RAG) approach, and provides a spoken audio output of the LLM response using Gemini TTS (Google GenAI API). |
|
|
|
|
|
## Features |
|
|
- Voice input for user queries |
|
|
- PDF document processing and context retrieval |
|
|
- Sentiment analysis using a pre-trained emotion classification model |
|
|
- Text-to-Speech output for the model's responses (using Gemini TTS) |
|
|
- Integration with the Groq API for advanced language model capabilities |
|
|
|
|
|
## Modular Structure |
|
|
|
|
|
This project is organized into modular components for maintainability and clarity: |
|
|
|
|
|
- `main.py`: Entry point to run the app. |
|
|
- `ui.py`: Gradio UI layout and event wiring. |
|
|
- `tts_gemini.py`: Gemini TTS logic (text-to-speech). |
|
|
- `emotion.py`: Emotion detection and tone mapping. |
|
|
- `rag.py`: PDF processing, chunking, embedding, FAISS, and context retrieval. |
|
|
- `llm.py`: LLM prompt construction and response logic. |
|
|
|
|
|
## Setup Instructions |
|
|
1. Clone the repository: |
|
|
``` |
|
|
git clone <repository-url> |
|
|
cd GoComet-C4 |
|
|
``` |
|
|
|
|
|
2. Run the setup script to create a virtual environment and install dependencies (Linux/macOS): |
|
|
``` |
|
|
bash setup.sh |
|
|
``` |
|
|
On Windows, run each command in `setup.sh` manually in your terminal: |
|
|
``` |
|
|
python -m venv venv |
|
|
venv\Scripts\activate |
|
|
pip install -r requirements.txt |
|
|
set GROQ_API_KEY="<your-groq-api-key>" |
|
|
set GEMINI_API_KEY="<your-gemini-api-key>" |
|
|
``` |
|
|
|
|
|
3. Activate the virtual environment: |
|
|
- Linux/macOS: |
|
|
``` |
|
|
source venv/bin/activate |
|
|
``` |
|
|
- Windows: |
|
|
``` |
|
|
venv\Scripts\activate |
|
|
``` |
|
|
|
|
|
## Usage |
|
|
To run the application, execute the following command: |
|
|
``` |
|
|
python main.py |
|
|
``` |
|
|
|
|
|
Once the application is running, you can upload PDF files and use the microphone to speak your queries. The application will process the audio, retrieve context from the PDFs, analyze sentiment, and provide the LLM output, sentiment, transcript, context, and a spoken audio response (.wav) in the interface. TTS audio files will be saved in a `tts_outputs` directory in your project root. |
|
|
|
|
|
## Dependencies |
|
|
This project requires the following Python libraries: |
|
|
- gradio |
|
|
- whisper |
|
|
- groq |
|
|
- transformers |
|
|
- PyPDF2 |
|
|
- sentence-transformers |
|
|
- faiss-cpu |
|
|
- soundfile |
|
|
- numpy |
|
|
- google-genai |
|
|
|
|
|
Install these dependencies using the `requirements.txt` file provided in the project. |
|
|
|
|
|
## Latency Logging |
|
|
|
|
|
After each run, the latency (processing time in seconds) for each pipeline component is logged in `logs/latency_log.csv`: |
|
|
|
|
|
| Whisper STT (s) | Document Retrieval (s) | Sentiment Analysis (s) | Response Gen (LLM) (s) | TTS Synthesis (s) | Total (s) | |
|
|
|-----------------|-----------------------|-----------------------|------------------------|-------------------|-----------| |
|
|
|
|
|
This file accumulates results from all runs, allowing you to analyze and monitor performance over time. |
