Hugging Face's logo

Spaces:
HealthifyLK
/
vedaMD
Sleeping

App Files Community
vedaMD / docs /implementation-plan /system-redesign-and-refinement.md
sniro23's picture
sniro23
Initial commit without binary files
19aaa42
preview code
|
raw
history blame contribute delete
5.17 kB

Implementation Plan: System Redesign and Refinement

1. Background and Motivation

The current system has several issues that are negatively impacting user experience and functionality:

  1. UI/UX: The user interface is outdated and does not match the new, improved design specification.
  2. Document Ingestion: The document processing pipeline is fragmented, with processed files scattered across two separate directories (src/processed_docs and src/processed_markdown). This is causing inconsistent and incomplete data to be loaded into the retrieval system.
  3. Answer Quality: The chatbot's responses are poorly formatted. They lack professional structure, such as bullet points for lists, bolding for key terms, and clearly separated citations, making them difficult to read and interpret.

This plan outlines the necessary steps to perform a comprehensive redesign and refinement of the entire system, addressing these critical issues to create a robust, user-friendly, and reliable clinical assistant.

2. Branch Name

refactor/system-redesign-v1

3. Key Challenges and Analysis

3.1. UI Redesign

  • Challenge: The new design is provided as a static HTML file. It needs to be broken down into reusable React components and integrated with the existing Next.js application logic.
  • Analysis: The current frontend/src/app/page.tsx file contains all the necessary state management and event handlers. The core task is to replace the JSX structure and Tailwind CSS classes with those from docs/design, while ensuring all interactive elements are correctly wired.

3.2. Document Pipeline Consolidation

  • Challenge: There are two directories with processed documents, and multiple scripts referencing the old src/processed_docs location.
  • Analysis: The investigation revealed that src/groq_medical_rag.py (the current RAG implementation) exclusively uses src/processed_markdown. The src/processed_docs directory and the scripts that use it appear to be remnants of a previous implementation.
  • Solution:
    • Standardize on src/processed_markdown as the single source of truth for processed documents.
    • Purge the obsolete src/processed_docs directory and associated scripts.
    • Ensure all document processing scripts write their output to the correct location.

3.3. Answer Formatting

  • Challenge: The AI-generated answers lack structured formatting.
  • Analysis: The frontend already uses react-markdown, so it is capable of rendering markdown. The issue lies in the backend prompt sent to the LLM, which does not explicitly request markdown output.
  • Solution: The system prompt in src/groq_medical_rag.py must be updated to include specific instructions for the LLM to use markdown for lists, bolding, and citations.

4. High-level Task Breakdown

The project will be executed in the following order, focusing on one major area at a time.

Phase 1: Backend Refinement

  • Task 1.1: Consolidate Document Pipeline.
    • Success Criteria:
      • All processing scripts (enhanced_pdf_processor.py, nougat_pdf_processor.py) are confirmed to write to src/processed_markdown.
      • The src/processed_docs directory is deleted.
      • Obsolete chunking scripts referencing processed_docs are deleted.
      • The RAG system successfully builds a vector store from only the src/processed_markdown directory.
  • Task 1.2: Improve Answer Formatting.
    • Success Criteria:
      • The system prompt in src/groq_medical_rag.py is updated to explicitly request markdown formatting.
      • Queries to the chatbot from the command line (using the main function in the script) produce well-formatted markdown answers with lists, bold text, and clear source citations.

Phase 2: Frontend Redesign

  • Task 2.1: Re-implement the UI based on the new design.
    • Success Criteria:
      • The frontend/src/app/page.tsx file is refactored to match the layout and styling of docs/design.
      • The UI is broken down into logical React components.
      • The application is fully interactive: users can send messages, receive responses, and the chat history is displayed correctly.
      • The new UI is responsive and looks good on different screen sizes.

Phase 3: Final Integration and Testing

  • Task 3.1: End-to-End Testing.
    • Success Criteria:
      • A full user flow test is completed: asking a question from the new UI, receiving a well-formatted answer from the refined backend, and seeing it displayed correctly.
      • The application is stable and free of critical bugs.

5. Project Status Board

  • Phase 1: Backend Refinement
    • Task 1.1: Consolidate Document Pipeline.
    • Task 1.2: Improve Answer Formatting.
  • Phase 2: Frontend Redesign
    • Task 2.1: Re-implement the UI based on the new design.
  • Phase 3: Final Integration and Testing
    • Task 3.1: End-to-End Testing.

6. Executor's Feedback or Assistance Requests

This section will be updated by the Executor during implementation.