e2e/README.md

End-to-End Test Suite for PromptAid Vision

This directory contains comprehensive end-to-end tests that validate the complete user experience through the entire application stack.

🎯 Overview

These are true E2E tests that:

• ✅ Hit the running app over HTTP via real browsers
• ✅ Test complete user workflows from start to finish
• ✅ Validate frontend, backend, and database integration
• ✅ Use real browser automation with Playwright
• ✅ Run against containerized services

🏗️ Architecture

e2e/
├── docker-compose.e2e.yml    # E2E environment setup
├── requirements.txt          # Python dependencies
├── pytest.ini              # Pytest configuration
├── conftest.py             # Test fixtures and setup
├── run_e2e_tests.sh        # Test runner script
├── pages/                  # Page Object Models
│   ├── base_page.py
│   ├── upload_page.py
│   ├── explore_page.py
│   └── admin_page.py
├── specs/                  # Test specifications
│   ├── upload_flow_spec.py
│   ├── admin_settings_spec.py
│   └── export_spec.py
└── fixtures/              # Test data
    └── test_image.jpg

🚀 Quick Start

Prerequisites

• Docker and Docker Compose
• Python 3.8+
• Git

Run E2E Tests

# Option 1: Use the automated script
chmod +x run_e2e_tests.sh
./run_e2e_tests.sh

# Option 2: Manual steps
docker-compose -f docker-compose.e2e.yml up -d --build
pip install -r requirements.txt
playwright install
pytest -m e2e -v
docker-compose -f docker-compose.e2e.yml down -v

🧪 Test Categories

1. Upload Flow Tests (upload_flow_spec.py)

• Complete upload workflow: File selection → Analysis → Success
• Invalid file handling: Error messages for wrong file types
• Large file handling: Performance with large images

2. Admin Settings Tests (admin_settings_spec.py)

• Authentication flow: Login/logout with correct/incorrect credentials
• Schema management: Admin interface for schema configuration
• Model configuration: VLM service configuration
• System monitoring: Health checks and monitoring

3. Export Tests (export_spec.py)

• Filtered data export: Export with applied filters
• Bulk export workflow: Export multiple selected items
• Export format validation: Different export formats
• Performance testing: Export with large datasets

🔧 Environment Setup

Docker Services

• PostgreSQL 16: Test database with health checks
• MinIO: S3-compatible storage for file uploads
• Backend: FastAPI with mock VLM provider
• Frontend: React application with Vite

Health Checks

• Backend: http://localhost:7860/health
• Frontend: http://localhost:3000
• Database: PostgreSQL connection check
• MinIO: S3 health endpoint

📊 Test Metrics

What We Measure

• Flakiness rate: Test stability and reliability
• Test duration: Median and 95th percentile times
• Critical path coverage: Key user workflows
• Failure triage speed: Debug information availability

What We Don't Measure

• ❌ Code coverage (not relevant for E2E)
• ❌ Individual test duration targets
• ❌ UI element coverage percentages

🎭 Playwright Configuration

Browser Settings

• Viewport: 1920x1080
• Video recording: Enabled for all tests
• Screenshots: On failure
• Traces: Available for debugging

Auto-wait Strategy

• No explicit sleep() calls
• Uses Playwright's built-in auto-wait
• Relies on expect().toBeVisible() assertions
• URL-based navigation verification

📁 Test Data Management

Data Isolation

• Per-test reset: /test/reset endpoint (E2E mode only)
• Volume cleanup: docker-compose down -v after suite
• Namespaced data: Unique filenames per test

Test Fixtures

• Sample images for upload testing
• Test schemas for validation
• Mock data for various scenarios

🚨 Troubleshooting

Common Issues

1. Services not ready

   # Check service health
   curl http://localhost:7860/health
   curl http://localhost:3000
   

2. Browser installation issues

   # Reinstall Playwright browsers
   playwright install
   

3. Test failures with traces

   # View test traces
   playwright show-trace test-results/trace.zip
   

Debug Mode

# Run tests with headed browser
pytest -m e2e --headed --slowmo=1000

# Run specific test with debugging
pytest specs/upload_flow_spec.py::TestUploadFlow::test_complete_upload_flow -v --headed

📈 CI/CD Integration

GitHub Actions Example

name: E2E Tests
on: [push, pull_request]
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Start E2E environment
        run: docker-compose -f e2e/docker-compose.e2e.yml up -d --build
      - name: Wait for services
        run: sleep 30
      - name: Install dependencies
        run: |
          cd e2e
          pip install -r requirements.txt
          playwright install
      - name: Run E2E tests
        run: |
          cd e2e
          pytest -m e2e -v
      - name: Upload test results
        uses: actions/upload-artifact@v3
        with:
          name: e2e-test-results
          path: e2e/test-results/

📋 Test Results

Output Locations

• Videos: test-results/videos/
• Screenshots: test-results/screenshots/
• Traces: test-results/har/
• Reports: Playwright HTML report

Success Criteria

• All critical user paths covered
• <5% flakiness rate
• <5 minutes total suite duration
• Clear failure debugging information

🔄 Maintenance

Regular Tasks

• Update test selectors when UI changes
• Refresh test data periodically
• Monitor flakiness trends
• Update dependencies

Best Practices

• Use stable data-testid selectors
• Keep page objects thin and focused
• Write descriptive test names
• Maintain test data isolation