logs.txt

NCHMARK_COLS: ['Perplexity']
=== END COLUMN SETUP ===
🔧 CHECKING MODEL TRACING AVAILABILITY...
   - Model tracing path: /home/user/app/src/evaluation/../../model-tracing
   - Path exists: True
   - main.py exists: True
🎯 Final MODEL_TRACING_AVAILABLE = True

.gitattributes:   0%|          | 0.00/2.46k [00:00<?, ?B/s]
.gitattributes: 100%|██████████| 2.46k/2.46k [00:00<00:00, 10.1MB/s]

(…)therAI_gpt-neo-1.3B_20250726_010247.json:   0%|          | 0.00/202 [00:00<?, ?B/s]
(…)therAI_gpt-neo-1.3B_20250726_010247.json: 100%|██████████| 202/202 [00:00<00:00, 748kB/s]

(…)s_facebook_opt-125m_20250726_020655.json:   0%|          | 0.00/205 [00:00<?, ?B/s]
(…)s_facebook_opt-125m_20250726_020655.json: 100%|██████████| 205/205 [00:00<00:00, 909kB/s]

(…)s_facebook_opt-350m_20250726_021737.json:   0%|          | 0.00/205 [00:00<?, ?B/s]
(…)s_facebook_opt-350m_20250726_021737.json: 100%|██████████| 205/205 [00:00<00:00, 850kB/s]

(…)ommunity_gpt2-large_20250726_013038.json:   0%|          | 0.00/214 [00:00<?, ?B/s]
(…)ommunity_gpt2-large_20250726_013038.json: 100%|██████████| 214/214 [00:00<00:00, 1.03MB/s]

(…)mmunity_gpt2-medium_20250726_015555.json:   0%|          | 0.00/216 [00:00<?, ?B/s]
(…)mmunity_gpt2-medium_20250726_015555.json: 100%|██████████| 216/216 [00:00<00:00, 730kB/s]

(…)enai-community_gpt2_20250725_231201.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250725_231201.json: 100%|██████████| 209/209 [00:00<00:00, 533kB/s]

(…)enai-community_gpt2_20250725_233155.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250725_233155.json: 100%|██████████| 209/209 [00:00<00:00, 905kB/s]

(…)enai-community_gpt2_20250725_235115.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250725_235115.json: 100%|██████████| 209/209 [00:00<00:00, 801kB/s]

(…)enai-community_gpt2_20250725_235748.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250725_235748.json: 100%|██████████| 209/209 [00:00<00:00, 856kB/s]

(…)enai-community_gpt2_20250726_000358.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250726_000358.json: 100%|██████████| 209/209 [00:00<00:00, 696kB/s]

(…)enai-community_gpt2_20250726_000650.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250726_000650.json: 100%|██████████| 209/209 [00:00<00:00, 792kB/s]

(…)enai-community_gpt2_20250726_015147.json:   0%|          | 0.00/209 [00:00<?, ?B/s]
(…)enai-community_gpt2_20250726_015147.json: 100%|██████████| 209/209 [00:00<00:00, 1.12MB/s]

🚀 STARTING GRADIO APP INITIALIZATION
📊 Initializing allowed models...

🚀 INITIALIZING ALLOWED MODELS
📋 Models to initialize: ['lmsys/vicuna-7b-v1.5', 'ibm-granite/granite-7b-base', 'EleutherAI/llemma_7b']

🧹 CLEANING NON-ALLOWED RESULT FILES
🗑️ Removing non-allowed model result: ./eval-results/EleutherAI/results_EleutherAI_gpt-neo-1.3B_20250726_010247.json (model: EleutherAI/gpt-neo-1.3B)
🗑️ Removing non-allowed model result: ./eval-results/facebook/results_facebook_opt-125m_20250726_020655.json (model: facebook/opt-125m)
🗑️ Removing non-allowed model result: ./eval-results/facebook/results_facebook_opt-350m_20250726_021737.json (model: facebook/opt-350m)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2-large_20250726_013038.json (model: openai-community/gpt2-large)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2-medium_20250726_015555.json (model: openai-community/gpt2-medium)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250725_231201.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250725_233155.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250725_235115.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250725_235748.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250726_000358.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250726_000650.json (model: openai-community/gpt2)
🗑️ Removing non-allowed model result: ./eval-results/openai-community/results_openai-community_gpt2_20250726_015147.json (model: openai-community/gpt2)
✅ Removed 12 non-allowed result files

🔧 CREATING RESULT FILE FOR: lmsys/vicuna-7b-v1.5
📁 Result file path: ./eval-results/lmsys_vicuna_7b_v1.5_float16.json
✅ Created result file: ./eval-results/lmsys_vicuna_7b_v1.5_float16.json

🔧 CREATING RESULT FILE FOR: ibm-granite/granite-7b-base
📁 Result file path: ./eval-results/ibm_granite_granite_7b_base_float16.json
✅ Created result file: ./eval-results/ibm_granite_granite_7b_base_float16.json

🔧 CREATING RESULT FILE FOR: EleutherAI/llemma_7b
📁 Result file path: ./eval-results/EleutherAI_llemma_7b_float16.json
✅ Created result file: ./eval-results/EleutherAI_llemma_7b_float16.json
✅ Initialized 3 model result files
📊 Creating initial results DataFrame...

📊 CREATE_RESULTS_DATAFRAME CALLED

=== GET_LEADERBOARD_DF DEBUG ===
Starting leaderboard creation...
Looking for results in: ./eval-results
Expected columns: ['T', 'Model', 'Average ⬆️', 'Perplexity', 'Match P-Value ⬇️', 'Type', 'Architecture', 'Precision', 'Hub License', '#Params (B)', 'Hub ❤️', 'Available on the hub', 'Model sha']
Benchmark columns: ['Perplexity']

Searching for result files in: ./eval-results
Found 0 result files

Processing 0 evaluation results

Returning 0 processed results

Found 0 raw results
No raw data found, creating empty DataFrame
Creating empty fallback DataFrame...
Empty DataFrame created with columns: ['T', 'Model', 'Average ⬆️', 'Perplexity', 'Match P-Value ⬇️', 'Type', 'Architecture', 'Precision', 'Hub License', '#Params (B)', 'Hub ❤️', 'Available on the hub', 'Model sha']
📋 Retrieved leaderboard df: (0, 13)
⚠️ DataFrame is None or empty, returning empty DataFrame
✅ Initial DataFrame created with shape: (0, 6)
📋 Columns: ['Model', 'Perplexity', 'Match P-Value', 'Average Score', 'Type', 'Precision']
🎨 Creating Gradio interface...
🎯 GRADIO INTERFACE SETUP COMPLETE
🚀 LAUNCHING GRADIO APP WITH MODEL TRACING INTEGRATION
📊 Features enabled:
   - Perplexity evaluation
   - Model trace p-value computation (vs GPT-2 base)
   - Match statistic with alignment
🎉 Ready to accept requests!
* Running on local URL:  http://0.0.0.0:7860, with SSR ⚡ (experimental, to disable set `ssr=False` in `launch()`)