|
|
--- |
|
|
datasets: |
|
|
- custom |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
pipeline_tag: text-classification |
|
|
library_name: transformers |
|
|
tags: |
|
|
- LLM |
|
|
- classification |
|
|
- instruction-tuned |
|
|
- multi-label |
|
|
- qwen |
|
|
--- |
|
|
|
|
|
# BenchHub-Cat-7b |
|
|
|
|
|
<div style="text-align: center;"> |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6576ace7769f3ee9bd7b1b88/P_Csm4Gc5eOLYcq5I__0R.png" width="200"/> |
|
|
</div> |
|
|
|
|
|
Project page: https://huggingface.co/BenchHub. |
|
|
Code: https://github.com/rladmstn1714/BenchHub |
|
|
|
|
|
**BenchHub-Cat-7b** is a category classification model based on **Qwen2.5-7B**, fine-tuned to assign natural language queries to structured category triplets: `(subject, skill, target)`. |
|
|
|
|
|
## 🔧 Model Details |
|
|
|
|
|
- **Base Model**: [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
|
|
- **Task**: Structured multi-label classification (triple: subject, skill, target) |
|
|
- **Prompting Style**: Instruction-style with expected format output |
|
|
- **Training Framework**: [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) + DeepSpeed ZeRO-3 |
|
|
|
|
|
## 🧪 Training Configuration |
|
|
|
|
|
| Hyperparameter | Value | |
|
|
|--------------------------|----------------------| |
|
|
| Sequence Length | 8192 | |
|
|
| Learning Rate | 2 × 10⁻⁵ | |
|
|
| Batch Size (Effective) | 256 | |
|
|
| Epochs | 3 | |
|
|
| Scheduler | Cosine Decay | |
|
|
| Warmup Ratio | 0.05 | |
|
|
| Optimizer | Method from [19] | |
|
|
| Trainer | DeepSpeed ZeRO-3 | |
|
|
| Hardware | 4× A6000 48GB GPUs | |
|
|
| Training Time | ~5 hours per run | |
|
|
|
|
|
## 🧠 Intended Use |
|
|
|
|
|
**Input**: Natural language question or instruction |
|
|
**Output**: Triplet `(subject, skill, target)`, such as: |
|
|
|
|
|
```yaml |
|
|
{ "subject_type": "history", |
|
|
"task_type": "reasoning", |
|
|
"target_type": "korea"} |
|
|
```` |
|
|
|
|
|
## ✨ Prompt Example |
|
|
|
|
|
``` |
|
|
### Instruction: |
|
|
Classify the following query into subject, skill, and target. |
|
|
|
|
|
### Query: |
|
|
How did Confucianism shape education in East Asia? |
|
|
|
|
|
### Output: |
|
|
{ "subject_type": "history", |
|
|
"task_type": "reasoning", |
|
|
"target_type": "korea"} |
|
|
``` |
|
|
|
|
|
## 📜 License |
|
|
|
|
|
Apache 2.0 |
