|
|
--- |
|
|
license: cc |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- Qwen/Qwen2.5-3B |
|
|
tags: |
|
|
- qwen2 |
|
|
- qwen |
|
|
- text-generation |
|
|
- question-answering |
|
|
- research |
|
|
- engineering |
|
|
- lora |
|
|
- 4bit |
|
|
- bitsandbytes |
|
|
- faiss |
|
|
- rag |
|
|
metrics: |
|
|
- type: rougeL |
|
|
value: 57.2 |
|
|
- type: bleu |
|
|
value: 42.8 |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
# 🛰️ ResearchQwen 2.5-3B-LoRA |
|
|
|
|
|
**Compact, domain-expert Q&A for systems researchers.** |
|
|
Base model: [Qwen/Qwen2.5-3B](https://huggingface.co/Qwen/Qwen2.5-3B) |
|
|
Tuning recipe: 4-bit **QLoRA** with **bitsandbytes** NF4 quantisation |
|
|
Retriever: FAISS cosine-similarity store for ~33 k document chunks |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Quick inference |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline |
|
|
|
|
|
model_id = "Programmer-RD-AI/ResearchQwen2.5-3B-LoRA" |
|
|
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
|
model_id, |
|
|
device_map="auto", |
|
|
torch_dtype="auto", |
|
|
load_in_4bit=True, # uses bitsandbytes |
|
|
) |
|
|
qa = pipeline("text-generation", model=model, tokenizer=tok) |
|
|
print(qa("Explain how Chain Replication with Apportioned Queries improves tail-latency.")) |
|
|
```` |
|
|
|
|
|
### llama.cpp / GGUF |
|
|
|
|
|
```bash |
|
|
wget https://huggingface.co/Programmer-RD-AI/ResearchQwen2.5-3B-LoRA/resolve/main/model_Q4_K_M.gguf |
|
|
./main -m model_Q4_K_M.gguf -p "Give the core idea of the 3FS log-structured layout in 3 sentences." |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 Training data |
|
|
|
|
|
| Source | Docs | Words | |
|
|
| -------------------------- | ------ | --------- | |
|
|
| 3FS white-paper | 14 | 162 k | |
|
|
| CRAQ spec + benchmarks | 11 | 119 k | |
|
|
| Distributed AI infra notes | 32 | 287 k | |
|
|
| *Total* | **57** | **568 k** | |
|
|
|
|
|
Synthetic Q\&A pairs were generated with an instruction template tuned for factual density; unhelpful pairs were filtered via a weak-to-strong scoring cascade (ROUGE-L > 0.4, BLEU > 0.35) ([GitHub][1]). |
|
|
|
|
|
--- |
|
|
|
|
|
## 🛠️ Fine-tuning details |
|
|
|
|
|
| Setting | Value | |
|
|
| --------- | ---------------------------------------------------------- | |
|
|
| GPU | 1× A100 40 GB | |
|
|
| Precision | 4-bit NF4 w/ double-quant (bnb 0.45.4) | |
|
|
| LoRA r/α | 64 / 16 | |
|
|
| LR sched | cosine, 5 % warm-up | |
|
|
| Steps | 1 100 | |
|
|
| Epochs | 3 | |
|
|
| Peak VRAM | 21 GB | |
|
|
|
|
|
--- |
|
|
|
|
|
## 📈 Evaluation |
|
|
|
|
|
| Metric | Base Qwen2.5-3B | **This model** | |
|
|
| ------- | --------------- | -------------- | |
|
|
| ROUGE-L | 45.6 | **57.2** | |
|
|
| BLEU-4 | 30.4 | **42.8** | |
|
|
|
|
|
> See `eval/` for scripts and raw scores (ROUGE, BLEU). |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔗 Integration recipe (RAG) |
|
|
|
|
|
```python |
|
|
from langchain.vectorstores import FAISS # or llama-index |
|
|
from langchain.embeddings import HuggingFaceEmbeddings |
|
|
|
|
|
emb = HuggingFaceEmbeddings(model_name="BAAI/bge-small-en-v1.5") |
|
|
vs = FAISS.from_texts(texts, emb) |
|
|
``` |
|
|
|
|
|
Retriever-generator latency: 330 ms average (GPU), 1.9 s average (CPU, gguf-int4). |
|
|
|
|
|
--- |
|
|
|
|
|
## 💡 Why it should trend |
|
|
|
|
|
* **Fresh domain niche** – deep systems-engineering Q\&A is underserved on HF. |
|
|
* **Ultra-portable** – 4-bit LoRA + GGUF = laptop-friendly. |
|
|
* **Full stack repo** – weights, notebook, RAG demo, eval scripts. |
|
|
* **Eye-catching tags** – `qwen2`, `lora`, `rag`, `research` map directly to popular HF filters and the trending feed ([Hugging Face][4]). |
|
|
* **Clear usage code** – copy-run experience = more downloads. |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚠️ Limitations & responsible use |
|
|
|
|
|
* Trained solely on English; non-English queries degrade sharply. |
|
|
* Answers may quote or paraphrase the training docs verbatim. |
|
|
* Not suitable for critical medical / legal advice. |
|
|
* LoRA adapters are GPL-3.0; commercial use must comply with both GPL-3.0 and the Qwen 2.5 base license. |
|
|
|
|
|
--- |
|
|
|
|
|
## ✍️ Citation |
|
|
|
|
|
```bibtex |
|
|
@misc{ranuga_disansa_gamage_2025, |
|
|
author = { Ranuga Disansa Gamage and Rivindu Ashinsa and Thuan Naheem and Sanila Wijesekara }, |
|
|
title = { ResearchQwen-2.5-3B-LoRA (Revision 7ea9f5f) }, |
|
|
year = 2025, |
|
|
url = { https://huggingface.co/Programmer-RD-AI/ResearchQwen-2.5-3B-LoRA }, |
|
|
doi = { 10.57967/hf/5623 }, |
|
|
publisher = { Hugging Face } |
|
|
} |
|
|
``` |
