main releases
Collection
small models aiming at language modeling without system prompts
•
3 items
•
Updated
•
2
You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
You agree to not use this model (or future versions) to conduct experiments that cause harm to any person or group.
Log in or Sign Up to review the conditions and access this model content.
In this repository, we propose the next iteration of arco, a new meta-learner small language model. Now with qwen as the base architecture for improvements.
During previous research, we first noticed a dramatic underpeformance on fewshot prompting from previous arco series (regardless of benchmark improvements on arc) so we decided that the main concept to work on was making a more robust fewshot learning by focusing directly on tasks that improve that skill with a stronger baseline model like qwen family.
After several merging iterations with some openly available models, we finally achieved a strong baseline for a meta-learner model which we called arco-3. This model will serve as the starting point for future fewshot finetunings and experiments.
prompt
There is no prompt intentionally set.
benchmarks
meta arena
We tested around 65 models against each other with fewshot tasks and used gemini-2.5-pro to chose the best answers from each one. Currently, it ranks 13th in meta-arena.
variance
We also tested the model against some popular small models on "power" distribution for our 5 typically chosen language modeling benchmarks.
language modeling
To our surprise, this model also improved some language modeling tasks over the base model on several well-known benchmarks.
| Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average |
|---|---|---|---|---|---|---|---|
| 0.6b | qwen 3 | 40.31 | 34.47 | 47.38 | 67.46 | 56.04 | 49.13 |
| 0.6b | arco 3 | 43.34 | 36.01 | 49.56 | 68.17 | 58.09 | 51.03 |
limitations
The model also comes with several limitations that shares with its base model:
- Lack of creative outputs
- Poor causality understanding
- Extremely bad summarization skills
- Hallucinations
We have a plan to tackle each one of these issues and are already planned to be corrected in the future.
supporters
trivia
arco means "bow" in spanish, which is just another way to say that hits its target fast and accurately.
Note: the model has not been tested as a chat assistant and it might not work as intended, use with caution.
- Downloads last month
- 29