license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
DeepSeekMath-7B-Caco
This repository hosts the DeepSeekMath-7B-Caco model, which is fine-tuned using the Caco framework. Caco (Code-Assisted Chain-of-ThOught) is a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-Chain-of-Thought (CoT) reasoning data through code-driven augmentation. It leverages executable code steps to generate comprehensive and logically correct reasoning paths, enhancing the reasoning capabilities of Large Language Models (LLMs).
The DeepSeekMath-7B-Caco model is built upon the deepseek-math-7b-base and has been trained on the Caco-1.3M dataset, achieving strong competitive performance on mathematical reasoning benchmarks, as detailed in the research paper.
For more details on the Caco framework, its methodology, and experimental results, please refer to the official paper: Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (accepted by NeurIPS 2025).
Code and Further Information
The official implementation, detailed instructions for data generation, training, and evaluation, can be found on the GitHub repository: https://github.com/LHL3341/Caco
You can also find the Caco dataset and other Caco-trained models within the Hugging Face collection: https://huggingface.co/collections/LHL3341/caco-68e0cb7b8a5f0071fac1f611
Citation
If you find this work useful, please cite the paper:
@article{caco,
title={Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning},
author={Honglin Lin and Qizhi Pei and Xin Gao and Zhuoshi Pan and Yu Li and Juntao Li and Conghui He and Lijun Wu},
journal={arXiv preprint arXiv:2510.04081},
year={2025}
}