|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- neptun-org/neptun.scraper |
|
|
- sequelbox/Titanium |
|
|
- MattCoddity/docker_ps |
|
|
- MattCoddity/dockerNLcommands |
|
|
language: |
|
|
- en |
|
|
- zh |
|
|
base_model: |
|
|
- BlinkDL/rwkv-5-world |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- Docker |
|
|
- Node.js |
|
|
- GitHub |
|
|
metrics: |
|
|
- code_eval |
|
|
--- |
|
|
|
|
|
# Neptun AI |
|
|
|
|
|
`RWKV-5-World-0.4B-v2-20231113-ctx4096.pth` tuned using [neptun.scraper](https://huggingface.co/datasets/neptun-org/neptun.scraper) and LoRA Finetune. |
|
|
Converted using [convert_ai00.py](https://github.com/Ai00-X/ai00_server/blob/main/assets/scripts/convert_ai00.py). |
|
|
|
|
|
* Precision: bf16 |
|
|
* Context Length: 4096 |
|
|
* Epoch Begin: 0 |
|
|
* Epoch Count: 20 |
|
|
* Epoch Steps: 1000 |
|
|
* Epoch Save: 1000 |
|
|
* Warmup Steps: 0 |
|
|
* Learning Rate Init: 2e-5 |
|
|
* Learning Rate Final: 2e-5 |
|
|
* Micro Batch Size: 1 |
|
|
* Accumulate Gradient Batches: 24 |
|
|
* Beta 1: 0,9 |
|
|
* Beta 2: 0,999 |
|
|
* LoRA R: 64 |
|
|
* LoRA Alpha: 192 |
|
|
* LoRA Dropout: 0,01 |
|
|
* Adam Epsilon: 1e-8 |
|
|
* Gradient Checkpoint: true |
|
|
* Devices: 1 |
|
|
|
|
|
## Dockerfiles |
|
|
|
|
|
```bash |
|
|
1 1.552413 4.7229 0.00002000 2025-01-28 14:47:38.809119 0 |
|
|
2 1.478129 4.3847 0.00002000 2025-01-28 15:37:13.198567 1 |
|
|
3 1.412234 4.1051 0.00002000 2025-01-28 16:26:29.863755 2 |
|
|
4 1.374447 3.9529 0.00002000 2025-01-28 17:16:15.815151 3 |
|
|
5 1.317078 3.7325 0.00002000 2025-01-28 18:06:02.229190 4 |
|
|
6 1.296596 3.6568 0.00002000 2025-01-28 18:55:40.765344 5 |
|
|
7 1.257023 3.5149 0.00002000 2025-01-28 19:45:23.963291 6 |
|
|
8 1.253057 3.5010 0.00002000 2025-01-28 20:34:39.364658 7 |
|
|
9 1.219242 3.3846 0.00002000 2025-01-28 21:23:37.893528 8 |
|
|
10 1.190037 3.2872 0.00002000 2025-01-28 22:12:28.025686 9 |
|
|
11 1.181473 3.2592 0.00002000 2025-01-28 23:02:49.004392 10 |
|
|
12 1.176299 3.2424 0.00002000 2025-01-28 23:52:07.349197 11 |
|
|
13 1.153920 3.1706 0.00002000 2025-01-29 00:42:01.340925 12 |
|
|
14 1.146656 3.1477 0.00002000 2025-01-29 01:30:47.291425 13 |
|
|
15 1.125227 3.0809 0.00002000 2025-01-29 02:19:49.021365 14 |
|
|
16 1.125680 3.0823 0.00002000 2025-01-29 03:08:48.035252 15 |
|
|
17 1.108051 3.0284 0.00002000 2025-01-29 03:57:08.758836 16 |
|
|
18 1.121515 3.0695 0.00002000 2025-01-29 04:45:28.513407 17 |
|
|
19 1.098906 3.0009 0.00002000 2025-01-29 05:33:47.381810 18 |
|
|
20 1.088252 2.9691 0.00002000 2025-01-29 06:22:06.299740 19 |
|
|
``` |
|
|
|
|
|
## V1 |
|
|
|
|
|
```bash |
|
|
0 1.328085 3.7738 0.00002000 2025-02-24 22:42:19.385036 0 |
|
|
1 1.250953 3.4937 0.00002000 2025-02-24 23:31:09.470548 1 |
|
|
2 1.220156 3.3877 0.00002000 2025-02-25 00:20:28.265176 2 |
|
|
3 1.189695 3.2861 0.00002000 2025-02-25 01:09:00.092662 3 |
|
|
4 1.173590 3.2336 0.00002000 2025-02-25 01:57:14.064583 4 |
|
|
5 1.155840 3.1767 0.00002000 2025-02-25 02:45:25.223964 5 |
|
|
6 1.146094 3.1459 0.00002000 2025-02-25 03:33:37.876671 6 |
|
|
7 1.128762 3.0918 0.00002000 2025-02-25 04:21:51.325201 7 |
|
|
8 1.119391 3.0630 0.00002000 2025-02-25 05:10:04.734031 8 |
|
|
9 1.112895 3.0432 0.00002000 2025-02-25 05:58:17.806360 9 |
|
|
10 1.109004 3.0313 0.00002000 2025-02-25 06:46:33.949976 10 |
|
|
11 1.101230 3.0079 0.00002000 2025-02-25 07:34:52.654477 11 |
|
|
12 1.092156 2.9807 0.00002000 2025-02-25 08:23:09.908699 12 |
|
|
13 1.087285 2.9662 0.00002000 2025-02-25 09:11:25.518814 13 |
|
|
14 1.081148 2.9481 0.00002000 2025-02-25 09:59:37.800531 14 |
|
|
15 1.074223 2.9277 0.00002000 2025-02-25 10:47:51.388452 15 |
|
|
16 1.074324 2.9280 0.00002000 2025-02-25 11:36:07.807562 16 |
|
|
17 1.068234 2.9102 0.00002000 2025-02-25 12:24:22.401156 17 |
|
|
18 1.059605 2.8852 0.00002000 2025-02-25 13:12:37.599124 18 |
|
|
19 1.055234 2.8726 0.00002000 2025-02-25 14:00:52.741654 19 |
|
|
``` |
|
|
|
|
|
## V2 |
|
|
|
|
|
```bash |
|
|
1 1.432728 4.1901 0.00002000 2025-03-01 18:28:38.262673 0 |
|
|
2 1.295736 3.6537 0.00002000 2025-03-01 19:17:53.931170 1 |
|
|
3 1.277158 3.5864 0.00002000 2025-03-01 20:07:08.693268 2 |
|
|
4 1.266635 3.5489 0.00002000 2025-03-01 20:56:43.106751 3 |
|
|
5 1.232101 3.4284 0.00002000 2025-03-01 21:46:04.793249 4 |
|
|
6 1.197623 3.3122 0.00002000 2025-03-01 22:36:19.047485 5 |
|
|
7 1.198684 3.3157 0.00002000 2025-03-01 23:26:23.852325 6 |
|
|
8 1.153539 3.1694 0.00002000 2025-03-02 00:15:25.319388 7 |
|
|
9 1.190393 3.2884 0.00002000 2025-03-02 01:04:04.914936 8 |
|
|
10 1.151376 3.1625 0.00002000 2025-03-02 01:52:36.610501 9 |
|
|
11 1.142265 3.1339 0.00002000 2025-03-02 02:41:09.421198 10 |
|
|
12 1.125220 3.0809 0.00002000 2025-03-02 03:29:40.905325 11 |
|
|
13 1.108536 3.0299 0.00002000 2025-03-02 04:18:13.008727 12 |
|
|
14 1.152535 3.1662 0.00002000 2025-03-02 05:06:43.459698 13 |
|
|
15 1.137459 3.1188 0.00002000 2025-03-02 05:55:18.549682 14 |
|
|
16 1.107539 3.0269 0.00002000 2025-03-02 06:43:50.482294 15 |
|
|
17 1.137236 3.1181 0.00002000 2025-03-02 07:32:25.326706 16 |
|
|
18 1.119796 3.0642 0.00002000 2025-03-02 08:20:57.625010 17 |
|
|
19 1.109311 3.0323 0.00002000 2025-03-02 09:09:29.684893 18 |
|
|
20 1.098197 2.9988 0.00002000 2025-03-02 09:58:02.055555 19 |
|
|
``` |
|
|
|
