File size: 4,565 Bytes
7383745 f367775 7383745 0550c3c 7383745 57000a2 0c172e0 1901362 6cf0f23 2d33eb7 74cb74d 2d33eb7 74cb74d 2d33eb7 74cb74d 2d33eb7 74cb74d e5639f4 0c172e0 e5639f4 0c172e0 798caa9 d6d85b8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
---
license: apache-2.0
datasets:
- neptun-org/neptun.scraper
- sequelbox/Titanium
- MattCoddity/docker_ps
- MattCoddity/dockerNLcommands
language:
- en
- zh
base_model:
- BlinkDL/rwkv-5-world
pipeline_tag: text-generation
tags:
- Docker
- Node.js
- GitHub
metrics:
- code_eval
---
# Neptun AI
`RWKV-5-World-0.4B-v2-20231113-ctx4096.pth` tuned using [neptun.scraper](https://huggingface.co/datasets/neptun-org/neptun.scraper) and LoRA Finetune.
Converted using [convert_ai00.py](https://github.com/Ai00-X/ai00_server/blob/main/assets/scripts/convert_ai00.py).
* Precision: bf16
* Context Length: 4096
* Epoch Begin: 0
* Epoch Count: 20
* Epoch Steps: 1000
* Epoch Save: 1000
* Warmup Steps: 0
* Learning Rate Init: 2e-5
* Learning Rate Final: 2e-5
* Micro Batch Size: 1
* Accumulate Gradient Batches: 24
* Beta 1: 0,9
* Beta 2: 0,999
* LoRA R: 64
* LoRA Alpha: 192
* LoRA Dropout: 0,01
* Adam Epsilon: 1e-8
* Gradient Checkpoint: true
* Devices: 1
## Dockerfiles
```bash
1 1.552413 4.7229 0.00002000 2025-01-28 14:47:38.809119 0
2 1.478129 4.3847 0.00002000 2025-01-28 15:37:13.198567 1
3 1.412234 4.1051 0.00002000 2025-01-28 16:26:29.863755 2
4 1.374447 3.9529 0.00002000 2025-01-28 17:16:15.815151 3
5 1.317078 3.7325 0.00002000 2025-01-28 18:06:02.229190 4
6 1.296596 3.6568 0.00002000 2025-01-28 18:55:40.765344 5
7 1.257023 3.5149 0.00002000 2025-01-28 19:45:23.963291 6
8 1.253057 3.5010 0.00002000 2025-01-28 20:34:39.364658 7
9 1.219242 3.3846 0.00002000 2025-01-28 21:23:37.893528 8
10 1.190037 3.2872 0.00002000 2025-01-28 22:12:28.025686 9
11 1.181473 3.2592 0.00002000 2025-01-28 23:02:49.004392 10
12 1.176299 3.2424 0.00002000 2025-01-28 23:52:07.349197 11
13 1.153920 3.1706 0.00002000 2025-01-29 00:42:01.340925 12
14 1.146656 3.1477 0.00002000 2025-01-29 01:30:47.291425 13
15 1.125227 3.0809 0.00002000 2025-01-29 02:19:49.021365 14
16 1.125680 3.0823 0.00002000 2025-01-29 03:08:48.035252 15
17 1.108051 3.0284 0.00002000 2025-01-29 03:57:08.758836 16
18 1.121515 3.0695 0.00002000 2025-01-29 04:45:28.513407 17
19 1.098906 3.0009 0.00002000 2025-01-29 05:33:47.381810 18
20 1.088252 2.9691 0.00002000 2025-01-29 06:22:06.299740 19
```
## V1
```bash
0 1.328085 3.7738 0.00002000 2025-02-24 22:42:19.385036 0
1 1.250953 3.4937 0.00002000 2025-02-24 23:31:09.470548 1
2 1.220156 3.3877 0.00002000 2025-02-25 00:20:28.265176 2
3 1.189695 3.2861 0.00002000 2025-02-25 01:09:00.092662 3
4 1.173590 3.2336 0.00002000 2025-02-25 01:57:14.064583 4
5 1.155840 3.1767 0.00002000 2025-02-25 02:45:25.223964 5
6 1.146094 3.1459 0.00002000 2025-02-25 03:33:37.876671 6
7 1.128762 3.0918 0.00002000 2025-02-25 04:21:51.325201 7
8 1.119391 3.0630 0.00002000 2025-02-25 05:10:04.734031 8
9 1.112895 3.0432 0.00002000 2025-02-25 05:58:17.806360 9
10 1.109004 3.0313 0.00002000 2025-02-25 06:46:33.949976 10
11 1.101230 3.0079 0.00002000 2025-02-25 07:34:52.654477 11
12 1.092156 2.9807 0.00002000 2025-02-25 08:23:09.908699 12
13 1.087285 2.9662 0.00002000 2025-02-25 09:11:25.518814 13
14 1.081148 2.9481 0.00002000 2025-02-25 09:59:37.800531 14
15 1.074223 2.9277 0.00002000 2025-02-25 10:47:51.388452 15
16 1.074324 2.9280 0.00002000 2025-02-25 11:36:07.807562 16
17 1.068234 2.9102 0.00002000 2025-02-25 12:24:22.401156 17
18 1.059605 2.8852 0.00002000 2025-02-25 13:12:37.599124 18
19 1.055234 2.8726 0.00002000 2025-02-25 14:00:52.741654 19
```
## V2
```bash
1 1.432728 4.1901 0.00002000 2025-03-01 18:28:38.262673 0
2 1.295736 3.6537 0.00002000 2025-03-01 19:17:53.931170 1
3 1.277158 3.5864 0.00002000 2025-03-01 20:07:08.693268 2
4 1.266635 3.5489 0.00002000 2025-03-01 20:56:43.106751 3
5 1.232101 3.4284 0.00002000 2025-03-01 21:46:04.793249 4
6 1.197623 3.3122 0.00002000 2025-03-01 22:36:19.047485 5
7 1.198684 3.3157 0.00002000 2025-03-01 23:26:23.852325 6
8 1.153539 3.1694 0.00002000 2025-03-02 00:15:25.319388 7
9 1.190393 3.2884 0.00002000 2025-03-02 01:04:04.914936 8
10 1.151376 3.1625 0.00002000 2025-03-02 01:52:36.610501 9
11 1.142265 3.1339 0.00002000 2025-03-02 02:41:09.421198 10
12 1.125220 3.0809 0.00002000 2025-03-02 03:29:40.905325 11
13 1.108536 3.0299 0.00002000 2025-03-02 04:18:13.008727 12
14 1.152535 3.1662 0.00002000 2025-03-02 05:06:43.459698 13
15 1.137459 3.1188 0.00002000 2025-03-02 05:55:18.549682 14
16 1.107539 3.0269 0.00002000 2025-03-02 06:43:50.482294 15
17 1.137236 3.1181 0.00002000 2025-03-02 07:32:25.326706 16
18 1.119796 3.0642 0.00002000 2025-03-02 08:20:57.625010 17
19 1.109311 3.0323 0.00002000 2025-03-02 09:09:29.684893 18
20 1.098197 2.9988 0.00002000 2025-03-02 09:58:02.055555 19
```
|