File size: 1,297 Bytes
a7721af |
1 2 3 4 5 6 7 8 9 10 11 |
ret =llm16.export_rkllm(f"orpheus-3b-0.1-ft_{quantized_dtype}_{target_platform}_16bit.rkllm")
INFO: Setting chat_template to "<|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 04 May 2025\n\n<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n[content]<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
INFO: Setting token_id of bos to 128000
INFO: Setting token_id of eos to 128009
INFO: Setting token_id of pad to 128004
INFO: Setting add_bos_token to True
Converting model: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 255/255 [00:00<00:00, 1331939.63it/s]
INFO: Setting max_context_limit to 4096
INFO: Exporting the model, please wait ....
[=================================================>] 597/597 (100%)
INFO: Model has been saved to orpheus_3b_0.1_ft_w8a8_RK3588_16bit.rkllm! |