| ret =llm16.export_rkllm(f"orpheus-3b-0.1-ft_{quantized_dtype}_{target_platform}_16bit.rkllm") | |
| INFO: Setting chat_template to "<|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 04 May 2025\n\n<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n[content]<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" | |
| INFO: Setting token_id of bos to 128000 | |
| INFO: Setting token_id of eos to 128009 | |
| INFO: Setting token_id of pad to 128004 | |
| INFO: Setting add_bos_token to True | |
| Converting model: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 255/255 [00:00<00:00, 1331939.63it/s] | |
| INFO: Setting max_context_limit to 4096 | |
| INFO: Exporting the model, please wait .... | |
| [=================================================>] 597/597 (100%) | |
| INFO: Model has been saved to orpheus_3b_0.1_ft_w8a8_RK3588_16bit.rkllm! |