| ## project link: | |
| https://github.com/ztxz16/fastllm | |
| # Install | |
| ## install in linux: | |
| ``` sh | |
| # (you should install cuda12 first) | |
| pip install ftllm -U | |
| ``` | |
| ## install in windows | |
| ``` sh | |
| pip install https://www.modelscope.cn/models/huangyuyang/fastllmdepend-windows/resolve/master/ftllmdepend-0.0.0.1-py3-none-win_amd64.whl | |
| pip install ftllm -U | |
| ``` | |
| # Launch | |
| ``` sh | |
| ftllm chat YOUR_MODEL_PATH # chat | |
| ftllm webui YOUR_MODEL_PATH # webui | |
| ftllm server YOUR_MODEL_PATH # server | |
| ``` | |