Instructions to use FunkEngine/FunkEngine-6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FunkEngine/FunkEngine-6B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FunkEngine/FunkEngine-6B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("FunkEngine/FunkEngine-6B") model = AutoModelForCausalLM.from_pretrained("FunkEngine/FunkEngine-6B") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use FunkEngine/FunkEngine-6B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FunkEngine/FunkEngine-6B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FunkEngine/FunkEngine-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/FunkEngine/FunkEngine-6B
- SGLang
How to use FunkEngine/FunkEngine-6B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FunkEngine/FunkEngine-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FunkEngine/FunkEngine-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FunkEngine/FunkEngine-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FunkEngine/FunkEngine-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use FunkEngine/FunkEngine-6B with Docker Model Runner:
docker model run hf.co/FunkEngine/FunkEngine-6B
Model Card for FunkEngine-6B
This is one of those times where it is entirely appropriate to put lipstick on a pig...
Model Details
Model Description
FunkEngine-6B is the pig we all know and love that experiment 2 created in January 2023, but wearing lipstick.
Use of this specific model allows one to bring the pig into places that have "standards" to keep the unfancy pigs out, because clearly this lipstick-adorned pig is upperclass.
- Downloads last month
- 12