Spaces:
Runtime error
Runtime error
File size: 1,827 Bytes
21324ec e85e97f 21324ec e85e97f 21324ec e85e97f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
title: Docker-Providers
emoji: 🐳
colorFrom: blue
colorTo: green
sdk: docker
app_port: 8000
---
# Docker-Providers
This project provides an OpenAI-compatible API interface for running the SmolLM2 model using Docker Model Runner. It allows you to use SmolLM2 as a drop-in replacement for OpenAI's API, making it easy to integrate with existing applications.
## Features
- OpenAI-compatible API endpoints
- Support for both chat completions and text completions
- Docker Model Runner integration with SmolLM2
- Easy deployment with Docker
## Prerequisites
- Docker
- Docker Model Runner
- Python 3.11+
## Getting Started
1. Pull the SmolLM2 model:
```bash
docker model pull ai/smollm2
```
2. Build and run the API:
```bash
# Build the Docker image
docker build -t smollm2-api .
# Run the container
docker run -d -p 8000:8000 --name smollm2-api smollm2-api
```
## API Endpoints
### Chat Completions
```bash
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ai/smollm2",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
]
}'
```
### Text Completions
```bash
curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ai/smollm2",
"prompt": "Once upon a time"
}'
```
### List Models
```bash
curl http://localhost:8000/v1/models
```
## Model Information
SmolLM2-360M is a compact language model with 360 million parameters, designed for:
- Chat assistants
- Text-extraction
- Rewriting and summarization
Available variants:
- ai/smollm2:360M-Q4_K_M (default)
- ai/smollm2:135M-Q4_0
- ai/smollm2:135M-Q4_K_M
- ai/smollm2:135M-F16
- ai/smollm2:135M-Q2_K
- ai/smollm2:360M-Q4_0
- ai/smollm2:360M-F16
## License
This project is licensed under the Apache 2.0 License. |