File size: 6,232 Bytes

715b96a

---
library_name: rkllm
pipeline_tag: text-generation
license: other
license_name: deepsek
license_link: >-
  https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct/blob/main/LICENSE
language:
- en
base_model:
- deepseek-ai/deepseek-coder-1.3b-instruct
tags:
- rkllm
- rk3588
- rockchip
- code
- edge-ai
- llm
---
# deepseek-coder-1.3b-instruct — RKLLM build for RK3588 boards

### Built with DeepSeek (DeepSeek License Agreement)

**Author:** @jamescallander  
**Source model:** [deepseek-ai/deepseek-coder-1.3b-instruct · Hugging Face](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct)

**Target:** Rockchip RK3588 NPU via RKNN-LLM Runtime

> This repository hosts a **conversion** of `deepseek-coder-1.3b-instruct` for use on Rockchip RK3588 single-board computers (Orange Pi 5 plus, Radxa Rock 5b+, Banana Pi M7, etc.). Conversion was performed using the [RKNN-LLM toolkit](https://github.com/airockchip/rknn-llm?utm_source=chatgpt.com)

#### Conversion details

- RKLLM-Toolkit version: v1.2.1
- NPU driver: v0.9.8
- Python: 3.12
- Quantization: `w8a8_g128`
- Output: single-file `.rkllm` artifact
- Tokenizer: not required at runtime (UI handles prompt I/O)

## ⚠️ Code generation disclaimer

🛑 **This model may produce incorrect or insecure code.**

- It is intended for **research, educational, and experimental purposes only**.
- Always **review, test, and validate code outputs** before using them in real projects.
- Do not rely on outputs for production, security-sensitive, or safety-critical systems.
- Use responsibly and in compliance with the source model’s license and restrictions.

## Intended use

- On-device deployment of a **code-specialized LLM** on RK3588 SBCs.
- deepseek-coder-1.3b-instruct is tuned for **programming tasks, code completion, and instruction-following in developer workflows**, optimized for smaller edge hardware.

## Limitations

- Requires 2.5GB free memory
- Quantized build (`w8a8_g128`) may show small quality differences vs. full-precision upstream.
- Tested on Radxa Rock 5B+; other devices may require different drivers/toolkit versions.
- Generated code should always be reviewed before use in production systems.

## Quick start (RK3588)

### 1) Install runtime

The RKNN-LLM toolkit and instructions can be found on the specific development board's manufacturer website or from [airockchip's github page](https://github.com/airockchip).

Download and install the required packages as per the toolkit's instructions.

### 2) Simple Flask server deployment

The simplest way the deploy the `.rkllm` converted model is using an example script provided in the toolkit in this directory: `rknn-llm/examples/rkllm_server_demo`

```bash
python3 <TOOLKIT_PATH>/rknn-llm/examples/rkllm_server_demo/flask_server.py \
  --rkllm_model_path <MODEL_PATH>/deepseek-coder-1.3b-instruct_w8a8_g128_rk3588.rkllm \
  --target_platform rk3588
```

### 3) Sending a request

A basic format for message request is:

```json
{
    "model":"deepseek-coder-1.3b-instruct",
    "messages":[{
        "role":"user",
        "content":"<YOUR_PROMPT_HERE>"}],
    "stream":false
}
```

Example request using `curl`:

```bash
curl -s -X POST <SERVER_IP_ADDRESS>:8080/rkllm_chat \
    -H 'Content-Type: application/json' \
    -d '{"model":"deepseek-coder-1.3b-instruct","messages":[{"role":"user","content":"Create a python function to calculate factorials using recursive method."}],"stream":false}'
```

The response is formated in the following way:

```json
{
    "choices":[{
        "finish_reason":"stop",
        "index":0,
        "logprobs":null,
        "message":{
            "content":"<MODEL_REPLY_HERE">,
            "role":"assistant"}}],
        "created":null,
        "id":"rkllm_chat",
        "object":"rkllm_chat",
        "usage":{
            "completion_tokens":null,
            "prompt_tokens":null,
            "total_tokens":null}
}
```

Example response:

```json
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Sure! Here is the Python code for calculating factorial of an number (n) by implementing it in Recursion Method : ```python def Factorial(num): # Define Function with parameter num if num == 1 or num==0:# Base Case to stop recursive call when we reach one. It's the definition for factorial of any number n, where fact(n) = n * (n-1)! return 1 else: # Recursion Call -> Factorial function is called again with decreasing value until it reaches base case return num*Factorial(num-1) ``` You can call this Function as follows : `print (fact_of_5())`, where 5 will be the number for which you want to find factorial. This function works only with non negative integers and zero! It doesn't work properly if called without arguments or it is given a floating point argument because of integer division in Python2 when using `num*Factorial(num-1)`, this would result into an infinite recursion loop as the base case will never be reached.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}
```

### 4) UI compatibility

This server exposes an **OpenAI-compatible Chat Completions API**.

You can connect it to any OpenAI-compatible client or UI (for example: [Open WebUI](https://github.com/open-webui/open-webui?utm_source=chatgpt.com))
- Configure your client with the API base: `http://<SERVER_IP_ADDRESS>:8080` and use the endpoint: `/rkllm_chat`
- Make sure the `model` field matches the converted model’s name, for example:

```json
{
 "model": "deepseek-coder-1.3b-instruct",
 "messages": [{"role":"user","content":"Hello!"}],
 "stream": false
}
```

# License

This conversion follows the [DeepSeek License Agreement](LICENSE)

- **Attribution:** Built with DeepSeek (© 2023 DeepSeek).
- **Required notice:** see [`NOTICE`](NOTICE)
- **Modifications:** quantization (w8a8_g128), export to `.rkllm` format for RK3588 SBCs.
- **Use Restrictions:** You may not use this model or its derivatives for prohibited purposes listed in Attachment A of the DeepSeek License Agreement (including military use, harming minors, generating PII without authorization, harassment, or discrimination)