File size: 2,803 Bytes
8d7ac55 3a14106 8d7ac55 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
---
language:
- en
license: mit
tags:
- gpt-oss
- openai
- mxfp4
- mixture-of-experts
- causal-lm
- text-generation
- cpu-gpu-offload
- colab
datasets:
- openai/gpt-oss-training-data # Placeholder; replace if known
pipeline_tag: text-generation
---
# gpt-oss-20b-offload
This is a CPU+GPU offload‑ready copy of **OpenAI’s GPT‑OSS‑20B** model, an open‑source, Mixture‑of‑Experts large language model released by OpenAI in 2025.
The model here retains OpenAI’s original **MXFP4 quantization** and is configured for **memory‑efficient loading in Colab or similar GPU environments**.
---
## Model Details
### Model Description
- **Developed by:** OpenAI
- **Shared by:** saurabh-srivastava (Hugging Face user)
- **Model type:** Decoder‑only transformer (Mixture‑of‑Experts) for causal language modeling
- **Active experts per token:** 4 / 32 total experts
- **Language(s):** English (with capability for multilingual text generation)
- **License:** MIT (per OpenAI GPT‑OSS release)
- **Finetuned from model:** `openai/gpt-oss-20b` (no additional fine‑tuning performed)
### Model Sources
- **Original model repository:** [https://huggingface.co/openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
- **OpenAI announcement:** [https://openai.com/index/introducing-gpt-oss/](https://openai.com/index/introducing-gpt-oss/)
---
## Uses
### Direct Use
- Text generation, summarization, and question answering.
- Running inference in low‑VRAM environments using CPU+GPU offload.
### Downstream Use
- Fine‑tuning for domain‑specific assistants.
- Integration into chatbots or generative applications.
### Out‑of‑Scope Use
- Generating harmful, biased, or false information.
- Any high‑stakes decision‑making without human oversight.
---
## Bias, Risks, and Limitations
Like all large language models, GPT‑OSS‑20B can:
- Produce factually incorrect or outdated information.
- Reflect biases present in its training data.
- Generate harmful or unsafe content if prompted.
### Recommendations
- Always use with a moderation layer.
- Validate outputs for factual accuracy before use in production.
---
## How to Get Started with the Model
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "your-username/gpt-oss-20b-offload"
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Load with CPU+GPU offload
max_mem = {0: "20GiB", "cpu": "64GiB"}
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto",
max_memory=max_mem
)
inputs = tokenizer("Explain GPT‑OSS‑20B in one paragraph.", return_tensors="pt").to(0)
outputs = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |