Post
103
I got 370 tokens/sec of Qwen3-30B-A3B 2507 on my desktop Z8 GPU workstation. My target is 400 t/s, and the last 10 % always tastes like victory!
Mitko Vasilev
mitkox
AI & ML interests
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
Recent Activity
posted
an
update
6 days ago
I run Qwen3-Coder 480B locally on my Z8, with a 1-million token context window. It’s the equivalent of parallel-parking a Nimitz-class carrier in a kiddie pool. Thanks to whatever dark pact the llama.cpp, CUDA, and kernel folks signed, hybrid inferencing + VRAM↔RAM offload let me stream the model’s synapses across Xeon, RAM, and four lonely A6000s without summoning either the OOM killer or a small house fire.
posted
an
update
12 days ago
I needed a distraction-free AI dev system. Omarchy-AI is an opinionated, minimalist, purpose-built OS layer for AI engineers. Arch Linux, stripped, hardened, and injected with pure, uncut AI developer ergonomics.
TL;DR Omarchy-AI. Vertical AF. One Job, Done Stupidly Well. Only cares about AI engineering.
It’s built on top of Arch Linux & Omarchy and further optimized for:
- Offline, on-the-go AI development. Yes, even on your gaming laptop, or freshly minted DIGITS
- Seamless shift to GPU server backends, because your PC shouldn’t train a 1T Kimi K2 model
- Pre-baked RAG pipelines, agentic workflows, and model fine-tuning
- Actual productivity to spend hours hacking local AI agents, not debugging uv conflicts.
How It Works (The Geeky Bits)
It’s One Curl Command to Rule Them All. Turning a vanilla Arch into a batteries-included local AI dev beast with Hyprland, CUDA, llama.cpp, gcc, and every CLI tool you pretend to know about.
Hyprland: Because Your GPU Deserves Glam Shots. Picked it for the nihilists, tweakers, and keyboard cowboys. It’s an independent Wayland compositor that works great and has zero questions like “how do I get this pretty?”
Make sure you own your AI. AI in the cloud is not aligned with you; it’s aligned with the company that owns it.
Organizations
Posts 14
Post
2032
I run Qwen3-Coder 480B locally on my Z8, with a 1-million token context window. It’s the equivalent of parallel-parking a Nimitz-class carrier in a kiddie pool. Thanks to whatever dark pact the llama.cpp, CUDA, and kernel folks signed, hybrid inferencing + VRAM↔RAM offload let me stream the model’s synapses across Xeon, RAM, and four lonely A6000s without summoning either the OOM killer or a small house fire.
models
19
mitkox/Foundation-Sec-8B-Q8_0-GGUF
Text Generation
•
8B
•
Updated
•
6
•
2
mitkox/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.0-Q4_K_M-GGUF
Text Generation
•
2B
•
Updated
•
2
•
1
mitkox/DeepSeek-R1-Distill-Llama-8B-Q4_K_M-GGUF
8B
•
Updated
•
26
mitkox/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-mlx
Text Generation
•
1B
•
Updated
•
88
mitkox/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-Q4_0-GGUF
Text Generation
•
8B
•
Updated
•
18
•
1
mitkox/OwnYourAI
Updated
•
53
mitkox/wavecoder-ultra-6.7b-Q6_K-GGUF
Text Generation
•
7B
•
Updated
•
7
mitkox/Starling-LM-7B-beta-RLAIF-4bit-MLX
Text Generation
•
1B
•
Updated
•
4
mitkox/Hermes-2-Pro-AlphaMonarch-GGUF
7B
•
Updated
•
6
mitkox/starchat2-15b
16B
•
Updated
•
2