2 11 23

Luc Robert--Villanueva

lucrbrtv

AI & ML interests

None yet

Recent Activity

reacted to sergiopaniego's post with 🔥 about 15 hours ago

nanochat is now in transformers! The LLM by @karpathy is officially in the library, and we wrote a blog covering: how did we port the model, differences from the original, and how to run or train it. go read it 🤓 https://huggingface.co/spaces/nanochat-students/transformers

liked a model 1 day ago

Tongyi-MAI/Z-Image-Turbo

liked a model 3 days ago

deepseek-ai/DeepSeek-Math-V2

View all activity

Organizations

None yet

reacted to sergiopaniego's post with 🔥 about 15 hours ago

Post

1320

nanochat is now in transformers!

The LLM by @karpathy is officially in the library, and we wrote a blog covering: how did we port the model, differences from the original, and how to run or train it.

go read it 🤓

nanochat-students/transformers

liked a model 1 day ago

Tongyi-MAI/Z-Image-Turbo

Text-to-Image • Updated 1 day ago • 31.5k • • 1.32k

liked a model 3 days ago

deepseek-ai/DeepSeek-Math-V2

Text Generation • 685B • Updated 3 days ago • 2.64k • 461

reacted to samerzaher80's post with 👍 3 days ago

Post

1583

AetherMind_SRL: How I beat 7B models on MMLU with 184M params and a $300 GPU
I’m Sameer, a solo researcher from Iraq working on a single RTX 3050 8GB laptop.Today I’m releasing AetherMind_SRL – a 184M-parameter NLI model that was trained only on tasks (SNLI, MNLI, ANLI, and a small clinical Alzheimer’s dataset).
It was never fine-tuned or even shown a single MMLU question during training.Yet here are the zero-shot MMLU (57 subjects) results:Model
MMLU Zero-Shot
Training Data
AetherMind_SRL (me)
184M
36.05 %
Only NLI (SNLI/MNLI/ANLI + ADNI)
DeBERTa-v3-base
278M
~30.8 %
General pre-training
BERT-large
340M
27–30 %
General pre-training
LLaMA-1 7B
7B
34–35 %
Massive text corpus
LLaMA-2 7B
7B
~45 %
Bigger + better data

Yes – my 184M model beats every classic 300–400M model and the original 7-billion-parameter LLaMA-1, all while running at 300+ samples/sec on a $300 laptop GPU.How did this happen?I built a standardized self-improvement loop called AetherMind Self-Reflective Learning (SRL) v1.0:Train normally on NLI
Let the model predict on hard adversarial data (ANLI)
Log every mistake + low-confidence case
Build a balanced “SMART” buffer (60% errors + 40% correct anchors)
Fine-tune with tiny LR and error-weighted loss
Repeat until stable
That’s it. No external knowledge, no MMLU data, no cluster.
Just pure reasoning transfer from entailment/contradiction patterns → real-world knowledge.Try it yourself python
from transformers import pipeline
import torch

nli_pipeline = pipeline(
"text-classification",
model="samerzaher80/AetherMind_SRL",
device=0 if torch.cuda.is_available() else -1
)

# DEFINE YOUR TEST HERE
premise = "Patient shows progressive memory decline."
hypothesis = "Patient shows progressive memory decline."

input_text = f"{premise} [SEP] {hypothesis}"
result = nli_pipeline(input_text)[0]
print(f"Prediction: {result['label']}")
print(f"Confidence: {result['score']:
Model: samerzaher80/AetherMind_SRL

liked a model 4 days ago

google/embeddinggemma-300m

liked a dataset 5 days ago

MrLight/paper-visa

Viewer • Updated Dec 6, 2024 • 103k • 312 • 1

upvoted an article 6 days ago

Article

The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix

27 days ago

•

reacted to Kseniase's post with ❤️ 9 days ago

Post

5986

12 Types of JEPA

Since Yann LeCun together with Randall Balestriero released a new paper on JEPA (Joint-Embedding Predictive Architecture), laying out its theory and introducing an efficient practical version called LeJEPA, we figured you might need even more JEPA. Here are 7 recent JEPA variants plus 5 iconic ones:

1. LeJEPA → LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics (2511.08544)
Explains a full theory for JEPAs, defining the “ideal” JEPA embedding as an isotropic Gaussian, and proposes the SIGReg objective to push JEPA toward this ideal, resulting in practical LeJEPA

2. JEPA-T → JEPA-T: Joint-Embedding Predictive Architecture with Text Fusion for Image Generation (2510.00974)
A text-to-image model that tokenizes images and captions with a joint predictive Transformer, enhances fusion with cross-attention and text embeddings before training loss, and generates images by iteratively denoising visual tokens conditioned on text

3. Text-JEPA → Speaking in Words, Thinking in Logic: A Dual-Process Framework in QA Systems (2507.20491)
Converts natural language into first-order logic, with a Z3 solver handling reasoning, enabling efficient, explainable QA with far lower compute than large LLMs

4. N-JEPA (Noise-based JEPA) → Improving Joint Embedding Predictive Architecture with Diffusion Noise (2507.15216)
Connects self-supervised learning with diffusion-style noise by using noise-based masking and multi-level schedules, especially improving visual classification

5. SparseJEPA → SparseJEPA: Sparse Representation Learning of Joint Embedding Predictive Architectures (2504.16140)
Adds sparse representation learning to make embeddings more interpretable and efficient. It groups latent variables by shared semantic structure using a sparsity penalty while preserving accuracy

6. TS-JEPA (Time Series JEPA) → Joint Embeddings Go Temporal (2509.25449)
Adapts JEPA to time-series by learning latent self-supervised representations and predicting future latents for robustness to noise and confounders

Read further below ↓
It you like it, also subscribe to the Turing Post: https://www.turingpost.com/subscribe

1 reply

upvoted an article 9 days ago

Article

Text-to-image Architectural Experiments

16 days ago

•

upvoted an article 10 days ago

Article

Why Did MiniMax M2 End Up as a Full Attention Model?

about 1 month ago

•

liked a model 10 days ago

janhq/Jan-v2-VL-high-gguf

Image-Text-to-Text • 8B • Updated 4 days ago • 44.2k • 24

upvoted an article 11 days ago

Article

We’re open-sourcing our text-to-image model and the process behind it

18 days ago

•

replied to codelion's post 19 days ago

Really cool!! Thanks :)

reacted to codelion's post with 👀 19 days ago

Post

3972

Want to experiment with pre-training dataset mixtures but don't want to process terabytes of data? We've got you covered.

We're releasing a collection of several carefully curated 1B token dataset samples specifically designed for rapid prototyping and pretraining experiments: https://huggingface.co/collections/codelion/pre-training-dataset-samples

These samples were created using reservoir sampling - an algorithm that guarantees statistically unbiased random samples from massive source datasets. This means results you get at the 1B token scale are representative of how these datasets behave at 100B+ token scales, letting you iterate quickly without the computational overhead.

The collection includes:
- finePDFs-1B: High-quality textbook-style educational content
- DCLM-baseline-1B: Filtered, diverse web content
- FineWeb-Edu-1B: Curated educational web resources

We used these exact samples to run 50+ systematic experiments on dataset mixing strategies, ultimately discovering that a 50-30-20 mixture of finePDFs + DCLM-baseline + FineWeb-Edu achieves 90%+ of GPT-2's performance with just 1/10th the training data.

Whether you're researching optimal data mixtures, testing curriculum learning strategies, or just want to quickly prototype a pretraining run, these samples give you a solid foundation to start experimenting immediately.

Read the full story of how we used these datasets to find the optimal pretraining recipe: https://huggingface.co/blog/codelion/optimal-dataset-mixing

1 reply

liked a dataset 21 days ago

HuggingFaceFW/finewiki

Viewer • Updated Oct 22 • 61.6M • 21.9k • 263

upvoted an article 21 days ago

Article

Streaming datasets: 100x More Efficient

Oct 27

•

liked a model 23 days ago

moonshotai/Kimi-K2-Thinking

Text Generation • Updated 22 days ago • 304k • • 1.45k

liked 2 Spaces 27 days ago

The Smol Training Playbook

📚

2.47k

The secrets to building world-class LLMs

The Ultra-Scale Playbook

🌌

3.52k

The ultimate guide to training LLM on large GPU Clusters

reacted to branikita's post with 🔥 about 1 month ago

Post

3267

https://robonine.com/testing-of-feetech-sts3215-servomotor-backlash-repeatability-and-torque/

🤖 New article: In-depth testing of the Feetech STS3215 servomotor — the popular choice for open-source robotic projects like SO-ARM 100 and others. We've conducted comprehensive performance analysis covering backlash, repeatability, torque characteristics, and thermal behavior. Discover how this affordable servo performs in real-world robotic applications, including its 12-bit magnetic encoder, metal gearbox design, and overload protection mechanisms.
#Robotics #Servomotor #EngineeringTesting #Automation #FeetechSTS3215 #OpenSourceRobotics #SOArm #RoboticSystems #ControlSystems #MotorTesting #AffordableRobotics #ClosedLoopControl

Luc Robert--Villanueva

AI & ML interests

Recent Activity

Organizations

lucrbrtv's activity

The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix

Text-to-image Architectural Experiments

Why Did MiniMax M2 End Up as a Full Attention Model?

We’re open-sourcing our text-to-image model and the process behind it

Streaming datasets: 100x More Efficient

The Smol Training Playbook

The Ultra-Scale Playbook