Hugging Face's logo
Pro Creations's picture
4 5 10

Pro Creations PRO

ProCreations
RustyTake-Off's profile picture esselte974's profile picture Alexandro14's profile picture
·
https://pro-ai.glitch.me

AI & ML interests

AGI and small scale-high quality AI

Recent Activity

replied to Ruurd's post about 22 hours ago
The past year I have been trying to get diffusion models to work for language generation, without having to retrain a LLM from scratch. And recently, we finally succeeded: We introduce "LAD: LoRA-Adapted Denoiser", a method to convert a LLaMA model into a text diffusion model using LoRA finetuning and structured input corruption. 🎯 Try the demo and read the write-up here! https://ruurdkuiper.github.io/tini-lad/ Unlike autoregressive (word-for-word) models like ChatGPT, diffusion models iteratively refine a noised sequence. However, most current diffusion approaches rely on all-parameter retraining and repeatedly remasking tokens, which is costly and slow during both training and inference! 🧠 With LAD: - We can finetune an autoregressive model for diffusive generation in just 10 hours on a single GPU. - Test-time compute is fully adjustable: fewer steps means faster outputs while more steps improve output quality. - Due to our unique noising schedule, remasking is not always needed during inference. All tokens are attended to in each iteration! 🔍 LAD is built using: – A frozen LLaMA-8B backbone – Structured noising: token swaps, duplications, replacements, span shifts – Modified attention masks for bidirectional decoding 💡 We show that even small, fast-trained models can perform diffusive generation — with competitive benchmark performance, perplexity and more flexible test-time behavior than traditional transformers.
liked a dataset 1 day ago
ProCreations/Ultra-FineWeb-EDU
posted an update 1 day ago
NEW DATASETTTTT https://huggingface.co/datasets/ProCreations/Ultra-FineWeb-EDU This dataset is an extremely filtered version of https://huggingface.co/datasets/openbmb/Ultra-FineWeb for educational content only, inspired by https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu Right now dataset size is small (64k examples) and more examples are being processed as you read this!
View all activity

Organizations

Hugging Face Discord Community's profile picture ProCreations Devoloper Team's profile picture

Posts 29

View all Posts

Collections 1