Investigating Regularization of Self-Play Language Models
Paper
•
2404.04291
•
Published
•
1
AI Theory
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Abdalgader
authored
2
papers
2 days ago
melaseddik
authored
4
papers
8 days ago
Performance Gaps in Multi-view Clustering under the Nested Matrix-Tensor Model
Paper
•
2402.10677
•
Published
Investigating Regularization of Self-Play Language Models
Paper
•
2404.04291
•
Published
•
1
Falcon2-11B Technical Report
Paper
•
2407.14885
•
Published
Do Vision and Language Encoders Represent the World Similarly?
Paper
•
2401.05224
•
Published
ybelkada
authored
a
paper
8 days ago
slimfrikha
authored
a
paper
8 days ago
melaseddik
authored
a
paper
8 days ago
Iheb-Chaabane
authored
2
papers
9 days ago
ybelkada
authored
a
paper
9 days ago
DhiyaEddine
authored
a
paper
9 months ago
ybelkada
authored
a
paper
10 months ago
Post
4479
Falcon Mamba now available now in llama.cpp !
Check out GGUF files uploaded here: tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a
Check out GGUF files uploaded here: tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a
- 3 replies
Post
4106
FalconMamba 7B - a new model from TII (Technology Innovation Institute) is out !
- Blogpost: https://huggingface.co/blog/falconmamba
- Link to collection: tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a
- Link to playground: tiiuae/falcon-mamba-playground
- Blogpost: https://huggingface.co/blog/falconmamba
- Link to collection: tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a
- Link to playground: tiiuae/falcon-mamba-playground
Post
Check out quantized weights from ISTA-DAS Lab directly in their organisation page:
ISTA-DASLab
! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)
Read more about these techniques below:
AQLM paper: Extreme Compression of Large Language Models via Additive Quantization (2401.06118)
QMoE: QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2310.16795)
Some useful links below:
AQLM repo: https://github.com/Vahe1994/AQLM
How to use AQLM & transformers: https://huggingface.co/docs/transformers/quantization#aqlm
How to use AQLM & PEFT: https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
Great work from @BlackSamorez and team !
ISTA-DASLab
! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)Read more about these techniques below:
AQLM paper: Extreme Compression of Large Language Models via Additive Quantization (2401.06118)
QMoE: QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2310.16795)
Some useful links below:
AQLM repo: https://github.com/Vahe1994/AQLM
How to use AQLM & transformers: https://huggingface.co/docs/transformers/quantization#aqlm
How to use AQLM & PEFT: https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
Great work from @BlackSamorez and team !
Post
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now!
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
AQLM method has been recently introduced on transformers main branch
The 2bit model can be found here: BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
And you can read more about the method here: https://huggingface.co/docs/transformers/main/en/quantization#aqlm
Great work @BlackSamorez and team!
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
AQLM method has been recently introduced on transformers main branch
The 2bit model can be found here: BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
And you can read more about the method here: https://huggingface.co/docs/transformers/main/en/quantization#aqlm
Great work @BlackSamorez and team!
-
- 5 replies
ybelkada
authored
a
paper
over 1 year ago
ybelkada
authored
a
paper
almost 2 years ago