TheClusterDev
/

Q3OI-FP8

compressed-tensors

Model card Files Files and versions

Q3OI-FP8 / recipe.yaml

TheClusterDev's picture

Compressed LLM

7a8c2bc verified 20 days ago

198 Bytes

	default_stage:
	default_modifiers:
	QuantizationModifier:
	targets: [Linear]
	ignore: [lm_head, 're:.mlp.gate$', 're:.shared_expert_gate$', 're:.*router$']
	scheme: FP8_DYNAMIC