GRMR-V3-G4B

GRMR-V3-G4B is a fine-tuned version of unsloth/gemma-3-4b-pt specifically optimized for grammar correction tasks.

IMPORTANT: Please ensure you are using the following sampler settings for optimal results:

temperature = 0.7
frequency_penalty = 0.0
presence_penalty = 0.0
min_p = 0.01
top_p = 0.95
top_k = 40

Model description

GRMR-V3-G4B is a grammar correction model built on Gemma 3 4B (base model). It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.

The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.

Model Details

Developed by: qingy2024

Finetuned from model: unsloth/gemma-3-4b-pt

Model type: Gemma 3

Language(s): English

License: apache-2.0

Training Dataset: qingy2024/grmr-v4-60k

Manual Testing Examples

Here are a few examples of grammar corrections this model can handle:

Original Text	Corrected Text
i dont know weather to bring a umbrella today	I don't know whether to bring an umbrella today.
she go to the store yesterday	She went to the store yesterday.
they is going to be late for the meeting	They are going to be late for the meeting.
the cat laying on the floor all day	The cat is laying on the floor all day.

Training procedure

The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.

Training hyperparameters

Batch size: 8
Gradient accumulation steps: 2
Learning rate: 5e-5
Epochs: 1
Optimizer: AdamW (8-bit)
Weight decay: 0.01
LR scheduler: Cosine
Warmup steps: 180
Max sequence length: 16,384
Training precision: Mixed precision (BF16 where available, FP16 otherwise)

Intended uses & limitations

This model is designed for grammar correction tasks. It can be used to:

Fix grammatical errors in written text
Correct punctuation
Address spelling mistakes
Improve sentence structure and clarity

Limitations

The model may struggle with highly technical or domain-specific content
It may not fully understand context-dependent grammar rules in all cases
Performance may vary for non-standard English or text with multiple errors

How to use

llama.cpp and projects based on it should be able to run this model like any others.

For pure transformers code, you can refer here:

from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model_name = "qingy2024/GRMR-V3-G4B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
text_to_correct = "i am going to the store tommorow and buy some thing for dinner"
messages = [
    {"role": "user", "content": text_to_correct}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=512,
    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
    do_sample=True
)
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected_text)

Using with the Hugging Face pipeline

from transformers import pipeline
pipe = pipeline(
    "text-generation",
    model="qingy2024/GRMR-V3-G4B",
    torch_dtype="auto",
    device_map="auto"
)
messages = [
    {"role": "user", "content": "i dont know weather to bring a umbrella today"}
]
result = pipe(
    messages,
    max_new_tokens=100,
    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
    do_sample=True,
    return_full_text=False
)[0]["generated_text"]
print(result)

Note: The Python examples above use temperature=0.1 for reproducibility in quick tests. For optimal grammar correction quality, please use the recommended sampler settings, especially temperature=0.7.

Custom Chat Template

The model uses a custom chat template with special formatting for grammar correction:

User inputs are formatted with <|text_start|> and <|text_end|> tags
Model outputs are formatted with <|corrected_start|> and <|corrected_end|> tags

The complete chat template is:

{{- bos_token }}
{#- Process messages with role mapping #}
{%- for message in messages %}
{%- if message['role'] == 'user' %}
    {{- 'text
'+ message['content'] | trim + '
' }}
{%- elif message['role'] == 'assistant' %}
    {{- 'corrected
'+ message['content'] | trim + '
' }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- 'corrected
' }}
{%- endif %}

Training Dataset

The model was fine-tuned on the qingy2024/grmr-v4-60k dataset, which contains 60,000 examples of original text and their grammatically corrected versions.

Bias, Risks, and Limitations

The model may reflect biases present in the training data
It may not perform equally well across different writing styles or domains
The model might occasionally introduce errors or change the meaning of text
It focuses on grammatical correctness rather than stylistic improvements

Citations

@article{gemma_2025,
    title={Gemma 3},
    url={https://goo.gle/Gemma3Report},
    publisher={Kaggle},
    author={Gemma Team},
    year={2025}
}

Contact

For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.

qingy2024
/

GRMR-V3-G4B