altkachenko11/my-mini-gpt

MiniGPT: A GPT Model Fine-Tuned on Shakespeare Texts

This is a custom GPT model fine-tuned on a small subset of Shakespeare's works. The model is built using PyTorch and Transformers, and is designed to generate creative text in the style of Shakespeare.

🚀 Overview

This project demonstrates the process of fine-tuning a small version of GPT on a custom dataset and deploying it to Hugging Face for easy use and access. The model has been trained using a tiny Shakespeare dataset (tiny_shakespeare.txt), and is capable of generating creative text based on a given prompt.

🧑‍💻 Setup

To use or replicate this model, follow the steps below to get started.

1. Installation

First, install the required dependencies:

📚 Dataset This model was fine-tuned on a small dataset of Shakespeare's works. You can use your own dataset by following the structure below

Load and tokenize data

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2") tokenizer.pad_token = tokenizer.eos_token # Important for padding

Load your dataset

file_path = "path/to/your_dataset.txt" with open(file_path, "r", encoding="utf-8") as f: raw_text = f.read()

encoded = tokenizer(raw_text, return_tensors="pt", truncation=True, padding=True, max_length=512) input_ids = encoded["input_ids"]

🧑‍💻 Training To fine-tune the model, follow the instructions below: from transformers import TrainingArguments import wandb

wandb.login()

Set up W&B logging configuration

wandb.init(project="mini-gpt", config={ "learning_rate": 0.001, "epochs": 5, "batch_size": 16 })

Training loop

train(model, optimizer, epochs=5, encoded_dataset=encoded_dataset, batch_size=4)

💾 Saving the Model Once the model is trained, you can save it like this: torch.save(model.state_dict(), "my-mini-gpt-model/pytorch_model.bin") tokenizer.save_pretrained("my-mini-gpt-tokenizer")

🚀 Uploading the Model to Hugging Face You can upload the model to Hugging Face Hub using the following commands: from huggingface_hub import login, create_repo, upload_folder

login(token="your_huggingface_token")

repo_id = "your_username/my-mini-gpt" create_repo(repo_id, exist_ok=True)

upload_folder(folder_path="my-mini-gpt-model", repo_id=repo_id) upload_folder(folder_path="my-mini-gpt-tokenizer", repo_id=repo_id) 🎮 Generating Text Once the model is uploaded to Hugging Face, you can easily generate text using the following code: from transformers import AutoModelForCausalLM, AutoTokenizer

Load model and tokenizer from Hugging Face

model = AutoModelForCausalLM.from_pretrained("your_username/my-mini-gpt") tokenizer = AutoTokenizer.from_pretrained("your_username/my-mini-gpt")

Generate text based on a prompt

prompt = "In the future, AI will" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate( **inputs, max_length=50, temperature=1.0, top_k=50, top_p=0.95, do_sample=True )

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True) print(generated_text) 📄 License This model is licensed under the MIT License.

📝 Citation If you use this model or code in your research, please cite it as: Tkachenko, A. (2025). MiniGPT: Fine-tuned GPT model for text generation. Hugging Face. Available at: https://huggingface.co/altkachenko11/my-mini-gpt