|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- MuzzammilShah/people-names |
|
|
language: |
|
|
- en |
|
|
model_name: Bigram Character-Level Language Model |
|
|
library_name: pytorch |
|
|
tags: |
|
|
- makemore |
|
|
- bigram |
|
|
- language-model |
|
|
- andrej-karpathy |
|
|
--- |
|
|
|
|
|
# Bigram Character-Level Language Model: Makemore (Part 1) |
|
|
|
|
|
Introduced to the concept of a bigram character-level language model, this repository explores its **training**, **sampling**, and **evaluation** processes. The model evaluation was conducted using the **Negative Log Likelihood (NLL)** loss to assess its quality. |
|
|
|
|
|
## Overview |
|
|
The model was trained in two distinct ways, both yielding identical results: |
|
|
1. **Frequency-Based Approach**: Directly counting and normalizing bigram frequencies. |
|
|
2. **Gradient-Based Optimization**: Optimizing the counts matrix using a gradient-based framework guided by minimizing the NLL loss. |
|
|
|
|
|
This demonstrated that **both methods converge to the same result**, showcasing their equivalence in achieving the desired outcome. |
|
|
|
|
|
## Documentation |
|
|
For a better reading experience and detailed notes, visit my **[Road to GPT Documentation Site](https://muzzammilshah.github.io/Road-to-GPT/Makemore-part1/)**. |
|
|
|
|
|
## Acknowledgments |
|
|
Notes and implementations inspired by the **Makemore - Part 1** video by [Andrej Karpathy](https://karpathy.ai/). |
|
|
|
|
|
For more of my projects, visit my [Portfolio Site](https://muhammedshah.com). |
