Build Your GPT

Build Your GPT Project Tracker

Complete all ML problems and your submissions assemble into a working GPT that generates text. Your code. Your project. Your GitHub.

Status	Problem	Difficulty
	Gradient Descent	Easy
	Sigmoid And Relu	Easy
	Softmax	Easy
	Cross Entropy Loss	Medium
	Linear Regression Forward	Medium
	Linear Regression Training	Medium

Your Project

Each file maps to a problem you solved. Green means your code is ready.

README.mdauto

requirements.txtauto

gradient_descent.pydone

sigmoid_relu.pydone

softmax.pydone

cross_entropy.pyCross entropy loss

linear_regression.pyLinear regression forward

linear_regression_train.pyLinear regression training

single_neuron.pydone

backprop.pydone

multi_layer_backprop.pyMulti layer backpropagation

mlp.pyMLP from scratch

weight_init.pyWeight initialization

pytorch_basics.pydone

training_loop.pydone

diagnostics.pyTraining diagnostics

dead_relu.pyDead ReLU detector

mnist.pyHandwritten digit classifier

nlp_intro.pyNLP intro

sentiment_analysis.pySentiment analysis

train.pyGPT training loop

generate.pyText generation

normalization.pyLayer normalization

batch_normalization.pyBatch normalization

rms_normalization.pyRMS normalization

embeddings.pydone

positional_encoding.pyPositional encoding

attention.pySelf-attention head

multi_head_attention.pyMulti-headed self-attention

transformer.pyTransformer block

gpt.pyGPT model

kv_cache.pyKV-Cache for fast inference

grouped_query_attention.pyGrouped query attention

tokenizer.pydone

vocabulary.pydone

edge_cases.pyTokenization edge cases

dataloader.pyGPT data loader

dataset.pyGPT dataset

Download project

Only includes files you've solved

Continue Your Journey

After building your GPT, go deeper with advanced tracks

Training at Scale

Coming Soon

Scale your GPT to multiple GPUs with DDP, ZeRO, mixed precision, RLHF, and more

12 problems

GPU Programming

Coming Soon

Write CUDA kernels from scratch — vector ops, shared memory, Flash Attention

15 problems

Math Foundations3/6

Build a Neural Net2/5

PyTorch1/4

Training1/4

NLP1/4

Attention & Transformers0/3

Build GPT2/10

10/36Solved

January 2026

Day 1--:--:-- left

SMTWTFS

Current Streak

0 days

Best Streak

0 days

Solve one problem a day to keep your streak