Project · 2026

titan-engine

GPT, built from scratch in Python.

pausedPython · PyTorch

Overview

titan-engine holds my notebooks following Andrej Karpathy's Zero to Hero: autograd from scratch, character-level models and a small GPT trained on Don Quixote.

The name comes from the original plan, a GPT-2 inference engine in C++ and CUDA. That part is not built: the repo is Python only.

Features

  • →Autograd from scratch
  • →Bigram, MLP and WaveNet
  • →GPT trained on Don Quixote
  • →GPT-2 training script

How it works

01micrograd: backpropagation on a tiny computation graph.
02makemore: from counting letter pairs to an MLP and WaveNet.
03GPT: self-attention, trained on the full text of Don Quixote.

Example

output.txt
cizando de ha mobre españor
 acha lubino
a, bino pla.

Esos pofecigo y, puerese si gunóle; que sarríi, que en corontene