Overview
titan-engine holds my notebooks following Andrej Karpathy's Zero to Hero: autograd from scratch, character-level models and a small GPT trained on Don Quixote.
The name comes from the original plan, a GPT-2 inference engine in C++ and CUDA. That part is not built: the repo is Python only.
Features
- →Autograd from scratch
- →Bigram, MLP and WaveNet
- →GPT trained on Don Quixote
- →GPT-2 training script
How it works
01micrograd: backpropagation on a tiny computation graph.
02makemore: from counting letter pairs to an MLP and WaveNet.
03GPT: self-attention, trained on the full text of Don Quixote.
Example
output.txt
cizando de ha mobre españor acha lubino a, bino pla. Esos pofecigo y, puerese si gunóle; que sarríi, que en corontene