Transformer From Scratch
Implemented a decoder-only transformer in PyTorch for text generation and an encoder-decoder transformer for English–German translation, with a full training pipeline: custom attention, tokenization and mixed-precision training. Built end to end — custom SentencePiece tokenizers, memory-mapped datasets, teacher forcing, checkpointing, and training on both local (MPS) and cloud (Colab) hardware.
- PyTorch
- SentencePiece
- Mixed precision
- Google Colab
- Python
2026
Personal project