Introducing MiniGPT
A PyTorch implementation of pre-training and fine-tuning scripts to train and fine-tune GPT-2 models
By Michael Hu
2 min read
Transformer based language models like GPT are a hot topic these days, with the success of ChatGPT in particular. We are excited to introduce our most recent project MiniGPT, an open-source implementation of OpenAI's GPT-2 model [1].
The project was implemented in PyTorch and provides comprehensive support for dataset preparation, pre-training, fine-tuning, and distributed training with PyTorch FSDP.