README.md

SET 1 - MICROGRAD 🔗

 

Overview

This repository contains the implementation of Backpropagation using an AutoGrad Engine, inspired by the Micrograd video by Andrej Karpathy. It explores the foundations of training neural networks and implementing key operations from scratch.

The repository contains:

• Manual Backpropagation: Building intuition and understanding of the gradient calculation process.
• Interactive Site Version: A pilot version of an interactive site that visualizes the functionality, currently under development.

✍🏻 Notes: Follow the notebooks in order for a structured learning path. Each notebook and note corresponds to a particular concept or milestone in the implementation.

 

🗂️Repository Structure

├── .gitignore
├── README.md
├── notes/
│   ├── A-main-video-lecture-notes.md
│   ├── chatgpt-motivation.md
│   ├── crux-node-backpropagation.md
│   ├── expanding-tanh-and-adding-more-operations.md
│   ├── micrograd-functionality.md
│   ├── multi-layer-perceptron.md
│   ├── neurons-explanation.md
│   ├── pytorch-comparision.md
│   └── value-object-creation.md
├── site/
│   ├── interactive_site_pilot_v1.2/
├── 1-derivative-simple-function.ipynb
├── 2-derivative-function-with-multiple-inputs.ipynb
├── 3-value-object.ipynb
├── 3_1-graph-visualisation.ipynb
├── 4_0-manual-backpropagation_simpleExpression.ipynb
├── ... (more implementation notebooks, there are a lot lol)

• Notes Directory: Contains Markdown files with notes and explanations for each topic.
• Interactive Site Directory: Contains files for the pilot version of the interactive visualization tool.
• Implementation Notebooks: Step-by-step code for implementing and understanding backpropagation and related concepts.

 

📄Instructions

1. Start by reading the notes in the notes/ directory for a theoretical understanding.
2. Proceed with the notebooks in the root directory in order to build up the implementation step by step.
3. Explore the site/ directory for the pilot interactive version of the AutoGrad Engine visualization (Idea concept, not yet implemented)

 

⭐Documentation

For a better reading experience and detailed notes, visit my Road to GPT Documentation Site.

💡Pro Tip: This site provides an interactive and visually rich explanation of the notes and code. It is highly recommended you view this project from there.

 

✍🏻Acknowledgments

Notes and implementations inspired by the Micrograd video by Andrej Karpathy.

For more of my projects, visit my Portfolio Site.