I Created a Neural Network and Tried Teaching it to Recognize Doodles

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images. Source code: https://github.com/SebLague/Neural-Network-Experiments Demo: https://sebastian.itch.io/neural-network-experiment If you'd like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: https://www.patreon.com/SebastianLague The data is from: Digits http://yann.lecun.com/exdb/mnist/ Fashion https://github.com/zalandoresearch/fashion-mnist Doodles https://github.com/googlecreativelab/quickdraw-dataset Cifar10 https://www.cs.toronto.edu/~kriz/cifar.html Chapters 0:00 Introduction 2:39 The decision boundary 3:49 Weights 5:42 Biases 6:45 Hidden layers 7:45 Programming the network 9:57 Activation functions 12:42 Cost 15:07 Gradient descent example 18:22 The cost landscape 19:55 Programming gradient descent 21:10 It's learning! (slowly) 23:21 Calculus example 27:34 The chain rule 29:50 Some partial derivatives 33:14 Backpropagation 39:25 Digit recognition 43:56 Drawing our own digits 47:37 Fashion 48:25 Doodles 52:00 The final challenge Music: Cosmic Waves - Michael FK Amber - The Stolen Orchestra Beyond the Horizon - Sounds Like Sander Air - Assaf Ayalon Purest Form - Sounds Like Sander Hear Wide Open - Sounds Like Sander Universal Wonder - Moments Roman P - Moments All In Good Time - Shimmer It Will Come Back - The Stolen Orchestra Frontier - Shimmer New Moon - Cloud Wave Sunflower - Cody Martin Inner Peace - Moments Enchanted - Cody Martin Just Around The Corner - Shimmer