How I think about the neural network backpropagation algorithm

My husband Bernie says that every mathematician has a favorite mathematical object, and if that is so, then my favorite mathematical objects are matrices. I try to chunk all my mathematical understandings into matrix expressions, essentially translating everything into my native language. It makes it easier for me to understand things, and to remember what I’ve understood.

In this writeup, I derive the backpropagation algorithm, which is an implementation of the chain rule for certain kinds of composite functions, in terms of matrices and matrix products.

This writeup owes a lot to the chapter on backpropagation in Michael Nielsen’s online book, “Neural Networks and Deep Learning“. It is just a slightly different way to look at things that sticks in my head a little better.

About Carolyn Johnston

I am the principal consultant at Johnston Consulting Services. I help small companies and nonprofits embrace new techniques, and win new business, in fields involving applied mathematics and geospatial technology.
This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to How I think about the neural network backpropagation algorithm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s