8- TRAINING A NEURAL NETWORK: Implementing backpropagation and gradient descent from scratch




[ad_1]

In this video, I implement backpropagation and gradient descent from scratch using the Python programming language. I also train the network to perform an incredibly hard task: the arithmetic sum 😀

Code:
https://github.com/musikalkemist/DeepLearningForAudioWithPython/tree/master/8-%20Training%20a%20neural%20network:%20Implementing%20back%20propagation%20from%20scratch/code

Melodrive:
https://melodrive.com/

Valerio’s Linkedin:
https://www.linkedin.com/in/valeriovelardo/

Valerio’s Twitter:
https://twitter.com/musikalkemist

Source


[ad_2]

Comment List

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Probably the only backprop tutorial I've been able to follow all the way through, thank you so much 🙂

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    16:19 Sir, why are you using _sigmoid_derivative(activations) instead of _sigmoid_derivative(h_{i+1}); am I missing something?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    at 21:07, instead of doing the reshape, can you not just do [[x] for x in self.activations[i]]?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Hi, great how to include biases in line with weight matrices and ReLU in line with Sigmoid activation?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Excellent course
    13/11/2020

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    I have question in MSE backprop don't we need to do derivative of error before sending to backprop function?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    I wish I could pay you the money worth of the time and money you have saved for me. I never imagined that I would find such an solid and brilliant implementation of nural network on YouTube. I am paying my deep gratitude to you for putting your time and brilliant work in the video. Wishing you the best and please keep teaching us.

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    You didn’t incorporate bias in the network. How would you reflect it since the bias needs updating in back prop?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    In the previous video you had mentioned that we should go down the gradient in order to get to the global minima, in this video we are adding it up to the sum_error instead subtracting it .
    Is that becuase in the previous one we were considering error = a(3) – y and here we are taking it as y – a(3) … ?hence the addition ?
    Also one more Doubt :
    if activations = sigmoid(x) <– this returns a single float value but @ 20:18 we are considering the activations to be a ndarray. Where do we acutally change the structure
    Edit 3: understood that sigmoid(x) takes in a numpy array and computes the activations and returns a numpy array itself !
    Thanks!

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Definitely the best code for entry level MLP and you seem to have the knack of explaining the basics and coding it in the most elegant way. Your approach of Teach By Example (data) worked out very well for me, since I am a Learn By Example person. Thanks for your time putting this series. Definitely learnt few things.

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    You, Sir, are a true Legend, a splendid tutorial boss. Godspeed m/

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    56:21 Epic moment.

    What a great example Valerio, I'm having a really good feeling that I'm getting all the concept, Thanks!

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Please anyone teach me why there are no Bias terms in this Neural network model.

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Thank you for this video

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    how come there is no bias vector in this network?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Thanks Valerio for your step-by-step and comprehensive explanation. I copied your code, got up to the end of training perfectly. However, when I run the code with a new set of inputs (executing the forward_propagate function only with a new dataset), I do not get the result I am expecting which is strange as the weights have been adjusted post training and the expected results should be generated. Any advice?

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Dear, this tutor is very much interesting in training of a neural net. It is highly appreciated how to create or supply array input datasets say using nested for loop method to train and implement forward/backward propagation in ANN. In this case, say 4 raster input datasets (R1, R2, R3 and R4) of which 2 datasets (R1 and R2) having time, longitude and latitude dimensions, and 2 of them (R3 and R4) have no time dimension. The shape of the Raster data is (time =1000, latitude =310, longitude =458). Thank you in advance for your time and knowledge sharing.

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    It works as is, but I can't change the number of input neurons, I tried to go from 2 to 4 in MLP() , I changed the inputs value as well (4 elements in each samples) but got a shape error between "activations" and "w" in np.dot()

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Woahh, I got it running!!! Thank you so much , Valerio!

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    Finally, a video that talks to engineers. Well done.

  • Valerio Velardo - The Sound of AI
    December 19, 2020

    This is one of the best NN series I've even watched! Very well explained! Thanks!

Write a comment