Coursera: Machine Learning (Week 5) Quiz - Neural Networks: Learning | Andrew NG

▸ Neural Networks: Learning :

Recommended Machine Learning Courses:

1. Coursera: Machine Learning  
2. Coursera: Deep Learning Specialization
3. Coursera: Machine Learning with Python
4. Coursera: Advanced Machine Learning Specialization
5. Udemy: Machine Learning
6. LinkedIn: Machine Learning
7. Eduonix: Machine Learning
8. edX: Machine Learning
9. Fast.ai: Introduction to Machine Learning for Coders

1. You are training a three layer neural network and would like to use backpropagation to compute the gradient of the cost function. In the backpropagation algorithm, one of the steps is to update
   for every i,j. Which of the following is a correct vectorization of this step?
   • 
   
   
   • 
   
   
   • 
   
   
   • 

     This version is correct, as it takes the “outer product” of the two vectors and which is a matrix such that the (i,j)-th entry is as desired.

   
   

---

2. Suppose Theta1 is a 5x3 matrix, and Theta2 is a 4x6 matrix. You set thetaVec = [Theta1(:), Theta2(:)]. Which of the following correctly recovers ?
   • reshape(thetaVec(16 : 39), 4, 6)

     This choice is correct, since Theta1 has 15 elements, so Theta2 begins at
     index 16 and ends at index 16 + 24 - 1 = 39.

   
   
   • reshape(thetaVec(15 : 38), 4, 6)
   
   
   • reshape(thetaVec(16 : 24), 4, 6)
   
   
   • reshape(thetaVec(15 : 39), 4, 6)
   
   
   • reshape(thetaVec(16 : 39), 6, 4)
   
   

---

3. Let . Let , and . Use the formula to numerically compute an approximation to the derivative at . What value do you get? (When , the true/exact derivati ve is .)
   • 8
   
   
   • 6.0002

     We compute .

   
   
   • 6
   
   
   • 5.9998
   
   

---

4. Which of the following statements are true? Check all that apply.
   • For computational efficiency, after we have performed gradient checking to verify that our backpropagation code is correct, we usually disable gradient checking before using backpropagation to train the network.

     Checking the gradient numerically is a debugging tool: it helps ensure a correct implementation, but it is too slow to use as a method for actually computing gradients.

   
   
   • Computing the gradient of the cost function in a neural network has the same efficiency when we use backpropagation or when we numerically compute it using the method of gradient checking.
   
   
   • Using gradient checking can help verify if one’s implementation of backpropagation is bug-free.

     If the gradient computed by backpropagation is the same as one computed numerically with gradient checking, this is very strong evidence that you have a correct implementation of backpropagation.

   
   
   • Gradient checking is useful if we are using one of the advanced optimization methods (such as in fminunc) as our optimization algorithm. However, it serves little purpose if we are using gradient descent.
   
   

---

Check-out our free tutorials on IOT (Internet of Things):

---

5. Which of the following statements are true? Check all that apply.
   • If we are training a neural network using gradient descent, one reasonable “debugging” step to make sure it is working is to plot J(Θ) as a function of the number of iterations, and make sure it is decreasing (or at least non-increasing) after each iteration.

     Since gradient descent uses the gradient to take a step toward parameters
     with lower cost (ie, lower J(Θ)), the value of J(Θ) should be equal or less at each iteration if the gradient computation is correct and the learning rate is set properly.

   
   
   • Suppose you have a three layer network with parameters (controlling the function mapping from the inputs to the hidden units) and (controlling the mapping from the hidden units to the outputs). If we set all the elements of to be 0, and all the elements of to be 1, then this suffices for symmetry breaking, since the neurons are no longer all computing the same function of the input.
   
   
   • Suppose you are training a neural network using gradient descent. Depending on your random initialization, your algorithm may converge to different local optima (i.e., if you run the algorithm twice with different random initializations, gradient descent may converge to two different solutions).

     The cost function for a neural network is non-convex, so it may have multiple minima. Which minimum you find with gradient descent depends on the initialization.

   
   
   • If we initialize all the parameters of a neural network to ones instead of zeros, this will suffice for the purpose of “symmetry breaking” because the parameters are no longer symmetrically equal to zero.
   
   

---

Click here to see solutions for all Machine Learning Coursera Assignments.
&
Click here to see more codes for Raspberry Pi 3 and similar Family.
&
Click here to see more codes for NodeMCU ESP8266 and similar Family.
&
Click here to see more codes for Arduino Mega (ATMega 2560) and similar Family.

Feel free to ask doubts in the comment section. I will try my best to answer it.
If you find this helpful by any mean like, comment and share the post.
This is the simplest way to encourage me to keep doing such work.

Thanks & Regards,
- APDaga DumpBox