Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization (Week 2) Quiz

byAkshay Daga (APDaga) -
0

▸Optimization algorithms :

Improving Deep Neural Networks Week-2 (MCQ)
Improving Deep Neural Networks Week-2 (MCQ)
Recommended Machine Learning Courses:
  1. Coursera: Machine Learning  
  2. Coursera: Deep Learning Specialization
  3. Coursera: Machine Learning with Python
  4. Coursera: Advanced Machine Learning Specialization
  5. Udemy: Machine Learning
  6. LinkedIn: Machine Learning
  7. Eduonix: Machine Learning
  8. edX: Machine Learning
  9. Fast.ai: Introduction to Machine Learning for Coders

  1. Which notation would you use to denote the 3rd layer’s activations when the input is the 7th example from the 8th minibatch?











  1. Which of these statements about mini-batch gradient descent do you agree with?

    • Training one epoch (one pass through the training set) using mini-batch gradient descent is faster than training one epoch using batch gradient descent.

    • You should implement mini-batch gradient descent without an explicit for-loop over different mini-batches, so that the algorithm processes all mini-batches at the same time (vectorization).

    • One iteration of mini-batch gradient descent (computing on a single mini-batch) is faster than one iteration of batch gradient descent.


  1. Why is the best mini-batch size usually not 1 and not m, but instead something in between?

    • If the mini-batch size is 1, you end up having to process the entire training set before making any progress.

    • If the mini-batch size is m, you end up with stochastic gradient descent, which is usually slower than mini-batch gradient descent.

    • If the mini-batch size is m, you end up with batch gradient descent, which has to process the whole training set before making progress.

    • If the mini-batch size is 1, you lose the benefits of vectorization across examples in the mini-batch.







    Suppose your learning algorithm’s cost J, plotted as a function of the number of iterations, looks like this:
    enter image description here
    Which of the following do you agree with?

    • If you’re using mini-batch gradient descent, this looks acceptable. But if you’re using batch gradient descent, something is wrong.

    • Whether you’re using batch gradient descent or mini-batch gradient descent, something is wrong.

    • Whether you’re using batch gradient descent or mini-batch gradient descent, this looks acceptable.

    • If you’re using mini-batch gradient descent, something is wrong. But if you’re using batch gradient descent, this looks acceptable.


  1. Suppose the temperature in Casablanca over the first three days of January are the same:

    Jan 1st:

    Jan 2nd:

    (We used Fahrenheit in lecture, so will use Celsius here in honor of the metric world.)
    Say you use an exponentially weighted average with to track the temperature: . If is the value computed after day 2 without bias correction, and is the value you compute with bias correction. What are these values?

    (You might be able to do this without a calculator, but you don’t actually need one. Remember what is bias correction doing.)

    • 7.5, 7.5

    • 10, 10

    • 7.5, 10

    • 10, 7.5







  1. Which of these is NOT a good learning rate decay scheme? Here, t is the epoch number.






  1. You use an exponentially weighted average on the London temperature dataset. You use the following to track the temperature: . The red line below was computed using . What would happen to your red curve as you vary β? (Check the two that apply)
    enter image description here


    • Decreasing β will shift the red line slightly to the right.

    • Increasing β will shift the red line slightly to the right.
      True, remember that the red line corresponds to β=0.9. In lecture we had a green line $$\beta = 0.98) that is slightly shifted to the right.

    • Decreasing β will create more oscillation within the red line.
      True, remember that the red line corresponds to β=0.9. In lecture we had a yellow line $$\beta = 0.98 that had a lot of oscillations.

    • Increasing β will create more oscillations within the red line.


  1. Consider this figure:
    enter image description here

    These plots were generated with gradient descent; with gradient descent with momentum (β = 0.5) and gradient descent with momentum (β = 0.9). Which curve corresponds to which algorithm?

    • (1) is gradient descent with momentum (small β), (2) is gradient descent with momentum (small β), (3) is gradient descent

    • (1) is gradient descent. (2) is gradient descent with momentum (large β) . (3) is gradient descent with momentum (small β)

    • (1) is gradient descent with momentum (small β). (2) is gradient descent. (3) is gradient descent with momentum (large β)

    • (1) is gradient descent. (2) is gradient descent with momentum (small β). (3) is gradient descent with momentum (large β)


Check-out our free tutorials on IOT (Internet of Things):


  1. Suppose batch gradient descent in a deep network is taking excessively long to find a value of the parameters that achieves a small value for the cost function . Which of the following techniques could help find parameter values that attain a small value for J? (Check all that apply)

    • Try mini-batch gradient descent

    • Try initializing all the weights to zero

    • Try better random initialization for the weights

    • Try tuning the learning rate α

    • Try using Adam


  1. Which of the following statements about Adam is False?

    • Adam combines the advantages of RMSProp and momentum

    • We usually use “default” values for the hyperparameters β1, β2 and ε in Adam (β1 = 0.9, β2 = 0.999, )

    • The learning rate hyperparameter α in Adam usually needs to be tuned.

    • Adam should be used with batch gradient computations, not with mini-batches.


Click here to see solutions for all Machine Learning Coursera Assignments.
&
Click here to see more codes for Raspberry Pi 3 and similar Family.
&
Click here to see more codes for NodeMCU ESP8266 and similar Family.
&
Click here to see more codes for Arduino Mega (ATMega 2560) and similar Family.

Feel free to ask doubts in the comment section. I will try my best to answer it.
If you find this helpful by any mean like, comment and share the post.
This is the simplest way to encourage me to keep doing such work.


Thanks & Regards,
- APDaga DumpBox
Tags:
إرسال تعليق (0)
أحدث أقدم