Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization (Week 1) Quiz

▸Practical aspects of deep learning :

Improving Deep Neural Networks Week-1 (MCQ)

Recommended Machine Learning Courses:

1. Coursera: Machine Learning  
2. Coursera: Deep Learning Specialization
3. Coursera: Machine Learning with Python
4. Coursera: Advanced Machine Learning Specialization
5. Udemy: Machine Learning
6. LinkedIn: Machine Learning
7. Eduonix: Machine Learning
8. edX: Machine Learning
9. Fast.ai: Introduction to Machine Learning for Coders

1. If you have 10,000,000 examples, how would you split the train/dev/test set?
   
   
   • 98% train . 1% dev . 1% test
   
   
   • 60% train . 20% dev . 20% test
   
   
   • 33% train . 33% dev . 33% test
   
   

---

2. The dev and test set should:
   
   
   • Come from the same distribution
   
   
   • Come from different distributions
   
   
   • Be identical to each other (same (x,y) pairs)
   
   
   • Have the same number of examples
   
   

---

3. If your Neural Network model seems to have high bias, what of the following would be promising things to try? (Check all that apply.)
   
   
   • Increase the number of units in each hidden layer
   
   
   • Add regularization
   
   
   • Get more training data
   
   
   • Make the Neural Network deeper
   
   
   • Get more test data
   
   

---

3. If your Neural Network model seems to have high variance, what of the following would be promising things to try?
   
   
   • Make the Neural Network deeper
   
   
   • Get more training data
   
   
   • Add regularization
   
   
   • Get more test data
   
   
   • Increase the number of units in each hidden layer
   
   

---

4. You are working on an automated check-out kiosk for a supermarket, and are building a classifier for apples, bananas and oranges. Suppose your classifier obtains a training set error of 0.5%, and a dev set error of 7%. Which of the following are promising things to try to improve your classifier? (Check all that apply.)
   
   
   • Increase the regularization parameter lambda
   
   
   • Decrease the regularization parameter lambda
   
   
   • Get more training data
   
   
   • Use a bigger neural network
   
   

---

5. What is weight decay?
   
   
   • A technique to avoid vanishing gradient by imposing a ceiling on the values of the weights.
   
   
   • A regularization technique (such as L2 regularization) that results in gradient descent shrinking the weights on every iteration.
   
   
   • The process of gradually decreasing the learning rate during training.
   
   
   • Gradual corruption of the weights in the neural network if it is trained on noisy data.
   
   

---

6. What happens when you increase the regularization hyperparameter lambda?
   
   
   • Weights are pushed toward becoming smaller (closer to 0)
   
   
   • Weights are pushed toward becoming bigger (further from 0)
   
   
   • Doubling lambda should roughly result in doubling the weights
   
   
   • Gradient descent taking bigger steps with each iteration (proportional to lambda)
   
   

---

7. With the inverted dropout technique, at test time:
   
   
   • You apply dropout (randomly eliminating units) and do not keep the 1/keep_prob factor in the calculations used in training
   
   
   • You do not apply dropout (do not randomly eliminate units) and do not keep the 1/keep_prob factor in the calculations used in training
   
   
   • You do not apply dropout (do not randomly eliminate units), but keep the 1/keep_prob factor in the calculations used in training.
   
   
   • You apply dropout (randomly eliminating units) but keep the 1/keep_prob factor in the calculations used in training.
   
   

---

8. Increasing the parameter keep_prob from (say) 0.5 to 0.6 will likely cause the following: (Check the two that apply)
   
   
   • Increasing the regularization effect
   
   
   • Reducing the regularization effect
   
   
   • Causing the neural network to end up with a higher training set error
   
   
   • Causing the neural network to end up with a lower training set error
   
   

---

Check-out our free tutorials on IOT (Internet of Things):

---

9. Which of these techniques are useful for reducing variance (reducing overfitting)? (Check all that apply.)
   
   
   • Gradient Checking
   
   
   • L2 regularization
   
   
   • Xavier initialization
   
   
   • Exploding gradient
   
   
   • Dropout
   
   
   • Vanishing gradient
   
   
   • Data augmentation
   
   

---

10. Why do we normalize the inputs x?
    
    
    • Normalization is another word for regularization–It helps to reduce variance
    
    
    • It makes it easier to visualize the data
    
    
    • It makes the cost function faster to optimize
    
    
    • It makes the parameter initialization faster
    
    

---

Click here to see solutions for all Machine Learning Coursera Assignments.
&
Click here to see more codes for Raspberry Pi 3 and similar Family.
&
Click here to see more codes for NodeMCU ESP8266 and similar Family.
&
Click here to see more codes for Arduino Mega (ATMega 2560) and similar Family.

Feel free to ask doubts in the comment section. I will try my best to answer it.
If you find this helpful by any mean like, comment and share the post.
This is the simplest way to encourage me to keep doing such work.

Thanks & Regards,
- APDaga DumpBox