Gradient Descent and Optimization Math Quiz

1. Which of the following best describes the convergence behavior of gradient descent with a well-tuned learning rate?

Monotonic decrease in cost with guaranteed convergence

Exponential decrease followed by oscillation

Generally decreasing cost with possible fluctuations

Alternating increase and decrease in cost

2. True or False: Gradient descent always finds a solution faster on smaller datasets.

True

False

Submit

4. Which of the following is a characteristic of convex optimization problems?

Multiple local minima exist

Any local minimum is also the global minimum

Gradient descent may diverge

The cost function is discontinuous

Submit

6. True or False: The gradient of a function is perpendicular to the level curves of that function.

True

False

7. Which scenario most likely causes gradient descent to get stuck in a saddle point?

Using too small a learning rate

Training on a highly non-convex loss surface with poor initialization

Using batch gradient descent instead of SGD

Having too many training samples

Submit

9. In adaptive learning rate methods, what does the RMSprop algorithm use to scale the learning rate?

Exponential moving average of squared gradients

Sum of all historical gradients

Maximum gradient encountered so far

Variance of the training data

10. True or False: Nesterov momentum always converges faster than standard momentum.

True

False

Submit

12. In gradient descent, the learning rate controls the size of steps taken toward the minimum. What happens if the learning rate is too large?

The algorithm converges faster

The algorithm may diverge or oscillate around the minimum

The cost function becomes smoother

Computational cost decreases significantly

Submit

14. True or False: A local minimum in a non-convex function must also be a global minimum.

True

False

15. The Hessian matrix in optimization contains second-order partial derivatives. What does it help determine?

The curvature of the cost function

The direction of steepest descent

The size of the mini-batch

The number of hidden layers

16. Which optimization algorithm adapts the learning rate for each parameter based on historical gradients?

Adam optimizer

Nesterov momentum

Standard gradient descent

Batch normalization

Submit

18. The term 'momentum' in optimization helps the algorithm by accumulating gradients over time. What is its primary benefit?

It reduces the learning rate automatically

It accelerates convergence and reduces oscillations

It guarantees convergence to a local minimum

It eliminates the need for a learning rate

19. A cost function is convex. True or False: Gradient descent is guaranteed to find the global minimum.

True

False

20. Which of the following is true about stochastic gradient descent (SGD) compared to batch gradient descent?

SGD uses the entire dataset for each update

SGD updates weights using single samples, making it faster but noisier

SGD always converges to the global minimum

SGD requires more memory than batch gradient descent

Submit

DataAI Gradient Descent and Optimization Math Quiz

1. Which of the following best describes the convergence behavior of gradient descent with a well-tuned learning rate?

2.

What first name or nickname would you like us to use?

2. True or False: Gradient descent always finds a solution faster on smaller datasets.

3. The Adam optimizer combines momentum with adaptive learning rates by tracking both the first and second moments of ______.

4. Which of the following is a characteristic of convex optimization problems?

5. In the update rule w := w - α∇L(w), the parameter α represents the ______ rate.

6. True or False: The gradient of a function is perpendicular to the level curves of that function.

7. Which scenario most likely causes gradient descent to get stuck in a saddle point?

8. The condition for convergence in gradient descent typically requires the learning rate to be inversely proportional to the ______ of the cost function.

9. In adaptive learning rate methods, what does the RMSprop algorithm use to scale the learning rate?

10. True or False: Nesterov momentum always converges faster than standard momentum.

11. The partial derivative ∂L/∂w represents the gradient of the loss function with respect to ______.

12. In gradient descent, the learning rate controls the size of steps taken toward the minimum. What happens if the learning rate is too large?

13. In mini-batch gradient descent, the batch size affects both convergence speed and ______ requirements.

14. True or False: A local minimum in a non-convex function must also be a global minimum.

15. The Hessian matrix in optimization contains second-order partial derivatives. What does it help determine?

16. Which optimization algorithm adapts the learning rate for each parameter based on historical gradients?

17. In the context of optimization, a stationary point where the gradient is zero is called a ______ point.

18. The term 'momentum' in optimization helps the algorithm by accumulating gradients over time. What is its primary benefit?

19. A cost function is convex. True or False: Gradient descent is guaranteed to find the global minimum.

20. Which of the following is true about stochastic gradient descent (SGD) compared to batch gradient descent?

21. The gradient of a function at a point indicates the direction of ______ steepest increase.