Neural Network Architecture Basics Quiz

1. In a transformer architecture, what mechanism allows the model to focus on different parts of the input?

Pooling

Self-attention

Convolution

Dropout

2. What is early stopping in the context of neural network training?

Halting training when validation performance stops improving

Reducing the learning rate gradually

Limiting the number of hidden layers

Removing poorly performing neurons

3. In a neural network, what does the bias term allow the model to do?

Shift decision boundaries independently of input

Reduce the number of parameters

Increase computational efficiency

Prevent overfitting

4. What is the key difference between max pooling and average pooling?

Max pooling takes the maximum value; average pooling computes the mean

Max pooling is slower than average pooling

Average pooling preserves more information

They produce identical results

5. How does weight initialization affect neural network training?

It has no significant impact

Poor initialization can slow convergence or cause training to fail

It only matters for shallow networks

It determines the final accuracy

6. What does the softmax function do in neural networks?

Converts logits to probabilities summing to 1

Applies element-wise non-linearity

Normalizes feature values

Computes prediction error

7. In LSTM networks, what is the purpose of the forget gate?

To reset all memory to zero

To control how much of the previous cell state to keep

To add new information to the cell state

To output the final prediction

8. What is the role of pooling layers in convolutional networks?

To reduce spatial dimensions and computational cost

To increase the number of feature maps

To apply non-linear activation

To normalize the input

9. Which loss function is typically used for binary classification?

Mean Squared Error

Cross-Entropy Loss

L1 Loss

Hinge Loss

10. What does a residual connection (skip connection) do in deep networks?

Adds the input of a layer directly to its output

Reduces the number of parameters

Increases network depth linearly

Prevents data leakage

11. What is the primary role of an activation function in a neural network?

To normalize input data

To introduce non-linearity and enable learning of complex patterns

To reduce computational cost

To prevent overfitting

12. What is batch normalization?

Normalizing inputs to have mean 0 and variance 1 within each batch

Dividing the dataset into training and testing sets

Scaling weights to prevent saturation

Averaging gradients across multiple batches

13. Which optimizer adapts the learning rate for each parameter based on gradient history?

Stochastic Gradient Descent

Momentum

Adam

Batch Gradient Descent

14. What is the purpose of dropout in neural networks?

To speed up training

To randomly deactivate neurons and reduce overfitting

To initialize weights properly

To normalize layer inputs

15. In recurrent neural networks, what allows information to persist across time steps?

Pooling layers

Hidden state or memory cell

Batch normalization

Skip connections

16. What does a convolutional layer do in a CNN?

Applies learned filters to detect local features in input data

Reduces dimensionality through averaging

Computes gradients for backpropagation

Normalizes the input distribution

17. Which activation function is most commonly used in hidden layers of modern networks?

Sigmoid

Tanh

ReLU

Linear

18. What is the vanishing gradient problem?

Gradients become very small during backprop, slowing learning in early layers

The network converges too quickly

Weights are initialized to zero

The loss function is not differentiable

19. In a feedforward neural network, what connects one layer to the next?

Activation functions

Weights and biases

Loss functions

Dropout layers

20. Which of the following best describes backpropagation?

A method to forward data through the network

An algorithm to compute gradients and update weights by moving backward through layers

A technique to prevent gradient vanishing

A way to initialize network parameters

DataAI Neural Network Architecture Basics Quiz