Crack Your Next ML Interview

Bias-variance tradeoff describes how model complexity affects generalization:

• High bias -> underfitting (model too simple, misses patterns).
• High variance -> overfitting (model too complex, memorizes noise).
• Goal is to find a balance that minimizes total error on unseen data.

Regularization, model choice, and data size all influence this tradeoff.

• How does regularization affect bias and variance?
• Give an example of a high‑bias model and a high‑variance model.
• How would you detect overfitting in practice?

Regularization adds a penalty term to the loss function to discourage overly complex models:

• L2 (Ridge): penalizes squared weights, encourages small but non‑zero weights.
• L1 (Lasso): penalizes absolute weights, encourages sparsity and feature selection.
• Reduces overfitting by controlling model capacity.

• When would you prefer L1 over L2?
• How does regularization interact with feature scaling?
• What happens if the regularization strength is too high?

Cross‑validation splits data into multiple folds to estimate how a model generalizes:

• Train on k‑1 folds, validate on the remaining fold; repeat for all folds.
• Reduces variance compared to a single train/validation split.
• Stratified CV is important for classification with imbalanced classes.

• When would you avoid k‑fold CV (e.g., time series)?
• How do you adapt CV for time‑dependent data?
• How does CV interact with hyperparameter tuning?

The curse of dimensionality refers to phenomena that arise in high‑dimensional spaces:

• Distances between points become less meaningful.
• Data becomes sparse; models need exponentially more data.
• Many algorithms (KNN, clustering) degrade as dimensions grow.

Dimensionality reduction (PCA, feature selection) helps mitigate this.

• How does PCA help with high‑dimensional data?
• When would you prefer feature selection over PCA?
• How does the curse of dimensionality affect KNN?

In imbalanced datasets, a model can achieve high accuracy by predicting the majority class only:

• Accuracy ignores class distribution and costs of different errors.
• Metrics like precision, recall, F1, and ROC/PR curves are more informative.
• For rare events (fraud, disease), recall and precision are critical.

• When would you optimize for recall over precision?
• How do you choose a decision threshold?
• How do ROC and PR curves differ in interpretation?

Backpropagation computes gradients of the loss with respect to weights using the chain rule:

• Forward pass: compute outputs and loss.
• Backward pass: propagate gradients layer by layer.
• Optimizer (SGD, Adam) updates weights using these gradients.

It enables efficient training of deep networks by reusing intermediate activations.

• Why do we need non‑linear activation functions?
• What happens if activations saturate?
• How does batch size affect gradient estimates?

In deep networks, repeated multiplication of gradients through layers can:

• Shrink towards zero (vanishing) with certain activations (sigmoid, tanh).
• Grow very large (exploding) with poor initialization or deep stacks.

Mitigations include ReLU‑like activations, careful initialization, residual connections, and normalization.

• How do residual connections help?
• Why are LSTMs more robust than vanilla RNNs?
• What role does gradient clipping play?

• CNNs: local receptive fields, weight sharing; great for images and spatial data.
• RNNs: sequential processing with hidden state; good for sequences but hard to parallelize.
• Transformers: self‑attention, fully parallel, capture long‑range dependencies efficiently.

Transformers have largely replaced RNNs in NLP due to better scaling and performance.

• Why is self‑attention more flexible than fixed convolution kernels?
• When might you still use CNNs today?
• How do Transformers handle very long sequences?

Batch normalization normalizes activations within a mini‑batch:

• Reduces internal covariate shift.
• Stabilizes and speeds up training.
• Allows higher learning rates and can have a regularization effect.

In Transformers, LayerNorm is more common due to different architecture patterns.

• Why is LayerNorm preferred in Transformers?
• What issues arise with very small batch sizes?
• How does batch norm interact with dropout?

MLOps focuses on operationalizing ML models:

• End‑to‑end lifecycle: data, training, deployment, monitoring, retraining.
• Emphasizes reliability, reproducibility, automation, and collaboration.
• Bridges ML with DevOps practices (CI/CD, infra as code, observability).

Traditional ML often stops at model training and offline evaluation.

• What are the main challenges when moving from notebook to production?
• How do you structure teams around MLOps?
• What tools have you used for MLOps?

A feature store is a centralized system for managing ML features:

• Stores feature definitions, values, and metadata.
• Serves features consistently for training and online inference.
• Helps prevent training/serving skew and duplicate feature logic.

• How does a feature store integrate with batch and streaming data?
• What problems arise without a feature store?
• Have you used any feature store tools (Feast, Tecton, etc.)?

• Data drift: input distribution changes over time (e.g., new user behavior).
• Model drift: model performance degrades, even if inputs look similar.
• Detection: monitor feature distributions, PSI, performance metrics, and business KPIs.

Retraining, recalibration, or model replacement may be needed depending on the cause.

• How would you set thresholds for drift alerts?
• What's your strategy for safe retraining?
• How do you handle concept drift in streaming systems?

CI/CD for ML extends software CI/CD with ML‑specific steps:

• CI: unit tests, data validation, training pipeline tests, reproducibility checks.
• CD: automated deployment to staging, canary or shadow deployments, rollback strategies.
• Model registry, versioning, and approval workflows for promotion to production.

• What tests are unique to ML compared to standard software?
• How do you handle model rollbacks?
• How would you integrate data quality checks into the pipeline?

Candidate should outline:

• Sigmoid function for probabilities.
• Binary cross‑entropy loss.
• Gradient descent or mini‑batch gradient descent updates.
• Convergence criteria and evaluation on a validation set.

Exact syntax is less important than a correct mathematical and implementation flow.

• How would you add L2 regularization?
• How do you handle numerical stability in the sigmoid?
• How would you extend this to multi‑class classification?

Candidate should:

• Compute TP, FP, FN from predictions and labels.
• Compute precision = TP / (TP + FP), recall = TP / (TP + FN).
• Compute F1 = 2 * precision * recall / (precision + recall).
• Handle edge cases (division by zero).

• How would you extend this to multi‑class (macro vs micro F1)?
• When is F1 more useful than accuracy?
• How would you choose a threshold to maximize F1?