How to Use This Question Bank

This is not a checklist assembled from whichever models are currently popular. It is ordered by evidence strength and skill dependency. First learn to write tensors, losses, MLPs, and training loops correctly and independently; then move to minimal attention, VAE, or diffusion components. First learn to define the data, labels, splits, and metrics; then discuss protein generative models.

P0  tensors and shapes
       ├─ losses: MSE, BCE, cross entropy
       ├─ modules: linear, MLP, activation
       ├─ optimization: zero_grad, backward, step
       ├─ loops: train, eval, no_grad, batch aggregation
       ├─ metrics: accuracy, precision/recall, Jaccard
       ├─ algorithms: pairwise distance, KNN, K-means
       └─ reasoning: baseline, split, leakage, imbalance, debugging

      P1  scientific ML
       ├─ variable-length sequences, padding masks, pooling
       ├─ scaled dot-product attention, position
       ├─ protein design: generate, filter, rank, test
       ├─ uncertainty, diversity, active learning
       └─ research deep dive, code ownership, failure analysis

      P2  advanced minimal components
       ├─ VAE reparameterization and KL
       └─ diffusion q_sample, timestep broadcasting, noise loss

LC01: Implement MSE from Scratch

Tags: P0, tensor, loss, 5–10 minutes
Evidence / Derivation: D. Derived jointly from the Tempus “train a model” task, the PathAI custom loss question, the Deep Genomics loss follow-up, and Lila’s stated focus on basic ML. No source claims this is an actual Lila question.

Prompt

Implement mse_loss(pred, target, reduction) without calling torch.nn.functional.mse_loss. Support none, sum, and mean. Both inputs must have the same shape.

Clarify First

• Is broadcasting allowed? Not here, so that shape bugs cannot pass silently.
• How should an empty tensor behave? Here mean follows whatever PyTorch produces; production code may reject that case separately.
• Is the dtype guaranteed to be floating point? Confirm this during the interview.

Approach

Validate the shape and the reduction first, then compute the elementwise squared error. none returns the tensor as is, sum aggregates it, and mean averages over every element. Avoid Python loops and keep the autograd graph intact.

Reference Implementation and Minimal Tests

import torch
      
      def mse_loss(pred: torch.Tensor, target: torch.Tensor, reduction: str = "mean"):
          if pred.shape != target.shape:
              raise ValueError(f"shape mismatch: {pred.shape} vs {target.shape}")
          if reduction not in {"none", "sum", "mean"}:
              raise ValueError(f"unsupported reduction: {reduction}")
          loss = (pred - target).square()
          if reduction == "none":
              return loss
          if reduction == "sum":
              return loss.sum()
          return loss.mean()
      
      pred = torch.tensor([1.0, 3.0], requires_grad=True)
      target = torch.tensor([2.0, 1.0])
      assert torch.allclose(mse_loss(pred, target, "none"), torch.tensor([1.0, 4.0]))
      assert torch.allclose(mse_loss(pred, target), torch.tensor(2.5))
      assert torch.allclose(mse_loss(pred, target, "sum"), torch.tensor(5.0))
      mse_loss(pred, target).backward()
      assert torch.allclose(pred.grad, torch.tensor([-1.0, 2.0]))

Scoring Rubric (10 points): formula 3; reduction 2; shape and argument checks 2; no detach and no loops 1; tests and gradient 2.
Likely Follow-ups: Why is MSE sensitive to outliers? When would you use MAE or Huber? When do a batch mean and a per sample mean differ?
Common Failure Modes: forgetting to square the error; averaging only over the batch dimension; calling .item() and breaking the gradient; letting [B,1] and [B] broadcast silently.

LC02: Implement Huber Loss from Scratch

Tags: P0, loss, robust regression, 10–15 minutes
Evidence / Derivation: D. A robust regression variant of the observed loss choice follow-ups, not a publicly reported question.

Prompt

Implement an elementwise Huber loss: quadratic where the absolute error is at most delta, linear beyond it, then take the mean.

Clarify First: whether the two branches must join continuously; that delta must be positive; whether an elementwise result is expected.
Approach: compute abs_error and switch between the quadratic and the linear regime with torch.where. The two branches agree in both value and first derivative at delta.

Reference Implementation and Minimal Tests

import torch
      
      def huber_loss(pred, target, delta=1.0):
          if pred.shape != target.shape:
              raise ValueError("pred and target must have the same shape")
          if delta <= 0:
              raise ValueError("delta must be positive")
          error = pred - target
          abs_error = error.abs()
          quadratic = 0.5 * error.square()
          linear = delta * (abs_error - 0.5 * delta)
          return torch.where(abs_error <= delta, quadratic, linear).mean()
      
      p = torch.tensor([0.0, 3.0], requires_grad=True)
      t = torch.tensor([0.5, 0.0])
      expected = torch.tensor((0.5 * 0.5**2 + (3.0 - 0.5)) / 2)
      assert torch.allclose(huber_loss(p, t), expected)
      huber_loss(p, t).backward()
      assert p.grad is not None
      assert torch.isfinite(p.grad).all()

Scoring Rubric: piecewise formula 4; continuity 2; vectorization 1; argument checks 1; tests 2.
Likely Follow-ups: How do you choose delta? How does this differ from gradient clipping?
Common Failure Modes: forgetting the delta factor in the linear region; a discontinuous boundary; branching on a tensor with a plain Python if.

LC03: Batched Linear Regression Module

Tags: P0, nn.Module, shape, 10–15 minutes
Evidence / Derivation: D. Lila explicitly names basic ML, and the Merck and Microsoft cases include predictive models, but no source reports this exact prompt.

Prompt

Without using nn.Linear, implement a linear regression module that maps [B,D] to [B,1]. The parameters must be discoverable by an optimizer.

Clarify First: whether a bias is needed; whether the input is always two dimensional; any initialization requirement.
Approach: subclass nn.Module and register the weight and the bias as nn.Parameter, then do the matrix multiply in forward. Keep the output at [B,1] so it cannot broadcast by accident against a [B] target.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      class LinearRegressor(nn.Module):
          def __init__(self, in_features):
              super().__init__()
              self.weight = nn.Parameter(torch.randn(in_features, 1) * 0.01)
              self.bias = nn.Parameter(torch.zeros(1))
      
          def forward(self, x):
              if x.ndim != 2 or x.shape[1] != self.weight.shape[0]:
                  raise ValueError("x must have shape [batch, in_features]")
              return x @ self.weight + self.bias
      
      model = LinearRegressor(3)
      x = torch.randn(5, 3)
      y = model(x)
      assert y.shape == (5, 1)
      assert len(list(model.parameters())) == 2
      y.sum().backward()
      assert model.weight.grad.shape == (3, 1)

Scoring Rubric: parameter registration 3; correct shapes 3; forward 2; tests and gradient 2.
Likely Follow-ups: Why does a plain tensor not show up in parameters()? How would you initialize the weights?
Common Failure Modes: transposing the weight shape; omitting super().__init__(); creating a new Parameter on every forward; squeezing the output unintentionally.

LC04: Binary Logits and Numerically Stable BCE

Tags: P0, classification, numerical stability, 15–20 minutes
Evidence / Derivation: B. The Merck vaccination classification case, the Microsoft classification case, and PathAI ML fundamentals. The specific stable formula is a practice variant.

Prompt

Implement binary cross entropy with logits, without applying a sigmoid first, and return the mean loss. Then write the prediction function.

Clarify First: labels are 0/1 floats; logits and targets share a shape; the threshold is a logit of 0.
Approach: use the stable identity max(x,0) - x*y + log1p(exp(-abs(x))). At prediction time, logit >= 0 is equivalent to a probability of at least 0.5.

Reference Implementation and Minimal Tests

import torch
      
      def stable_bce_with_logits(logits, targets):
          if logits.shape != targets.shape:
              raise ValueError("shape mismatch")
          targets = targets.to(logits.dtype)
          positive_part = logits.clamp_min(0)
          correction = torch.log1p(torch.exp(-logits.abs()))
          losses = positive_part - logits * targets + correction
          return losses.mean()
      
      def binary_predict(logits):
          return (logits >= 0).long()
      
      logits = torch.tensor([-100.0, 0.0, 100.0], requires_grad=True)
      targets = torch.tensor([0.0, 1.0, 1.0])
      loss = stable_bce_with_logits(logits, targets)
      reference = torch.nn.functional.binary_cross_entropy_with_logits(logits, targets)
      assert torch.allclose(loss, reference)
      assert binary_predict(logits).tolist() == [0, 1, 1]
      loss.backward()
      assert torch.isfinite(logits.grad).all()

Scoring Rubric: stable formula 4; no duplicate sigmoid 2; shape and dtype 1; prediction 1; extreme value tests 2.
Likely Follow-ups: How would you add pos_weight? How does probability calibration differ from the classification threshold?
Common Failure Modes: applying a sigmoid before a logits loss; overflowing with a direct log(sigmoid(x)); passing integer targets without converting them.

LC05: Multiclass Cross Entropy

Tags: P0, classification, shape, 15–20 minutes
Evidence / Derivation: D. Drawn from the classification cases and ML fundamentals; not a publicly reported exact prompt.

Prompt

Given logits [B,C] and class indices [B], implement the mean cross entropy without F.cross_entropy.

Clarify First: targets are class indices rather than one-hot vectors; whether class weights are needed; whether an ignore index exists.
Approach: build a stable log-softmax with logsumexp, then gather the log probability of the correct class.

Reference Implementation and Minimal Tests

import torch
      
      def cross_entropy_from_logits(logits, targets):
          if logits.ndim != 2 or targets.shape != (logits.shape[0],):
              raise ValueError("expected logits [B,C] and targets [B]")
          if targets.dtype != torch.long:
              raise TypeError("targets must be torch.long")
          log_probs = logits - torch.logsumexp(logits, dim=1, keepdim=True)
          rows = torch.arange(logits.shape[0], device=logits.device)
          return -log_probs[rows, targets].mean()
      
      logits = torch.tensor([[2.0, 0.0], [0.0, 2.0]], requires_grad=True)
      targets = torch.tensor([0, 1])
      actual = cross_entropy_from_logits(logits, targets)
      expected = torch.nn.functional.cross_entropy(logits, targets)
      assert torch.allclose(actual, expected)
      actual.backward()
      assert logits.grad.shape == logits.shape

Scoring Rubric: stable log-softmax 3; correct gather 3; shape and dtype 2; tests 2.
Likely Follow-ups: How would you write label smoothing? How would you handle one-hot targets?
Common Failure Modes: taking the softmax over the wrong dimension; applying a softmax to probabilities again; a target of shape [B,1]; using a naive log(exp().sum()).

LC06: Two-Layer MLP

Tags: P0, MLP, PyTorch, 10–20 minutes
Evidence / Derivation: A. A Zoom Research Scientist candidate directly reports a PyTorch MLP, and Lila explicitly names basic PyTorch.

Prompt

Implement a two-layer MLP: input [B,D], hidden width H, and C output logits. Explain clearly why the forward pass does not apply a softmax.

Clarify First: the task is classification; the output must be logits; the activation is ReLU; whether dropout is required.
Approach: Linear(D,H) → ReLU → Linear(H,C). The classification loss consumes logits, so the model itself does not apply a softmax.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      class MLP(nn.Module):
          def __init__(self, in_dim, hidden_dim, out_dim):
              super().__init__()
              self.fc1 = nn.Linear(in_dim, hidden_dim)
              self.fc2 = nn.Linear(hidden_dim, out_dim)
      
          def forward(self, x):
              if x.ndim != 2:
                  raise ValueError("x must be [batch, features]")
              hidden = self.fc1(x)
              activated = torch.relu(hidden)
              return self.fc2(activated)
      
      model = MLP(4, 8, 3)
      x = torch.randn(5, 4)
      logits = model(x)
      assert logits.shape == (5, 3)
      loss = torch.nn.functional.cross_entropy(logits, torch.tensor([0, 1, 2, 1, 0]))
      loss.backward()
      for parameter in model.parameters():
          assert parameter.grad is not None

Scoring Rubric: module structure 3; shapes 2; correct logits 2; gradient test 2; clarity of explanation 1.
Likely Follow-ups: How many parameters does it have? What are the strengths and weaknesses of ReLU? When would you add normalization or dropout?
Common Failure Modes: applying a final sigmoid or softmax and then using cross entropy; dropping the activation; confusing the input and output dimensions.

LC07: Residue-Level MLP and Flattening

Tags: P0, shape, protein overlay, 15–20 minutes
Evidence / Derivation: D. A safe transfer of the observed MLP question onto biological sequences; not claimed to be an actual question.

Prompt

Given per sequence residue embeddings [B,L,D], use one shared MLP to produce C class logits at every position, shape [B,L,C].

Clarify First: parameters are shared across positions; padding stays out of the loss for now; no loop over L is needed.
Approach: nn.Linear already acts on the last dimension, so no manual flattening is required. Keep the batch and length dimensions intact.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      class ResidueClassifier(nn.Module):
          def __init__(self, d_model, hidden, classes):
              super().__init__()
              self.fc1 = nn.Linear(d_model, hidden)
              self.fc2 = nn.Linear(hidden, classes)
      
          def forward(self, x):
              if x.ndim != 3:
                  raise ValueError("x must be [B,L,D]")
              hidden = self.fc1(x)
              activated = torch.relu(hidden)
              return self.fc2(activated)
      
      x = torch.randn(2, 7, 5)
      model = ResidueClassifier(5, 9, 4)
      out = model(x)
      assert out.shape == (2, 7, 4)
      out.sum().backward()
      assert model.fc1.weight.grad is not None

Scoring Rubric: understanding that Linear acts on the last dimension 4; output shape 2; no loop over positions 2; tests 2.
Likely Follow-ups: How would you exclude padded positions from the loss? How would you obtain a sequence level prediction?
Common Failure Modes: a reshape that mixes the batch and length dimensions; building a separate layer per position; returning [B,C,L] without saying so.

LC08: One Correct Training Step

Tags: P0, training loop, autograd, 15–20 minutes
Evidence / Derivation: A. The Tempus train a model test, the TORC training loop comprehension question, and Cohere model training.

Prompt

Given model, optimizer, x, y, implement one classification training step that returns the loss as a Python float. No gradient accumulation.

Clarify First: the loss is cross entropy; one call handles exactly one batch; the return value may be detached.
Approach: set train mode, clear the gradients, run the forward pass, compute the loss, call backward, step the optimizer, and only then convert the loss to a Python number.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      def train_step(model, optimizer, x, y):
          model.train()
          optimizer.zero_grad()
          logits = model(x)
          loss = torch.nn.functional.cross_entropy(logits, y)
          loss.backward()
          optimizer.step()
          return loss.detach().item()
      
      torch.manual_seed(0)
      model = nn.Linear(3, 2)
      optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
      x = torch.randn(8, 3)
      y = torch.randint(0, 2, (8,))
      before = model.weight.detach().clone()
      value = train_step(model, optimizer, x, y)
      assert isinstance(value, float)
      assert not torch.equal(before, model.weight.detach())
      for parameter in model.parameters():
          assert parameter.grad is not None

Scoring Rubric: correct ordering 5; mode 1; detached return value 1; parameter update test 2; explanation 1.
Likely Follow-ups: When would you use zero_grad(set_to_none=True)? How would you implement gradient accumulation?
Common Failure Modes: forgetting zero_grad; calling .item() before backward; putting optimizer.step() before backward; rebuilding the optimizer on every step.

LC09: A Complete Train and Eval Epoch

Tags: P0, training loop, evaluation, 20–30 minutes
Evidence / Derivation: A. Supported jointly by the training and debugging questions at TORC, Wayve, NVIDIA, and Tempus.

Prompt

Implement a reusable Meter, then write train_epoch(model, batches, optimizer) and evaluate(model, batches) for multiclass classification. Both epoch functions return the sample weighted average cross entropy and accuracy.

Clarify First: each batch is (x, y) and is already on the correct device; the last batch may be smaller; cross entropy uses its default batch mean; an empty iterator returns (nan, nan).
Approach: keep aggregation in one Meter. Multiply each batch mean by its batch size so every sample has equal weight. Keep training and evaluation explicit: training performs the optimizer step, while evaluation uses model.eval() and torch.inference_mode().

Reference Implementation and Minimal Tests

import torch
      import torch.nn.functional as F
      
      class Meter:
          def __init__(self):
              self.loss_sum = 0.0
              self.correct = 0
              self.n = 0
      
          def update(self, loss, logits, y):
              bs = y.shape[0]
              self.loss_sum += loss.detach().item() * bs
              self.correct += (logits.argmax(dim=1) == y).sum().item()
              self.n += bs
      
          def result(self):
              if self.n == 0:
                  return float("nan"), float("nan")
              return self.loss_sum / self.n, self.correct / self.n
      
      def train_epoch(model, batches, optimizer):
          model.train()
          meter = Meter()
          for x, y in batches:
              optimizer.zero_grad(set_to_none=True)
              logits = model(x)
              loss = F.cross_entropy(logits, y)
              loss.backward()
              optimizer.step()
              meter.update(loss, logits, y)
          return meter.result()
      
      @torch.inference_mode()
      def evaluate(model, batches):
          model.eval()
          meter = Meter()
          for x, y in batches:
              logits = model(x)
              loss = F.cross_entropy(logits, y)
              meter.update(loss, logits, y)
          return meter.result()
      
      model = torch.nn.Linear(2, 2)
      batches = [(torch.randn(3, 2), torch.tensor([0, 1, 0])),
                 (torch.randn(1, 2), torch.tensor([1]))]
      optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
      weight_before_train = model.weight.detach().clone()
      train_loss, train_acc = train_epoch(model, batches, optimizer)
      assert train_loss >= 0
      assert 0 <= train_acc <= 1
      assert not torch.equal(weight_before_train, model.weight)
      
      weight_before_eval = model.weight.detach().clone()
      eval_loss, eval_acc = evaluate(model, batches)
      assert eval_loss >= 0
      assert 0 <= eval_acc <= 1
      assert torch.equal(weight_before_eval, model.weight)
      empty_loss, empty_acc = evaluate(model, [])
      assert empty_loss != empty_loss  # NaN
      assert empty_acc != empty_acc  # NaN

Scoring Rubric: reusable meter and sample weighted aggregation 3; correct training step 2; evaluation mode and inference context 2; accuracy 1; empty input and tests 2.
Likely Follow-ups: What changes if the loss reduction is sum? How would you support mixed precision?
Common Failure Modes: averaging batch means instead of weighting by batch size; forgetting zero_grad; building a graph during evaluation; forgetting model.eval(); dividing by zero on empty input.

LC10: Mini-Batch Iterator without DataLoader

Tags: P0, batching, indexing, 10–15 minutes
Evidence / Derivation: D. A building block of the observed end to end model training tasks.

Prompt

Given x and y with a matching first dimension, write a generator that yields mini-batches. Support shuffling and an incomplete final batch.

Clarify First: whether the last batch should be dropped; how randomness is controlled; whether the data already sits on the target device.
Approach: build an index order, slice it by batch size, and use the same indices for both x and y.

Reference Implementation and Minimal Tests

import torch
      
      def iterate_minibatches(x, y, batch_size, shuffle=True, generator=None):
          if x.shape[0] != y.shape[0]:
              raise ValueError("x and y must have equal first dimension")
          if batch_size <= 0:
              raise ValueError("batch_size must be positive")
          n = x.shape[0]
          if shuffle:
              order = torch.randperm(n, generator=generator, device=x.device)
          else:
              order = torch.arange(n, device=x.device)
          for start in range(0, n, batch_size):
              idx = order[start:start + batch_size]
              yield x[idx], y[idx]
      
      x = torch.arange(15).reshape(5, 3)
      y = torch.arange(5)
      batches = list(iterate_minibatches(x, y, 2, shuffle=False))
      batch_sizes = []
      target_batches = []
      for batch_x, batch_y in batches:
          batch_sizes.append(batch_x.shape[0])
          target_batches.append(batch_y)
      assert batch_sizes == [2, 2, 1]
      assert torch.equal(torch.cat(target_batches), y)

Scoring Rubric: synchronized indexing 3; last batch 2; shuffling 2; argument checks 1; tests 2.
Likely Follow-ups: How do you keep this reproducible across epochs? What happens if the data is on CPU and the indices are on GPU?
Common Failure Modes: shuffling x and y separately; dropping the final batch; a batch size of zero breaking the range.

LC11: Accuracy, Precision, and Recall

Tags: P0, metrics, class imbalance, 15–20 minutes
Evidence / Derivation: A. Tempus asks candidates to design metrics for an ML pipeline, and Flagship discusses skewed data directly.

Prompt

Given binary logits and 0/1 labels, compute accuracy, precision, and recall. Return 0 when a denominator is zero.

Clarify First: the threshold is a probability of 0.5, that is a logit of 0; the positive class is 1; the outputs are Python floats.
Approach: threshold to get the predictions, then count TP, FP, FN, and the total number correct. Handle zero denominators explicitly.

Reference Implementation and Minimal Tests

import torch
      
      def binary_metrics(logits, labels):
          if logits.shape != labels.shape:
              raise ValueError("shape mismatch")
          pred = logits >= 0
          truth = labels.bool()
          tp = (pred & truth).sum().item()
          fp = (pred & ~truth).sum().item()
          fn = (~pred & truth).sum().item()
          correct = (pred == truth).sum().item()
          n = labels.numel()
          if n == 0:
              accuracy = 0.0
          else:
              accuracy = correct / n
          precision_denominator = tp + fp
          if precision_denominator == 0:
              precision = 0.0
          else:
              precision = tp / precision_denominator
          recall_denominator = tp + fn
          if recall_denominator == 0:
              recall = 0.0
          else:
              recall = tp / recall_denominator
          return {"accuracy": accuracy, "precision": precision, "recall": recall}
      
      m = binary_metrics(torch.tensor([2.0, 1.0, -1.0, -2.0]), torch.tensor([1, 0, 1, 0]))
      assert m == {"accuracy": 0.5, "precision": 0.5, "recall": 0.5}
      z = binary_metrics(torch.tensor([-2.0]), torch.tensor([0]))
      assert z["precision"] == 0.0
      assert z["recall"] == 0.0

Scoring Rubric: confusion counts 4; threshold 1; zero denominator 2; shape 1; tests 2.
Likely Follow-ups: Why is accuracy misleading under severe imbalance? How do macro and micro averaging differ?
Common Failure Modes: treating logits as probabilities and thresholding at 0.5; swapping precision and recall; hiding a definitional problem behind an epsilon.

LC12: Jaccard Coefficient

Tags: P0, metric, set operation, 10–15 minutes
Evidence / Derivation: B. A PathAI candidate directly reports being asked to code the Jaccard coefficient.

Prompt

Implement the Jaccard coefficient of two boolean tensors, that is intersection over union. When both are empty sets, return 1 by the convention stated in the question.

Clarify First: the convention for an empty union; whether the metric covers the whole tensor or each sample; whether the inputs can be cast to bool.
Approach: validate the shapes, cast to bool, and count the logical and and the logical or separately.

Reference Implementation and Minimal Tests

import torch
      
      def jaccard(a, b):
          if a.shape != b.shape:
              raise ValueError("shape mismatch")
          a = a.bool()
          b = b.bool()
          intersection = (a & b).sum()
          union = (a | b).sum()
          if union.item() == 0:
              return torch.tensor(1.0, device=a.device)
          return intersection.float() / union.float()
      
      first = torch.tensor([1, 1, 0])
      second = torch.tensor([1, 0, 1])
      expected = torch.tensor(1 / 3)
      assert torch.allclose(jaccard(first, second), expected)
      assert jaccard(torch.zeros(3), torch.zeros(3)).item() == 1.0

Scoring Rubric: formula 4; bool conversion 1; empty case 2; shape 1; tests 2.
Likely Follow-ups: How would you compute this in a batched way? How does it relate to Dice and F1?
Common Failure Modes: dividing the intersection by the length; treating addition as union; ignoring the empty union convention.

LC13: Masked Mean Pooling

Tags: P0, mask, sequence, 15–20 minutes
Evidence / Derivation: B. Derived from the shared needs of protein sequence settings, attention masks, and the Latent positional embedding follow-up; not a publicly reported question.

Prompt

Given embeddings [B,L,D] and a mask [B,L] marking the valid positions, return the masked mean [B,D]. A fully padded sequence returns a zero vector.

Clarify First: True in the mask means valid; how a fully padded row is defined; the dtype and device should follow the input.
Approach: expand the mask to [B,L,1], sum over L after masking, and clamp the denominator to at least 1.

Reference Implementation and Minimal Tests

import torch
      
      def masked_mean(x, mask):
          if x.ndim != 3 or mask.shape != x.shape[:2]:
              raise ValueError("expected x [B,L,D] and mask [B,L]")
          weights = mask.to(x.dtype).unsqueeze(-1)
          total = (x * weights).sum(dim=1)
          count = weights.sum(dim=1).clamp_min(1)
          return total / count
      
      x = torch.tensor([[[1., 2.], [3., 4.], [100., 100.]],
                        [[8., 9.], [7., 6.], [5., 4.]]])
      mask = torch.tensor([[1, 1, 0], [0, 0, 0]], dtype=torch.bool)
      out = masked_mean(x, mask)
      assert torch.allclose(out[0], torch.tensor([2., 3.]))
      assert torch.equal(out[1], torch.zeros(2))

Scoring Rubric: shapes 3; mask broadcasting 3; fully padded case 2; tests 2.
Likely Follow-ups: How would you handle a masked max? What order information does mean pooling discard?
Common Failure Modes: dividing by a fixed L; unsqueezing the wrong dimension; zeroing the padding but still dividing by L.

LC14: Pairwise Squared Distances

Tags: P0, tensor, broadcasting, 15–20 minutes
Evidence / Derivation: A. The PathAI KNN question directly requires nearest neighbors between two 2D arrays, and the Zoom K-means question depends on the same distance matrix.

Prompt

Given a query [M,D] and a reference [N,D], return the squared Euclidean distances [M,N] without an M by N double loop.

Clarify First: whether broadcasting is allowed; whether the data size can hold an [M,N,D] intermediate; this version uses the algebraic expansion to keep intermediate memory low.
Approach: use ||q||² + ||r||² - 2 q rᵀ, then clamp to remove the tiny negative values that floating point error can produce.

Reference Implementation and Minimal Tests

import torch
      
      def pairwise_sq_dist(query, reference):
          if query.ndim != 2 or reference.ndim != 2:
              raise ValueError("expected [M,D] and [N,D]")
          if query.shape[1] != reference.shape[1]:
              raise ValueError("expected [M,D] and [N,D]")
          q2 = query.square().sum(dim=1, keepdim=True)
          r2 = reference.square().sum(dim=1).unsqueeze(0)
          cross_term = 2 * query @ reference.T
          distances = q2 + r2 - cross_term
          return distances.clamp_min(0)
      
      q = torch.tensor([[0., 0.], [1., 1.]])
      r = torch.tensor([[1., 0.], [2., 2.]])
      expected = torch.tensor([[1., 8.], [1., 2.]])
      assert torch.allclose(pairwise_sq_dist(q, r), expected)

Scoring Rubric: algebraic formula 4; shapes 2; numerical clamp 1; argument checks 1; tests 2.
Likely Follow-ups: How does the memory of direct broadcasting compare with the matrix formula? How would you write cosine distance?
Common Failure Modes: dropping the square; getting the matmul dimensions wrong; returning [N,M]; taking the square root of the distance matrix before sorting it.

LC15: K-Nearest Neighbors

Tags: P0, KNN, top-k, 20–25 minutes
Evidence / Derivation: B. A PathAI candidate directly reports a 60 minute KNN assessment.

Prompt

Building on LC14, return the indices and squared distances of the k nearest references for each query.

Clarify First: the valid range of k; whether the query itself must be excluded; ties need not have a stable order.
Approach: compute the [M,N] distances, then apply topk(k, largest=False) along the last dimension.

Reference Implementation and Minimal Tests

import torch
      
      def knn(query, reference, k):
          if not 1 <= k <= reference.shape[0]:
              raise ValueError("k out of range")
          q2 = query.square().sum(dim=1, keepdim=True)
          r2 = reference.square().sum(dim=1).unsqueeze(0)
          cross_term = 2 * query @ reference.T
          distances = (q2 + r2 - cross_term).clamp_min(0)
          values, indices = torch.topk(distances, k, dim=1, largest=False, sorted=True)
          return values, indices
      
      ref = torch.tensor([[0., 0.], [2., 0.], [5., 0.]])
      query = torch.tensor([[1., 0.], [4., 0.]])
      dist, idx = knn(query, ref, 2)
      assert idx.tolist() == [[0, 1], [2, 1]]
      assert torch.allclose(dist, torch.tensor([[1., 1.], [1., 4.]]))

Scoring Rubric: distances 3; topk 3; k check 1; shape 1; tests 2.
Likely Follow-ups: What if N is very large? How would you exclude a self neighbor?
Common Failure Modes: letting topk default to the largest values; sorting along the query dimension; k out of range; misunderstanding the tie order.

LC16: K-Means Assignment Step

Tags: P0, K-means, tensor, 15–20 minutes
Evidence / Derivation: A. A Zoom Research Scientist candidate directly reports a K-means implementation question, and it recurs across several general ML interview reports.

Prompt

Given data [N,D] and centroids [K,D], return the nearest cluster id [N] for every point.

Clarify First: squared Euclidean distance is used; ties may follow the default argmin result.
Approach: build the [N,K] distance matrix and take argmin along the K dimension.

Reference Implementation and Minimal Tests

import torch
      
      def assign_clusters(x, centroids):
          if x.ndim != 2 or centroids.ndim != 2:
              raise ValueError("incompatible shapes")
          if x.shape[1] != centroids.shape[1]:
              raise ValueError("incompatible shapes")
          differences = x[:, None, :] - centroids[None, :, :]
          distances = differences.square().sum(dim=-1)
          return distances.argmin(dim=1)
      
      x = torch.tensor([[0., 0.], [1., 0.], [9., 0.]])
      c = torch.tensor([[0., 0.], [10., 0.]])
      assert assign_clusters(x, c).tolist() == [0, 0, 1]

Scoring Rubric: broadcast shapes 4; argmin dimension 2; checks 2; tests 2.
Likely Follow-ups: How would the matrix formula from LC14 reduce memory? What is the time complexity?
Common Failure Modes: taking argmin over the wrong dimension; reducing over the D dimension as well; taking the square root of the distances.

LC17: K-Means Centroid Update and Empty Clusters

Tags: P0, K-means, edge case, 20–25 minutes
Evidence / Derivation: A. From the same K-means family. The empty cluster case is a frequent follow-up rather than observed question text.

Prompt

Given x [N,D], assignments [N], and the previous centroids [K,D], compute the new centroids. An empty cluster keeps its old value.

Clarify First: the empty cluster policy; whether the assignments are guaranteed to lie in [0,K); whether full vectorization is required.
Approach: accumulate the per cluster feature sums and counts with index_add_, divide for the nonempty clusters, and keep the cloned old values for the empty ones.

Reference Implementation and Minimal Tests

import torch
      
      def update_centroids(x, assignments, old_centroids):
          k, d = old_centroids.shape
          if x.shape[1] != d or assignments.shape != (x.shape[0],):
              raise ValueError("incompatible shapes")
          sums = torch.zeros_like(old_centroids)
          sums.index_add_(0, assignments, x)
          counts = torch.zeros(k, dtype=x.dtype, device=x.device)
          point_counts = torch.ones(
              x.shape[0],
              dtype=x.dtype,
              device=x.device,
          )
          counts.index_add_(0, assignments, point_counts)
          new = old_centroids.clone()
          nonempty = counts > 0
          new[nonempty] = sums[nonempty] / counts[nonempty, None]
          return new
      
      x = torch.tensor([[0., 0.], [2., 0.], [10., 0.]])
      a = torch.tensor([0, 0, 1])
      old = torch.tensor([[1., 1.], [9., 1.], [99., 99.]])
      new = update_centroids(x, a, old)
      assert torch.allclose(new, torch.tensor([[1., 0.], [10., 0.], [99., 99.]]))

Scoring Rubric: sums and counts 4; empty clusters 3; shape and device 1; tests 2.
Likely Follow-ups: What other empty cluster policies exist? How would you write this with scatter_add?
Common Failure Modes: dividing by zero for an empty cluster; creating a CPU tensor and causing a device mismatch; overwriting the old centroids in place and making debugging harder.

LC18: Full K-Means with a Fixed Iteration Count

Tags: P0, algorithm composition, 25–30 minutes
Evidence / Derivation: A. The Zoom K-means question directly. This is the version that fits a half hour ceiling.

Prompt

Initialize from the first k data points, run a fixed number of assignment and update rounds, and return the centroids and assignments. An empty cluster keeps its old center.

Clarify First: k-means++ is not needed; the iteration count is fixed and no tolerance is required; the input has at least k points.
Approach: compose the assignment and update steps, then reassign once after the loop so the returned assignments match the final centers.

Reference Implementation and Minimal Tests

import torch
      
      def kmeans(x, k, iterations=10):
          if x.ndim != 2:
              raise ValueError("invalid input")
          if not 1 <= k <= x.shape[0]:
              raise ValueError("invalid input")
          if iterations < 0:
              raise ValueError("invalid input")
          centroids = x[:k].clone()
          for _ in range(iterations):
              differences = x[:, None, :] - centroids[None, :, :]
              distances = differences.square().sum(dim=-1)
              assignment = distances.argmin(dim=1)
              sums = torch.zeros_like(centroids)
              counts = torch.zeros(k, dtype=x.dtype, device=x.device)
              sums.index_add_(0, assignment, x)
              point_counts = torch.ones(
                  x.shape[0],
                  dtype=x.dtype,
                  device=x.device,
              )
              counts.index_add_(0, assignment, point_counts)
              nonempty = counts > 0
              centroids[nonempty] = sums[nonempty] / counts[nonempty, None]
          final_differences = x[:, None, :] - centroids[None, :, :]
          final_distances = final_differences.square().sum(dim=-1)
          return centroids, final_distances.argmin(dim=1)
      
      x = torch.tensor([[0.], [1.], [9.], [10.]])
      centroids, assignment = kmeans(x, 2, 5)
      assert torch.allclose(centroids.sort(dim=0).values, torch.tensor([[0.5], [9.5]]))
      assert set(assignment.tolist()) == {0, 1}

Scoring Rubric: correct loop 3; assignment and update 3; edge checks 1; final consistency 1; tests 2.
Likely Follow-ups: What convergence criterion would you use? How much does initialization affect the result? How would you batch or chunk this?
Common Failure Modes: returning the stale assignments; NaN from an empty cluster; initializing from a view of x that a later in place write corrupts.

LC19: Amino Acid Tokenization and One-Hot Encoding

Tags: P1, protein overlay, indexing, 15–20 minutes
Evidence / Derivation: D. The Lila role involves biological sequences. This exercise only adds the domain framing and does not claim a publicly reported question.

Prompt

Encode a list of equal length sequences over the 20 standard amino acid characters into integer ids [B,L] and one-hot vectors [B,L,20]. Raise on an unknown character.

Clarify First: whether padding or unknown tokens are needed; this version handles only equal length standard characters.
Approach: build a fixed alphabet mapping and convert the ids character by character, using the PyTorch API for the one-hot step. A Python loop is reasonable here for string parsing; the model computation itself needs no loop.

Reference Implementation and Minimal Tests

import torch
      
      AMINO_ACIDS = "ACDEFGHIKLMNPQRSTVWY"
      AA_TO_ID = {}
      for amino_acid_id, amino_acid in enumerate(AMINO_ACIDS):
          AA_TO_ID[amino_acid] = amino_acid_id
      
      def encode_proteins(sequences):
          if not sequences:
              raise ValueError("sequences must be nonempty and equal length")
          sequence_length = len(sequences[0])
          for sequence in sequences:
              if len(sequence) != sequence_length:
                  raise ValueError("sequences must be nonempty and equal length")
          token_rows = []
          try:
              for sequence in sequences:
                  token_ids = []
                  for amino_acid in sequence:
                      token_ids.append(AA_TO_ID[amino_acid])
                  token_rows.append(token_ids)
          except KeyError as exc:
              raise ValueError(f"unknown amino acid: {exc.args[0]}") from exc
          ids = torch.tensor(token_rows, dtype=torch.long)
          one_hot = torch.nn.functional.one_hot(
              ids,
              num_classes=len(AMINO_ACIDS),
          ).float()
          return ids, one_hot
      
      ids, one_hot = encode_proteins(["ACD", "WYA"])
      assert ids.shape == (2, 3)
      assert one_hot.shape == (2, 3, 20)
      assert torch.equal(one_hot.sum(-1), torch.ones(2, 3))

Scoring Rubric: stable vocabulary 2; id shape 2; one-hot 2; input validation 2; tests 2.
Likely Follow-ups: How would you design the padding and unknown tokens? What is the trade-off between one-hot and a learned embedding?
Common Failure Modes: building the vocabulary from a set and getting an unstable order; building a tensor directly from unequal lengths; treating padding as a real amino acid.

LC20: Sequence Classifier with Padding

Tags: P1, embedding, mask, MLP, 25–30 minutes
Evidence / Derivation: B. Combines three supported capabilities, the MLP, sequences, and masking, into one half hour scientific ML question.

Prompt

Given padded token ids [B,L] and a valid mask [B,L], use an embedding, a masked mean, and an MLP to produce sequence level logits [B,C].

Clarify First: whether the padding id is reserved; a fully padded row returns a zero pooled vector; the output is logits.
Approach: embed to [B,L,D], average under the mask to [B,D], then pass through a two-layer MLP.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      class SequenceClassifier(nn.Module):
          def __init__(self, vocab_size, embed_dim, hidden_dim, classes, padding_idx=0):
              super().__init__()
              self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=padding_idx)
              self.fc1 = nn.Linear(embed_dim, hidden_dim)
              self.fc2 = nn.Linear(hidden_dim, classes)
      
          def forward(self, token_ids, mask):
              if token_ids.shape != mask.shape:
                  raise ValueError("token_ids and mask must have shape [B,L]")
              x = self.embedding(token_ids)
              weights = mask.to(x.dtype).unsqueeze(-1)
              pooled = (x * weights).sum(1) / weights.sum(1).clamp_min(1)
              hidden = self.fc1(pooled)
              activated = torch.relu(hidden)
              return self.fc2(activated)
      
      model = SequenceClassifier(21, 8, 12, 3)
      tokens = torch.tensor([[1, 2, 3, 0], [4, 5, 0, 0]])
      mask = tokens != 0
      logits = model(tokens, mask)
      assert logits.shape == (2, 3)
      logits.sum().backward()
      assert model.embedding.weight.grad is not None

Scoring Rubric: embedding 2; masked pooling 3; MLP and logits 2; shape 1; tests 2.
Likely Follow-ups: What are the limits of mean pooling? How would you switch to attention pooling? Does the padding embedding get updated?
Common Failure Modes: averaging over the padding; a final softmax; a mask with the wrong device or dtype; dividing by zero on a fully padded row.

LC21: Scaled Dot-Product Attention

Tags: P1, attention, shape, 20–25 minutes
Evidence / Derivation: A. A Cohere candidate reports an attention module implementation, Mistral and xAI candidates report PyTorch multi-head attention, BigHat reports a Transformer implementation discussion, and Lila explicitly says Transformers could be covered.

Prompt

Given q [B,Lq,D], k [B,Lk,D], and v [B,Lk,Dv], implement scaled dot-product attention without masking, returning both the output and the weights.

Clarify First: the softmax runs along the key dimension; the scale is sqrt(D); no dropout is needed.
Approach: q @ k.transpose(-2,-1) gives [B,Lq,Lk]. Divide by the scale, take the softmax along the last dimension, then multiply by v.

Reference Implementation and Minimal Tests

import math
      import torch
      
      def scaled_dot_product_attention(q, k, v):
          if q.ndim != 3 or k.ndim != 3 or v.ndim != 3:
              raise ValueError("q, k, v must be rank 3")
          if q.shape[0] != k.shape[0]:
              raise ValueError("incompatible shapes")
          if k.shape[:2] != v.shape[:2]:
              raise ValueError("incompatible shapes")
          if q.shape[-1] != k.shape[-1]:
              raise ValueError("incompatible shapes")
          scores = q @ k.transpose(-2, -1) / math.sqrt(q.shape[-1])
          weights = torch.softmax(scores, dim=-1)
          output = weights @ v
          return output, weights
      
      q = torch.tensor([[[1., 0.]]])
      k = torch.tensor([[[1., 0.], [0., 1.]]])
      v = torch.tensor([[[2., 0.], [0., 4.]]])
      out, weights = scaled_dot_product_attention(q, k, v)
      assert out.shape == (1, 1, 2)
      assert weights.shape == (1, 1, 2)
      assert torch.allclose(weights.sum(-1), torch.ones(1, 1))

Scoring Rubric: score shape 3; scale 2; softmax dimension 2; output 1; tests 2.
Likely Follow-ups: Why scale at all? What is the time and space complexity? Can Dv differ from D?
Common Failure Modes: transposing the wrong dimension; taking the softmax over queries; forgetting the scale; passing integer tensors and hitting a dtype problem.

LC22: Attention with a Padding Mask

Tags: P1, attention, mask, 20–25 minutes
Evidence / Derivation: B. A safe combination of the observed attention implementation with variable length biological sequences.

Prompt

Extend LC21 with a key validity mask [B,Lk] so that False positions receive no attention weight. Assume every sequence has at least one valid key.

Clarify First: the mask applies to keys only; True means valid; an all False row is undefined input.
Approach: broadcast the mask to [B,1,Lk], fill the invalid scores with the smallest finite value of the dtype, then take the softmax.

Reference Implementation and Minimal Tests

import math
      import torch
      
      def masked_attention(q, k, v, key_valid):
          if key_valid.shape != k.shape[:2]:
              raise ValueError("each sequence needs at least one valid key")
          if not key_valid.any(dim=1).all():
              raise ValueError("each sequence needs at least one valid key")
          scores = q @ k.transpose(-2, -1) / math.sqrt(q.shape[-1])
          invalid = ~key_valid.bool().unsqueeze(1)
          fill_value = torch.finfo(scores.dtype).min
          scores = scores.masked_fill(invalid, fill_value)
          weights = torch.softmax(scores, dim=-1)
          output = weights @ v
          return output, weights
      
      q = torch.randn(1, 2, 3)
      k = torch.randn(1, 3, 3)
      v = torch.randn(1, 3, 4)
      out, w = masked_attention(q, k, v, torch.tensor([[1, 1, 0]], dtype=torch.bool))
      assert out.shape == (1, 2, 4)
      assert torch.equal(w[..., 2], torch.zeros(1, 2))
      assert torch.allclose(w.sum(-1), torch.ones(1, 2))

Scoring Rubric: mask semantics 2; broadcasting 3; masking before the softmax 2; edge check 1; tests 2.
Likely Follow-ups: How would you handle query padding? Why can you not simply zero the weights after the softmax?
Common Failure Modes: inverting the mask; filling with zero instead of a large negative value; masking after the softmax without renormalizing; getting NaN or uniform weights for a fully padded row.

LC23: Causal Self-Attention Mask

Tags: P1, Transformer, mask, 15–20 minutes
Evidence / Derivation: A for the attention family; this causal variant is a practice extension. A protein encoder does not necessarily need a causal mask.

Prompt

Given self-attention scores [B,L,L], build a causal mask so that position i cannot see any j > i, and return the softmax weights.

Clarify First: a position may see itself; the mask is identical across the batch; padding masks are out of scope here.
Approach: build an invalid mask that is True strictly above the diagonal, broadcast it over the batch, then apply a masked fill.

Reference Implementation and Minimal Tests

import torch
      
      def causal_weights(scores):
          if scores.ndim != 3 or scores.shape[-2] != scores.shape[-1]:
              raise ValueError("scores must be [B,L,L]")
          length = scores.shape[-1]
          future = torch.triu(
              torch.ones(length, length, dtype=torch.bool, device=scores.device),
              diagonal=1,
          )
          fill_value = torch.finfo(scores.dtype).min
          masked = scores.masked_fill(future.unsqueeze(0), fill_value)
          return torch.softmax(masked, dim=-1)
      
      scores = torch.zeros(1, 3, 3)
      w = causal_weights(scores)
      assert torch.equal(w[0, 0], torch.tensor([1., 0., 0.]))
      assert w[0, 1, 2].item() == 0.0
      assert torch.allclose(w.sum(-1), torch.ones(1, 3))

Scoring Rubric: mask orientation 4; diagonal rule 2; device 1; softmax 1; tests 2.
Likely Follow-ups: Why do an encoder and an autoregressive decoder use different masks? How would you combine this with a padding mask?
Common Failure Modes: masking the past with a lower triangle; masking the diagonal too; a mask shape that only happens to work at batch size 1.

LC24: Multi-Head Split and Merge

Tags: P1, attention, reshape, 20–25 minutes
Evidence / Derivation: A. Mistral and xAI candidates directly report implementing multi-head attention from scratch. For a half hour question, split and merge is the safer granularity.

Prompt

Implement split_heads mapping [B,L,D] → [B,H,L,D/H] and its inverse merge_heads. D must be divisible by H.

Clarify First: the head dimension goes before the sequence dimension; whether the returned tensor must be contiguous.
Approach: view out the head dimension and transpose L and H. The inverse transposes back, then reshapes after making the tensor contiguous.

Reference Implementation and Minimal Tests

import torch
      
      def split_heads(x, heads):
          if x.ndim != 3 or x.shape[-1] % heads != 0:
              raise ValueError("D must be divisible by heads")
          b, length, dim = x.shape
          reshaped = x.reshape(b, length, heads, dim // heads)
          return reshaped.transpose(1, 2)
      
      def merge_heads(x):
          if x.ndim != 4:
              raise ValueError("x must be [B,H,L,Dh]")
          b, heads, length, head_dim = x.shape
          transposed = x.transpose(1, 2).contiguous()
          return transposed.reshape(b, length, heads * head_dim)
      
      x = torch.arange(2 * 3 * 8).reshape(2, 3, 8)
      h = split_heads(x, 4)
      assert h.shape == (2, 4, 3, 2)
      assert torch.equal(merge_heads(h), x)

Scoring Rubric: split shape 3; transpose 2; inverse 3; tests 2.
Likely Follow-ups: Why does each head scale by sqrt(D/H)? Why is contiguous so often needed after a transpose?
Common Failure Modes: reshaping straight to [B,H,L,Dh] without swapping dimensions; scrambling elements on merge; not checking divisibility.

LC25: Sinusoidal Positional Encoding

Tags: P1, position, tensor construction, 20–25 minutes
Evidence / Derivation: B. A Latent Labs candidate was directly asked about positional embeddings; this specific sinusoidal implementation is a practice variant.

Prompt

Implement a sinusoidal positional encoding [L,D] for length L and dimension D. Even columns use sine and odd columns use cosine. Odd values of D must work.

Clarify First: the classic base of 10000 is used; the output is float; no batch dimension is needed.
Approach: build positions as [L,1] and an exponent vector over the even dimensions. Form the angles, assign the two column families separately, and truncate the cosine columns when D is odd.

Reference Implementation and Minimal Tests

import math
      import torch
      
      def sinusoidal_encoding(length, dim, device=None):
          if length < 0 or dim <= 0:
              raise ValueError("invalid length or dimension")
          position = torch.arange(length, device=device, dtype=torch.float32).unsqueeze(1)
          even = torch.arange(0, dim, 2, device=device, dtype=torch.float32)
          exponent = -math.log(10000.0) * even / dim
          frequencies = torch.exp(exponent)
          angles = position * frequencies
          pe = torch.zeros(length, dim, device=device)
          pe[:, 0::2] = torch.sin(angles)
          if dim > 1:
              pe[:, 1::2] = torch.cos(angles[:, :pe[:, 1::2].shape[1]])
          return pe
      
      pe = sinusoidal_encoding(4, 5)
      assert pe.shape == (4, 5)
      assert torch.allclose(pe[0, 0::2], torch.zeros(3))
      assert torch.allclose(pe[0, 1::2], torch.ones(2))

Scoring Rubric: frequencies 3; sine and cosine layout 3; odd D 2; tests 2.
Likely Follow-ups: How do learned positions, RoPE, and sinusoidal encodings differ? Why does a sequence model need positional information at all?
Common Failure Modes: using the column index instead of the even index in the exponent; an assignment mismatch when D is odd; broadcasting the position and feature dimensions incorrectly.

LC26: VAE Reparameterization

Tags: P2, VAE, stochastic, 10–15 minutes
Evidence / Derivation: C. The Lila recruiting conversation confirms only that a VAE “could” appear, and no specific VAE coding question at an adjacent company was verified.

Prompt

Given mu and logvar, sample z during training and return mu during evaluation. Accepting a generator argument is out of scope.

Clarify First: logvar is the log variance, not the log standard deviation; the train and eval semantics; the shapes match exactly.
Approach: std=exp(0.5*logvar), eps=randn_like(std), z=mu+std*eps.

Reference Implementation and Minimal Tests

import torch
      
      def reparameterize(mu, logvar, training=True):
          if mu.shape != logvar.shape:
              raise ValueError("shape mismatch")
          if not training:
              return mu
          std = torch.exp(0.5 * logvar)
          noise = torch.randn_like(std)
          return mu + std * noise
      
      torch.manual_seed(0)
      mu = torch.zeros(10000)
      logvar = torch.zeros(10000)
      z = reparameterize(mu, logvar)
      assert abs(z.mean().item()) < 0.05
      assert abs(z.std().item() - 1.0) < 0.05
      assert torch.equal(reparameterize(mu, logvar, training=False), mu)

Scoring Rubric: the 0.5 factor 3; randn_like 2; a gradient preserving expression 2; eval semantics 1; tests 2.
Likely Follow-ups: Why does this allow backpropagation? Must evaluation always return mu?
Common Failure Modes: treating exp(logvar) as the standard deviation; drawing a sample from Normal(mu,std) that cannot be reparameterized; using torch.randn(mu.shape) and causing a device mismatch.

LC27: Diagonal Gaussian KL

Tags: P2, VAE, loss, 10–15 minutes
Evidence / Derivation: C. The same low confidence minimal VAE coverage as LC26.

Prompt

Implement the KL divergence of q(z|x)=N(mu, diag(exp(logvar))) against a standard normal prior. Sum over the latent dimension first, then average over the batch.

Clarify First: the latent dimension is the last one; the input is [B,Z]; the return value is a scalar mean.
Approach: per sample, -0.5 * sum(1 + logvar - mu² - exp(logvar)).

Reference Implementation and Minimal Tests

import torch
      
      def gaussian_kl(mu, logvar):
          if mu.shape != logvar.shape or mu.ndim < 2:
              raise ValueError("expected matching [B,...,Z] tensors")
          variance = logvar.exp()
          per_element = 1 + logvar - mu.square() - variance
          per_sample = -0.5 * per_element.sum(dim=-1)
          return per_sample.mean()
      
      mu = torch.zeros(4, 3, requires_grad=True)
      logvar = torch.zeros(4, 3, requires_grad=True)
      kl = gaussian_kl(mu, logvar)
      assert torch.allclose(kl, torch.tensor(0.0))
      kl.backward()
      assert mu.grad is not None
      assert logvar.grad is not None
      mu2 = torch.ones(2, 3)
      assert torch.allclose(gaussian_kl(mu2, torch.zeros_like(mu2)), torch.tensor(1.5))

Scoring Rubric: formula 5; reduction 2; shape 1; tests 2.
Likely Follow-ups: Why is the KL nonnegative? How does beta-VAE change it? What is posterior collapse?
Common Failure Modes: flipping the sign; dropping exp(logvar); averaging over the batch and the latent dimension together and changing the scale.

LC28: Diffusion Forward Noising

Tags: P1, diffusion, broadcasting, 15–20 minutes
Evidence / Derivation: A/B. BigHat discusses a diffusion implementation in an antibody design case, Latent uses a protein diffusion take-home, and Lila confirms diffusion “could” appear. This q_sample is a safe reconstruction of a small component.

Prompt

Given a clean sample x0 [B,...], noise of the same shape, and a per sample alpha_bar_t [B], produce x_t.

Clarify First: alpha_bar_t has already been gathered by timestep; it lies in [0,1]; x0 may have any rank.
Approach: reshape [B] to [B,1,...,1], then apply sqrt(alpha_bar)*x0 + sqrt(1-alpha_bar)*noise.

Reference Implementation and Minimal Tests

import torch
      
      def q_sample(x0, noise, alpha_bar_t):
          if x0.shape != noise.shape or alpha_bar_t.shape != (x0.shape[0],):
              raise ValueError("incompatible shapes")
          if ((alpha_bar_t < 0) | (alpha_bar_t > 1)).any():
              raise ValueError("alpha_bar_t must be in [0,1]")
          batch_size = x0.shape[0]
          extra_dimensions = x0.ndim - 1
          broadcast_shape = (batch_size,) + (1,) * extra_dimensions
          a = alpha_bar_t.reshape(broadcast_shape)
          a = a.to(dtype=x0.dtype, device=x0.device)
          return a.sqrt() * x0 + (1 - a).sqrt() * noise
      
      x0 = torch.tensor([[1., 2.], [3., 4.]])
      noise = torch.tensor([[10., 20.], [30., 40.]])
      out = q_sample(x0, noise, torch.tensor([1.0, 0.0]))
      assert torch.equal(out[0], x0[0])
      assert torch.equal(out[1], noise[1])

Scoring Rubric: formula 4; generic broadcasting 3; range and shape checks 1; boundary tests 2.
Likely Follow-ups: How does alpha_bar relate to alpha and beta? Why is t sampled randomly during training?
Common Failure Modes: multiplying [B] against [B,L,D] directly and broadcasting incorrectly; confusing alpha with alpha_bar; forgetting the square root on both coefficients.

LC29: Diffusion Noise Prediction Loss

Tags: P1, diffusion, training objective, 20–25 minutes
Evidence / Derivation: Same as LC28. This is the minimal training objective of a full diffusion pipeline, not an exact prompt from any report.

Prompt

Given a model, x0, a batch of timesteps t, and the cumulative schedule alpha_bars [T], sample the noise, build x_t, have the model predict the noise, and return the MSE.

Clarify First: the model takes (x_t,t) and returns the same shape; t is long and within range; only the training loss is required, not a sampler.

Approach: gather alpha_bars[t], draw the noise, build xt, then compare the prediction against the true noise with MSE.

Reference Implementation and Minimal Tests

import torch
      from torch import nn
      
      def diffusion_noise_loss(model, x0, t, alpha_bars):
          if t.dtype != torch.long or t.shape != (x0.shape[0],):
              raise ValueError("t must be long [B]")
          if (t < 0).any() or (t >= alpha_bars.numel()).any():
              raise ValueError("t out of range")
          noise = torch.randn_like(x0)
          batch_size = x0.shape[0]
          extra_dimensions = x0.ndim - 1
          broadcast_shape = (batch_size,) + (1,) * extra_dimensions
          a = alpha_bars[t].to(x0).reshape(broadcast_shape)
          xt = a.sqrt() * x0 + (1 - a).sqrt() * noise
          predicted = model(xt, t)
          if predicted.shape != noise.shape:
              raise ValueError("model output shape mismatch")
          error = predicted - noise
          return error.square().mean()
      
      class ZeroNoiseModel(nn.Module):
          def forward(self, xt, t):
              return torch.zeros_like(xt)
      
      torch.manual_seed(0)
      x0 = torch.randn(3, 4)
      timesteps = torch.tensor([0, 1, 2])
      alpha_bars = torch.tensor([0.9, 0.5, 0.1])
      loss = diffusion_noise_loss(ZeroNoiseModel(), x0, timesteps, alpha_bars)
      assert loss.ndim == 0
      assert loss.item() >= 0
      assert torch.isfinite(loss)

Scoring Rubric: schedule gather 2; broadcasting 2; q_sample 2; target and loss 2; validation and tests 2.
Likely Follow-ups: How does predicting x0 or v differ? Should different values of t be weighted? Where does conditional input enter?
Common Failure Modes: using one t for the whole batch; taking the loss against x0 instead of the noise; wasting time on a reverse sampler; not checking the model output shape.

LC30: Training Loop Code Review and Fix

Tags: P0, debugging, code comprehension, 25–30 minutes
Evidence / Derivation: A. TORC, Wayve, NVIDIA, PathAI, and Tempus all provide signal on reading, debugging, or coding style in a training loop.

Prompt

Review the following faulty logic, whose stated goal is to “train a binary classification model”: the model outputs logits, yet a sigmoid is applied before binary_cross_entropy_with_logits; gradients are never cleared; validation runs in train mode and builds a graph. Write separate, correct training and evaluation epoch functions.

Clarify First: each batch is (x, y); the target can be reshaped to match the logits; the loss must be averaged with sample weighting.
Approach: keep the logits and make training and evaluation separate entry points. The training function owns zero_grad/backward/step; the evaluation function owns eval and inference mode. Both accumulate the detached batch loss times the batch size.

Reference Implementation and Minimal Tests

import torch
      import torch.nn.functional as F
      
      def train_binary_epoch(model, batches, optimizer):
          model.train()
          loss_sum = 0.0
          sample_count = 0
          for x, y in batches:
              optimizer.zero_grad()
              logits = model(x)
              targets = y.to(logits.dtype).reshape_as(logits)
              loss = F.binary_cross_entropy_with_logits(logits, targets)
              loss.backward()
              optimizer.step()
              batch_size = x.shape[0]
              loss_sum += loss.detach().item() * batch_size
              sample_count += batch_size
          if sample_count == 0:
              raise ValueError("empty epoch")
          return loss_sum / sample_count
      
      @torch.inference_mode()
      def evaluate_binary_epoch(model, batches):
          model.eval()
          loss_sum = 0.0
          sample_count = 0
          for x, y in batches:
              logits = model(x)
              targets = y.to(logits.dtype).reshape_as(logits)
              loss = F.binary_cross_entropy_with_logits(logits, targets)
              batch_size = x.shape[0]
              loss_sum += loss.item() * batch_size
              sample_count += batch_size
          if sample_count == 0:
              raise ValueError("empty epoch")
          return loss_sum / sample_count
      
      model = torch.nn.Linear(2, 1)
      batches = [(torch.randn(3, 2), torch.tensor([0, 1, 0])),
                 (torch.randn(1, 2), torch.tensor([1]))]
      optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
      weight_before_train = model.weight.detach().clone()
      train_loss = train_binary_epoch(model, batches, optimizer)
      assert train_loss >= 0
      assert not torch.equal(weight_before_train, model.weight)
      
      weight_before_eval = model.weight.detach().clone()
      gradient_before_eval = model.weight.grad.detach().clone()
      eval_loss = evaluate_binary_epoch(model, batches)
      assert eval_loss >= 0
      assert torch.equal(weight_before_eval, model.weight)
      assert torch.equal(gradient_before_eval, model.weight.grad)

Scoring Rubric: spotting the double sigmoid 2; clearing gradients 2; separate train and eval modes plus inference mode 2; aggregation 2; tests 2.
Likely Follow-ups: Why can stale .grad values still exist after validation? How would you prove that no new graph was built? How would you add gradient clipping?
Common Failure Modes: assuming eval() disables autograd; setting the grads to None before validation and then writing a broken test; thresholding logits at 0.5.

LC31: ReLU Forward and Local Backward

Tags: P0, activation, manual gradient, 10–15 minutes
Evidence / Derivation: D. A Zoom Research Scientist candidate was asked about ReLU and its weaknesses, while first-person research interview retrospectives include manual backward passes. This exact two-part prompt is a bounded fundamentals variant.

Prompt

Given x and an upstream gradient grad_out of the same shape, implement ReLU’s forward output and its local backward gradient. Define the derivative at zero as zero. Do not use custom autograd.

Clarify First: elementwise ReLU only; no in-place update; x and grad_out are floating tensors of the same shape.
Approach: clamp the forward value at zero, then multiply the upstream gradient by the positive-input mask.

Reference Implementation and Minimal Tests

import torch
      
      def relu_forward_backward(x, grad_out):
          if x.shape != grad_out.shape:
              raise ValueError("shape mismatch")
          y = x.clamp_min(0)
          grad_x = grad_out * (x > 0).to(grad_out.dtype)
          return y, grad_x
      
      x = torch.tensor([-2.0, 0.0, 3.0])
      grad_out = torch.tensor([1.0, 2.0, -4.0])
      y, grad_x = relu_forward_backward(x, grad_out)
      assert torch.equal(y, torch.tensor([0.0, 0.0, 3.0]))
      assert torch.equal(grad_x, torch.tensor([0.0, 0.0, -4.0]))
      
      x_auto = x.clone().requires_grad_()
      torch.relu(x_auto).backward(grad_out)
      assert torch.equal(grad_x, x_auto.grad)

Scoring Rubric: forward 3; chain rule 3; zero convention 1; shape check and tests 3.
Likely Follow-ups: Why can ReLU units die? How would LeakyReLU change the mask?
Common Failure Modes: returning only the binary mask; forgetting grad_out; assigning derivative one at zero without stating it.

LC32: Linear Layer Forward and Local Backward

Tags: P0, linear layer, manual gradient, 15–20 minutes
Evidence / Derivation: D. Manual backward passes and neural-layer forward/backward exercises appear in Silvia Sapora’s and Nishanth Kumar’s first-person research interview retrospectives. This exact linear-layer prompt is a short practice reconstruction.

Prompt

For x [B,D], weight [D,H], bias [H], and upstream grad_out [B,H], return y, grad_x, grad_weight, and grad_bias. Do not call autograd inside the implementation.

Clarify First: the batch gradient is summed for parameters; no activation or loss is included.
Approach: apply the chain rule to y = x @ weight + bias; the three gradients are matrix products plus a batch sum.

Reference Implementation and Minimal Tests

import torch
      
      def linear_forward_backward(x, weight, bias, grad_out):
          if x.ndim != 2 or weight.ndim != 2:
              raise ValueError("invalid parameter shapes")
          if bias.shape != (weight.shape[1],):
              raise ValueError("invalid parameter shapes")
          expected_grad_shape = (x.shape[0], weight.shape[1])
          if x.shape[1] != weight.shape[0]:
              raise ValueError("incompatible shapes")
          if grad_out.shape != expected_grad_shape:
              raise ValueError("incompatible shapes")
          y = x @ weight + bias
          grad_x = grad_out @ weight.T
          grad_weight = x.T @ grad_out
          grad_bias = grad_out.sum(dim=0)
          return y, grad_x, grad_weight, grad_bias
      
      x = torch.tensor([[1.0, 2.0], [-1.0, 3.0]], requires_grad=True)
      weight = torch.tensor([[2.0, -1.0], [0.5, 4.0]], requires_grad=True)
      bias = torch.tensor([0.25, -0.5], requires_grad=True)
      grad_out = torch.tensor([[1.0, -2.0], [3.0, 0.5]])
      y, grad_x, grad_weight, grad_bias = linear_forward_backward(
          x,
          weight,
          bias,
          grad_out,
      )
      (x @ weight + bias).backward(grad_out)
      assert y.shape == grad_out.shape
      assert torch.allclose(grad_x, x.grad)
      assert torch.allclose(grad_weight, weight.grad)
      assert torch.allclose(grad_bias, bias.grad)

Scoring Rubric: forward 2; grad_x 2; parameter gradients 4; shapes and tests 2.
Likely Follow-ups: Compose this with LC31 for one MLP block. Why is grad_bias summed over the batch?
Common Failure Modes: transposing the wrong operand; averaging parameter gradients without being asked; omitting the batch dimension check.

LC33: Collate Variable-Length Protein Sequences

Tags: P1, data loading, padding mask, 15–20 minutes
Evidence / Derivation: A for the capability. An OpenAI Research Engineer candidate reports implementing a data loader and tests; a Rakuten Research Scientist candidate reports a large-JSON data loader. Protein collation is the shorter domain transfer.

Prompt

Implement collate_sequences(batch, pad_id=0) for a nonempty list of (token_ids, label) pairs. Each sequence is a nonempty 1D LongTensor. Return padded tokens [B,Lmax], a boolean valid-token mask, and labels [B].

Clarify First: all sequences share dtype and device; labels are integer classes; this question stops at collation, not a full Dataset or worker pipeline.
Approach: compute lengths, allocate once, fill each row, and obtain the mask by comparing positions with lengths.

Reference Implementation and Minimal Tests

import torch
      
      def collate_sequences(batch, pad_id=0):
          if not batch:
              raise ValueError("empty batch")
          sequences, labels = zip(*batch)
          length_values = []
          for sequence in sequences:
              if sequence.ndim != 1 or sequence.numel() == 0:
                  raise ValueError("sequences must be nonempty 1D tensors")
              length_values.append(sequence.numel())
          device = sequences[0].device
          lengths = torch.tensor(length_values, device=device)
          max_length = int(lengths.max().item())
          tokens = torch.full(
              (len(sequences), max_length),
              pad_id,
              dtype=sequences[0].dtype,
              device=device,
          )
          positions = torch.arange(max_length, device=device)
          valid = positions[None, :] < lengths[:, None]
          for row, sequence in zip(tokens, sequences):
              row[:sequence.numel()] = sequence
          return tokens, valid, torch.tensor(labels, device=tokens.device)
      
      batch = [(torch.tensor([4, 8, 2]), 1), (torch.tensor([7]), 0)]
      tokens, valid, labels = collate_sequences(batch)
      assert torch.equal(tokens, torch.tensor([[4, 8, 2], [7, 0, 0]]))
      assert torch.equal(valid, torch.tensor([[True, True, True], [True, False, False]]))
      assert torch.equal(labels, torch.tensor([1, 0]))

Scoring Rubric: lengths and allocation 3; padding 2; mask 2; labels and tests 2; empty-batch check 1.
Likely Follow-ups: When would IterableDataset be preferable for large JSON? How should padding interact with attention?
Common Failure Modes: confusing valid and padding masks; padding labels; implementing multiprocessing before the collate function works.

LC34: LayerNorm over the Last Dimension

Tags: P1, normalization, tensor operations, 10–15 minutes
Evidence / Derivation: D. A Mistral candidate reports debugging a pre-norm Transformer, but no verified source reports this exact LayerNorm implementation prompt. It is a bounded prerequisite exercise, checked against the official PyTorch semantics.

Prompt

Implement layer_norm_last(x, gamma, beta, eps) using basic tensor operations. Normalize only the last dimension, use the population variance, and preserve every leading dimension.

Clarify First: gamma and beta have shape [D], where D = x.shape[-1]; return the same shape as x.
Approach: retain the last dimension while computing mean and variance, multiply by rsqrt(var + eps), then apply the affine parameters by broadcasting.

Reference Implementation and Minimal Tests

import torch
      
      def layer_norm_last(x, gamma, beta, eps=1e-5):
          if x.ndim == 0:
              raise ValueError("invalid shapes")
          if gamma.shape != (x.shape[-1],):
              raise ValueError("invalid shapes")
          if beta.shape != gamma.shape:
              raise ValueError("invalid shapes")
          mean = x.mean(dim=-1, keepdim=True)
          var = (x - mean).square().mean(dim=-1, keepdim=True)
          normalized = (x - mean) * torch.rsqrt(var + eps)
          return normalized * gamma + beta
      
      x = torch.tensor([[1.0, 3.0], [2.0, 2.0]])
      gamma = torch.tensor([2.0, 0.5])
      beta = torch.tensor([0.25, -1.0])
      actual = layer_norm_last(x, gamma, beta)
      expected = torch.nn.functional.layer_norm(x, (2,), gamma, beta)
      assert torch.allclose(actual, expected)
      assert torch.allclose(actual[1], beta)

Scoring Rubric: reduction axes 3; population variance and epsilon 2; affine broadcast 2; shape check and tests 3.
Likely Follow-ups: How does LayerNorm differ from BatchNorm? What changes for RMSNorm?
Common Failure Modes: normalizing across the batch; using sample variance; dropping leading dimensions.

Live Coding Coverage Audit

The advanced models do not outnumber the fundamentals. The VAE appears in only two questions, both at evidence grade C, and diffusion covers only the forward pass and the noise loss, with no full sampler. LC31–LC34 add missing fundamentals as separate 10–20 minute components rather than one oversized exercise.

MF01: How Should You Choose among MSE, MAE, and Huber Loss?

Tags: P0, loss
Evidence: PathAI custom loss, Deep Genomics loss follow-up, and the Tempus model-training task; the explicit three-way comparison is a fundamentals coverage variant.

Answer

• MSE corresponds to the negative log-likelihood under Gaussian noise. It is smooth and its gradient grows with the error, but it is highly sensitive to outliers.
• MAE corresponds to Laplace-like noise and is more robust, but it is not differentiable at zero; in practice, a subgradient can be used.
• Huber uses a quadratic term for small errors and a linear term for large errors, balancing stable optimization with robustness to outliers.
• The choice should not be based on the loss name alone. Examine how labels are generated, whether extreme values are errors or real signals, the evaluation metric, and the output scale.
• Always report a validation metric aligned with the business or scientific objective; do not compare only training loss.

Likely Follow-ups: What if multitask labels have different scales? How would you model heteroscedastic noise?
Common Failure Modes: Claiming that MAE is always better; failing to determine whether an outlier is an error or an important signal.

MF02: Why Is BCEWithLogitsLoss Usually Better than Sigmoid Followed by BCE?

Tags: P0, numerical stability
Evidence: Fundamentals coverage for classification and training questions; official PyTorch semantics were verified.

Answer

• A logits-based loss combines sigmoid and log loss in a numerically stable log-sum-exp formulation, avoiding underflow and log(0) when probabilities approach 0 or 1.
• Keep logits in the model forward pass and feed them directly to the training loss; apply sigmoid only when probabilities are needed.
• In binary classification, a probability threshold of 0.5 is equivalent to a logit threshold of 0.
• Class imbalance can be handled with pos_weight or sample weights, but training weights must be distinguished from final calibration.

Likely Follow-ups: How do the loss shapes differ for multilabel and multiclass classification?
Common Failure Modes: Applying sigmoid in the model and again inside the loss; thresholding logits at 0.5.

MF03: What Inputs Does Cross Entropy Expect?

Tags: P0, classification, shape
Evidence: Prerequisite coverage for classification cases and PathAI ML fundamentals.

Answer

• Standard multiclass input is unnormalized logits with shape [B,C]; the target is usually a long tensor of class indices with shape [B].
• It is equivalent to log-softmax followed by negative log-likelihood, so softmax should not be applied beforehand.
• Softmax operates along the class dimension, and every sample’s target must be within the valid class range.
• Class weights, an ignore index, and label smoothing all change the loss semantics; state the reduction as well.

Likely Follow-ups: How would you handle soft targets or one-hot targets?
Common Failure Modes: Using floating-point class IDs as the target; applying softmax along the batch dimension.

MF04: What Are Bias and Variance?

Tags: P0, generalization
Evidence: BenevolentAI and InstaDeep candidates directly reported bias-variance and regularization questions.

Answer

• High bias means the model or hypothesis class is too constrained, so both training and validation performance are poor.
• High variance means the model is sensitive to perturbations in the training samples, yielding strong training performance but weak validation performance.
• More data usually reduces variance most directly. A more expressive model can reduce bias but may increase variance.
• Regularization, early stopping, sensible features, and a simpler model can reduce variance.
• In practice, inspect learning curves, variability across splits, and error decomposition rather than relying on one score.

Likely Follow-ups: Where does irreducible error from label noise appear?
Common Failure Modes: Attributing every validation gap to excessive model complexity while ignoring leakage or distribution shift.

MF05: How Do L1, L2, Dropout, and Early Stopping Differ?

Tags: P0, regularization
Evidence: PathAI regularization and BenevolentAI regularization.

Answer

• L1 encourages sparse weights; L2 penalizes weight magnitude and often improves conditioning and stability.
• Weight decay in an optimizer is not always equivalent to adding an L2 term to the loss, especially for some adaptive optimizers, so answer carefully.
• Dropout randomly masks activations during training and is disabled during evaluation; it changes representation-learning noise and creates ensemble-like behavior.
• Early stopping limits effective training time based on validation performance.
• Choose based on the model, data volume, and failure mode, then validate the choice instead of enabling every method at once.

Likely Follow-ups: Why is model.eval() important for dropout?
Common Failure Modes: Treating regularization as a remedy for data leakage.

MF06: What Are the Roles of the Training, Validation, and Test Sets?

Tags: P0, evaluation
Evidence: Zendesk directly asked about train, validation, and test sets; evaluation recurs across multiple cases.

Answer

• The training set is used to fit parameters.
• The validation set is used to choose models, hyperparameters, thresholds, and stopping points.
• The test set provides a final, ideally one-time estimate after the model and analysis decisions have been locked.
• Repeatedly changing the model based on test results turns the test set into a validation set and makes the estimate optimistically biased.
• In scientific ML, define the split unit explicitly, such as patient, protein family, experimental batch, or time, rather than merely splitting rows at random.

Likely Follow-ups: When is cross-validation appropriate? What does external validation add?
Common Failure Modes: Allowing the same donor or sequence family to appear in both train and test sets.

MF07: What Are the Most Common Forms of Leakage in Life-Science Data?

Tags: P0, scientific data
Evidence: Derived from evaluation requirements in DNA, protein, RNA, and assay cases; no source claims this is an actual Lila question.

Answer

• The same patient, sample, protein family, or close homolog appears across splits.
• Feature selection, normalization, or label-derived preprocessing is performed on the full dataset before splitting.
• Assay batch, site, plate, or time is strongly correlated with the label, so the model learns batch rather than biological mechanism.
• Features include future information unavailable at prediction time.
• Repeated measurements or highly similar candidates are not grouped during splitting.
• Use group, time, family or scaffold, or external splits, and check nearest-neighbor and batch baselines.

Likely Follow-ups: How would you quantify train-test similarity?
Common Failure Modes: Saying only that the data should be shuffled and randomly split.

MF08: How Should You Evaluate a Severely Imbalanced Classification Problem?

Tags: P0, metrics
Evidence: Flagship directly asked about skewed data; Tempus directly asked about pipeline metrics.

Answer

• Accuracy is often misleading. Report precision, recall, and PR-AUC, then choose an operating point based on the intended use.
• ROC-AUC can be supplementary, but when positives are rare, the precision-recall curve more directly exposes the relevant trade-off.
• Choose the threshold on validation data, not on the test set.
• Check calibration, subgroup performance, and confidence intervals.
• Training may use resampling, class weighting, or an appropriate loss, but evaluation should reflect the real prevalence.

Likely Follow-ups: In screening, should you prioritize recall or precision, and which downstream cost determines that choice?
Common Failure Modes: Reporting only F1 without explaining costs; treating a rebalanced validation set as the true deployment distribution.

MF09: How Do Calibration and Discrimination Differ?

Tags: P1, uncertainty, evaluation
Evidence: Safely derived from Tempus metrics, Merck interpretability, and Latent large-scale screening.

Answer

• Discrimination measures ranking or class-separation ability, as in AUROC.
• Calibration measures whether predicted probabilities match observed frequencies; among samples predicted at 0.8, roughly 80% should be positive.
• Good ranking does not guarantee good calibration, and the reverse is also true.
• Inspect reliability diagrams, Brier score, and expected calibration error. Methods such as temperature scaling should be fitted on an independent validation set.
• In experimental candidate selection, calibration affects the expected hit rate and the allocation of resources.

Likely Follow-ups: What happens to calibration under distribution shift?
Common Failure Modes: Calling the maximum softmax value a trustworthy probability without validation.

MF10: What Is a p-Value, and What Is It Not?

Tags: P0, statistics
Evidence: A Genentech Data Scientist candidate was directly asked to define a p-value.

Answer

• Conditional on the null hypothesis and test assumptions being true, the p-value is the probability of observing the current test statistic or one more extreme.
• It is not the probability that the null hypothesis is true, nor the probability that an effect is practically meaningful.
• The p-value depends on sample size, the chosen test, and multiple comparisons.
• Report effect size, confidence interval, experimental design, and assumption checks alongside it.
• When testing many genes or candidates, address multiple testing, for example by controlling the false discovery rate.

Likely Follow-ups: How do statistical significance and scientific significance differ?
Common Failure Modes: Saying that p=0.03 means the null hypothesis has only a 3% probability of being true.

MF11: Why Are Ordinary Gaussian Regression Models Often Inappropriate for RNA Count Data?

Tags: P0, omics, distribution
Evidence: insitro directly used an RNA-seq scenario with follow-ups on distributions and statistical tests.

Answer

• Counts are nonnegative and discrete, their variance often changes with the mean, and they exhibit varying library sizes and overdispersion.
• Poisson is a useful starting point, but real RNA counts often have variance greater than the mean, making the negative binomial more common.
• Normalization or offsets, batch effects, zero structure, and biological replicates all affect the model.
• A deep model still needs a sensible likelihood or transformation; using a neural network does not remove the need to model the observation distribution.
• Final testing should address multiple comparisons and effect size.

Likely Follow-ups: Is zero inflation always necessary? How would you distinguish technical zeros from biological zeros?
Common Failure Modes: Applying MSE directly to untreated raw counts.

MF12: How Should You Choose among SGD, Momentum, and Adam?

Tags: P0, optimization
Evidence: PathAI optimization and several reported training-loop interviews.

Answer

• SGD uses the current batch gradient. It is simple and memory-efficient but sensitive to learning rate and conditioning.
• Momentum smooths updates and accelerates movement along persistent directions.
• Adam maintains first- and second-moment estimates for each parameter. It often reaches a usable solution faster with sparse gradients or gradients at different scales, but generalization and weight-decay semantics must be validated.
• Tune the optimizer together with the learning-rate schedule, batch size, normalization, and training budget.
• Regardless of optimizer, monitor training and validation curves, gradient norms, and numerical stability.

Likely Follow-ups: Why does AdamW decouple weight decay?
Common Failure Modes: Claiming Adam is always better than SGD.

MF13: What Is Second-Order Optimization?

Tags: P1, optimization
Evidence: Second-order optimization appeared directly in a 2025 PathAI Machine Learning Engineer interview.

Answer

• First-order methods use only gradients; second-order methods also use the Hessian or an approximation to describe curvature.
• The idealized Newton step has the form H^{-1}g, which can correct for differences in conditioning across directions.
• For large neural networks, computing, storing, and inverting the full Hessian is generally infeasible.
• Practical methods use approximations such as quasi-Newton, Gauss-Newton, natural gradient, or Hessian-vector products.
• Compare per-step cost, memory, convergence speed, and behavior in nonconvex problems rather than merely saying that fewer iterations are required.

Likely Follow-ups: Why might the Hessian be indefinite?
Common Failure Modes: Equating Adam’s second-moment estimate with a true Newton method.

MF14: Why Do Vanishing and Exploding Gradients Occur?

Tags: P0, deep learning
Evidence: An InstaDeep AI Research Intern candidate was directly asked about vanishing gradients; training-debugging reports provide corroboration.

Answer

• In deep networks or long sequences, the chain rule multiplies Jacobians. Singular values persistently below 1 cause gradients to vanish, while values above 1 cause them to explode.
• Saturated sigmoid or tanh activations, poor initialization, and long recurrent paths worsen the problem.
• Residual connections, suitable initialization, normalization, nonsaturating activations, and gradient clipping mitigate different aspects.
• Clipping limits explosions but does not solve long-range information propagation or structural problems in the model.
• Diagnose the issue by inspecting gradient norms by layer, activation distributions, and training curves.

Likely Follow-ups: Why do residual connections help?
Common Failure Modes: Calling every failure to converge a vanishing-gradient problem.

MF15: Why Does Initialization Matter?

Tags: P0, optimization, MLP
Evidence: A necessary prerequisite for MLP and training implementation; the specific question is fundamentals coverage.

Answer

• Training becomes unstable if activation or gradient variance rapidly shrinks or grows across layers.
• Xavier-style initialization aims to preserve forward and backward variance and is often associated with tanh or linear assumptions; He-style initialization accounts for ReLU discarding roughly half of the activations.
• Initializing every weight to the same value prevents symmetry breaking among neurons.
• Biases can often start at zero, although architecture and normalization may change that choice.
• Initialization is not an isolated hyperparameter; consider it with activation, normalization, residual scaling, and numerical precision.

Likely Follow-ups: Why can all-zero initialization work for linear regression but not for a two-layer MLP?
Common Failure Modes: Memorizing initialization names without being able to explain variance propagation.

MF16: What Is the Core Difference between BatchNorm and LayerNorm?

Tags: P1, normalization, Transformer
Evidence: Prerequisite coverage for Transformer and MLP fundamentals; no source reports this as an actual Lila question.

Answer

• BatchNorm uses batch statistics, behaves differently during training and evaluation, and maintains running statistics.
• LayerNorm normalizes the feature dimension within each sample. It does not depend on other samples in the batch, and its training and evaluation formulas are usually the same.
• LayerNorm is more natural for small batches, variable-length sequences, and autoregressive settings; BatchNorm is common in CNNs with large batches.
• Normalization affects optimization, scale, and to some extent regularization, but it does not replace sound data preprocessing.
• Be able to identify exactly which dimensions are normalized for a given tensor.

Likely Follow-ups: How do pre-norm and post-norm Transformers differ during training?
Common Failure Modes: Saying only that one is for computer vision and the other for NLP without explaining the statistical dimensions.

MF17: What Is Representation Learning, and Why Is It Important?

Tags: P1, representation
Evidence: A Flagship Machine Learning Scientist candidate was directly asked, “What is representation learning and why is it important?”

Answer

• Representation learning lets a model learn task-useful features from data instead of relying entirely on hand-designed features.
• A good representation preserves information relevant to downstream tasks while being appropriately invariant to irrelevant variation.
• In life science, abundant unlabeled sequences or structures can support pretraining, while scarce experimental labels support transfer learning.
• Evaluation should go beyond attractive visualizations and include linear probes, few-shot learning, transfer, retrieval, out-of-distribution tests, and downstream performance.
• A representation may encode confounders such as batch, species, or experimental source, so use probing and stratified evaluation.

Likely Follow-ups: What is harmful representation collapse?
Common Failure Modes: Equating a high-dimensional embedding with a good representation.

MF18: How Would You Evaluate a Protein Embedding?

Tags: P1, protein, evaluation
Evidence: Lila biological-sequence foundation models and Flagship representation learning; the specific question is a safe reconstruction.

Answer

• First define the intended use: function, localization, structure, binding, generation conditioning, or retrieval.
• Freeze the embedding and use a linear probe to separate representation quality from downstream model capacity.
• Use family- or homology-aware splits so near duplicates do not inflate performance.
• Compare simple baselines such as one-hot encoding, k-mers, known descriptors, and a smaller pretrained model.
• Build data-efficiency curves and evaluate out-of-distribution family performance, calibration, and errors.
• Check whether the embedding primarily reflects length, species, or dataset source.

Likely Follow-ups: How would you evaluate per-residue and sequence-level embeddings separately?
Common Failure Modes: Drawing scientific conclusions from a t-SNE plot alone.

MF19: What Does Self-Attention Compute?

Tags: P1, Transformer
Evidence: Cohere attention implementation, Mistral and xAI MHA, BigHat Transformer, and Zoom Transformer.

Answer

• Each query is compared with the keys, scaled by sqrt(d_k), passed through a mask and softmax to obtain weights, and then used to compute a weighted sum of the values.
• In self-attention, Q, K, and V are different linear projections of the same sequence; in cross-attention, the query and key-value inputs come from different sources.
• Multiple heads let different subspaces learn different relationships, but they do not guarantee that every head has a human-interpretable meaning.
• Standard dense attention has quadratic time and attention-map memory complexity in sequence length.
• The semantics of padding and causal masks must be explicit.

Likely Follow-ups: Why not share Q and K directly? Can attention weights establish causal importance?
Common Failure Modes: Describing attention as merely finding the most similar token while ignoring learned projections and value aggregation.

MF20: Why Does a Transformer Need Positional Information?

Tags: P1, position
Evidence: Latent Labs directly followed up on positional embeddings.

Answer

• Without positional encoding, self-attention is permutation equivariant and cannot distinguish different orders of the same set of tokens.
• Learned absolute or sinusoidal encodings inject position; relative bias and RoPE encode relative relationships more directly.
• The choice affects length extrapolation, implementation cost, and inductive bias.
• Relative distance matters in a protein sequence, but sequence distance is not the same as three-dimensional distance.
• Padding positions are not real residues and still need to be masked.

Likely Follow-ups: Why might sinusoidal encoding support length extrapolation?
Common Failure Modes: Assuming token embeddings automatically contain positional information.

MF21: What Terms Make Up the VAE ELBO?

Tags: P2, VAE
Evidence: Lila confirmed only that a VAE question “could” appear; no verified direct VAE question was found.

Answer

• The encoder approximates the posterior q(z|x), the decoder defines p(x|z), and the prior is often a standard normal distribution.
• The ELBO is the expected reconstruction term minus KL(q(z|x)||p(z)); maximizing it is equivalent to minimizing reconstruction loss plus KL.
• The form of the reconstruction term depends on the observation likelihood and should not automatically be MSE.
• Reparameterization moves randomness into independent noise, allowing parameters to receive low-variance pathwise gradients.
• KL encourages the latent space to match the prior, but excessive pressure can harm reconstruction or cause posterior collapse.

Likely Follow-ups: Why is the ELBO a lower bound on log likelihood?
Common Failure Modes: Describing a VAE as an ordinary autoencoder with random noise added.

MF22: What Is Posterior Collapse?

Tags: P2, VAE failure mode
Evidence: A natural follow-up for minimum VAE coverage, not a publicly observed original question.

Answer

• When a strong decoder can explain the data without using z, q(z|x) may collapse toward the prior, KL approaches zero, and the latent code carries little information about the input.
• Diagnose it with per-dimension KL, a mutual-information proxy, latent ablation, and sensitivity of reconstruction to z.
• Mitigations include KL warm-up or annealing, free bits, restricting the decoder, adjusting beta, or changing the training schedule.
• A nonzero KL is not sufficient evidence of success; verify that the latent representation is useful for generation and downstream tasks.

Likely Follow-ups: What do excessively large and excessively small KL terms imply?
Common Failure Modes: Assuming good reconstruction means the VAE learned a useful latent space.

MF23: What Are the Forward and Reverse Processes in Diffusion?

Tags: P1, diffusion
Evidence: BigHat diffusion implementation discussion, the Latent protein-diffusion take-home, and Lila’s statement that it “could” appear.

Answer

• The forward process gradually adds noise to data according to a predefined schedule and usually allows direct sampling of any xt from x0.
• The reverse process learns to denoise a noisy sample step by step, approximating the generative direction of the data distribution.
• Common objectives train the network to predict noise, x0, or velocity; these parameterizations have different weighting properties.
• Generation requires multiple reverse-sampling steps and is usually more expensive than one autoregressive or decoder forward pass.
• For protein sequences, coordinates, or discrete states, the noise process and equivariance or constraints must match the data type.

Likely Follow-ups: Why can training sample one random t instead of unrolling the entire chain?
Common Failure Modes: Describing forward noising as a learned process.

MF24: What Does the Noise Schedule Affect?

Tags: P2, diffusion
Evidence: A natural follow-up to a minimal diffusion component; not a publicly reported original question.

Answer

• The schedule determines the signal-to-noise ratio, task difficulty, and training-weight distribution across timesteps.
• Destroying signal too quickly can make intermediate states hard to learn; changing it too slowly adds redundant steps and sampling cost.
• The effects of linear, cosine, or learned schedules depend on the data and parameterization.
• Inspect per-timestep loss, sample quality, diversity, constraint satisfaction, and compute together.
• Noise must have sensible semantics for the modality; coordinates and discrete sequences should not mechanically share the same Gaussian process.

Likely Follow-ups: How are alpha, beta, and cumulative alpha-bar related?
Common Failure Modes: Memorizing schedule names without explaining signal-to-noise ratio.

MF25: When Should You Choose an Autoregressive Model, a VAE, or Diffusion?

Tags: P1, generative-model choice
Evidence: Generate protein generative-model case, BigHat Transformer and diffusion, and Latent diffusion; the VAE evidence is weaker.

Answer

• Begin with the data and decision objective: do you need likelihood, fast sampling, global structure, a controllable latent space, inpainting, or constrained generation?
• Autoregressive models provide a natural likelihood, but sampling is sequential and errors may accumulate.
• VAEs sample quickly and provide an explicit latent space, but may suffer posterior collapse and a trade-off involving a blurry or weakly expressive decoder.
• Diffusion is often stable to train and supports conditional generation, but sampling is expensive and the noise process matters.
• Compare against simple retrieval, mutation, ranking, or optimization baselines; a generative model is not the default answer.
• Make the final choice based on validity, novelty, diversity, synthesizability, and experimental success.

Likely Follow-ups: How does the need to generate one million candidates affect the choice?
Common Failure Modes: Choosing a model based on publication recency.

MF26: How Do Epistemic and Aleatoric Uncertainty Differ?

Tags: P1, uncertainty
Evidence: Lila Residency directly involves uncertainty; Latent and BigHat screening plus Lila’s closed loop support transfer.

Answer

• Aleatoric uncertainty comes from randomness or noise in the observations and may not disappear with more data.
• Epistemic uncertainty comes from limited model knowledge of the data space and can often be reduced by collecting data in the relevant region.
• A heteroscedastic likelihood can represent input-dependent aleatoric uncertainty; ensembles and Bayesian approximations can estimate part of epistemic uncertainty.
• Softmax entropy does not automatically separate the two and may still be overconfident out of distribution.
• Active learning is especially interested in uncertainty that a new label could reduce, but it must also account for diversity and experimental cost.

Likely Follow-ups: How would you verify that an uncertainty estimate is useful?
Common Failure Modes: Treating disagreement among models as a perfect estimate of real-world uncertainty.

MF27: What Is the Basic Active-Learning Loop?

Tags: P1, closed-loop discovery
Evidence: Lila’s official closed-loop responsibilities, Lila Residency active-learning material, and BigHat and Latent screening scenarios.

Answer

• Train a model on an initial labeled set and calibrate its uncertainty.
• Compute utility over the candidate pool, incorporating factors such as uncertainty, expected improvement, diversity, cost, and feasibility.
• During batch selection, avoid highly similar candidates and account for experimental capacity and controls.
• After labels arrive, update the data, retrain, and use a fixed evaluation set to determine whether the loop truly improved.
• Define a stopping criterion, an exploration-exploitation balance, and how selection bias will be handled.
• Compare against random, diversity-only, and greedy-score baselines.

Likely Follow-ups: Why can selecting only the most uncertain samples fail?
Common Failure Modes: Ignoring out-of-distribution junk and the probability of experimental failure.

MF28: Why Do Strategies Differ for Scarce, Skewed, and Very Large Data?

Tags: P0, data regime
Evidence: Flagship directly asked about the edge cases of scarce, skewed, and large data.

Answer

• Scarce data: prioritize simple baselines, rigorous validation, transfer or self-supervision, sensible priors, data quality, and active sampling.
• Skewed data: use stratified splits, cost-aware metrics, sampling or weighting, threshold selection, and calibration.
• Large data: consider streaming, distributed sampling, mixed precision, compute-efficient models, representative evaluation, and data curation.
• These regimes can occur together, for example a large dataset with very few positives, so separate the bottlenecks.
• For every method, state whether it addresses variance, bias, compute, label scarcity, or evaluation.

Likely Follow-ups: Why can a model still overfit a large dataset?
Common Failure Modes: Answering all three regimes with “add regularization.”

MF29: What Are the Basic Choices for Multimodal Fusion?

Tags: P1, multimodal
Evidence: Lila’s official responsibilities explicitly include biological sequences, molecular structures, and multimodal experimental data; no specific interview question was observed.

Answer

• Early fusion concatenates or combines representations early, enabling cross-modal relationships but requiring alignment and a strategy for missing inputs.
• Late fusion models modalities independently and combines predictions. It is modular and more robust to missing modalities, but offers fewer fine-grained cross-modal interactions.
• Cross-attention and shared latent spaces provide intermediate fusion options.
• Training data often lacks complete observations for every modality, so distinguish missing at random, systematic missingness, and modality dropout.
• Evaluation should include complete-modality, missing-modality, single-modality baseline, and cross-source out-of-distribution settings.

Likely Follow-ups: How would you prove that the second modality contributes complementary information rather than a batch shortcut?
Common Failure Modes: Concatenating every embedding without checking alignment or missingness.

MF30: What Questions Do Baselines and Ablations Answer?

Tags: P0, research reasoning
Evidence: Deep Genomics design choices, the BigHat case, and Genentech and Flagship research discussions.

Answer

• A baseline asks whether the complex method beats a reasonable simpler choice.
• An ablation asks which component of a complex system causes the improvement.
• Baselines should include a trivial predictor, a classical model, a strong existing method, and a cost-matched alternative.
• An ablation should change one clearly defined factor while keeping the data split, compute, tuning budget, and evaluation fixed.
• Use multiple seeds, confidence intervals, or statistical tests to determine whether a difference is robust.
• Report negative results as well so that only successful configurations are not selectively shown.

Likely Follow-ups: What if ablated components interact?
Common Failure Modes: Calling a model trained for fewer steps a fair ablation.

MF31: What Types of Distribution Shift and OOD Settings Exist?

Tags: P1, robustness
Evidence: Safely derived from scientific cases, cross-experiment and cross-source evaluation, and closed-loop discovery.

Answer

• Covariate shift: the input distribution changes while the conditional relationship may remain approximately fixed.
• Label shift: class priors change.
• Concept shift: the relationship from input to label changes, which is harder.
• Scientific ML shifts also include new protein families, assay protocols, laboratories or sites, species, and time periods.
• Use group, time, and external splits, report performance by shift source, and recheck calibration.
• OOD detection is not a universal safeguard; unknown shifts and changes in label definition are the hardest cases.

Likely Follow-ups: Under what assumptions does reweighting work?
Common Failure Modes: Calling every validation drop an OOD problem.

MF32: How Would You Systematically Debug Training Loss That Does Not Decrease?

Tags: P0, debugging
Evidence: TORC, Wayve, NVIDIA, and PathAI debugging, plus the Tempus coding-style review.

Answer

1. First try to overfit a tiny dataset to verify that the pipeline can learn at all.
2. Check labels, shapes, dtypes, masks, normalization, and batch alignment.
3. Inspect model outputs and the loss range; determine whether gradients are None, NaN, or zero and whether the parameters are actually in the optimizer.
4. Check the zero_grad/backward/step order, learning rate, and frozen parameters.
5. Compare against a constant or linear baseline and, if useful, create synthetic learnable data.
6. Record activation and gradient norms by layer to localize the problem.
7. Only then try a more complex architecture or a large hyperparameter search.

Likely Follow-ups: How would the debugging order change if training loss decreased but validation performance did not improve?
Common Failure Modes: Reacting first by switching to a Transformer or adding compute.

MF33: Which Output Activation Should You Use?

Tags: P0, activation, output semantics
Evidence: A DeepL Research Scientist candidate directly reports a positive-regression output-activation question with targets from 0.5 to 5,000; a Pinterest MLE candidate was directly asked about ReLU and sigmoid.

Answer

• Start from the mathematical support of the target and the loss, not the model name.
• Single-label multiclass models usually emit logits for cross entropy. Binary and multilabel models also keep logits for a logits-based BCE, applying sigmoid only for inference probabilities.
• Unbounded regression usually uses a linear output. For a strictly positive target, compare Softplus, a log-transformed target, or another physically meaningful parameterization.
• ReLU enforces nonnegativity, but its negative half has zero gradient and it can output exact zeros. It is not the default answer for every positive regression problem.
• If the target has a real bound [a,b], a + (b-a) * sigmoid(z) can encode it. Do not hard-code bounds that merely describe the observed sample.
• For a target spanning 0.5 to 5,000, also compare target scaling or log-space modeling and report error on the original scale.

Likely Follow-ups: Why are logits usually passed to the training loss instead of probabilities? What does Softplus do for large positive inputs?
Common Failure Modes: Treating “currently positive” as a known physical upper bound; ignoring the target scale.

Fundamentals Coverage Audit

• P0 foundations: 14 questions among MF01–MF16, plus MF28, MF30, MF32, and MF33.
• Representation and Transformer: MF17–MF20.
• VAE: MF21–MF22, P2 only.
• Diffusion: MF23–MF24, plus model selection in MF25.
• Scientific ML: MF26–MF31.
• Actual question count: 33.

CS01: One Million Protein Designs, a Budget of One Thousand Experiments

Tags: P0, protein design, ranking, uncertainty, 30–45 minutes
Evidence: A Latent Labs candidate directly reports being asked “how would you generate one million protein designs” together with a protein diffusion take-home. The budget and the full set of constraints are a practice reconstruction.

Prompt

Generate roughly one million protein candidates for a target function, but only 1,000 of them can be tested experimentally in the first round. Describe the end to end plan.

Structured Answer

1. Define the decision first: the objective is not to maximize the model score but to maximize the number of real hits, the information gain, or a combination of the two within the 1,000 experimental slots. Confirm the function, structure, expression, stability, safety, and synthesizability constraints up front.
2. Define the candidate space: decide whether this is de novo design, local redesign around a fixed scaffold, or mutation of an existing family. The search space and the acceptable level of novelty are completely different in each case.
3. Audit the data: what assay produced the training labels, whether they span batches or labs, how the negatives were generated, and whether there is censoring, replication, or failed experiments. Fold sequence family, assay batch, and time into the split.
4. Baselines: nearest neighbor retrieval against known sequences; mutation after rule based filtering; a simple property predictor plus diversity; random or diversity only selection.
5. Generation: compare autoregressive models, diffusion, and optimization of existing sequences; condition on the target function and the hard constraints. Define validity before generating rather than discovering after the fact that a candidate cannot be tested.
6. Cheap filtering: deduplicate; remove illegal residues and lengths; check motifs and manufacturability; drop obvious structural conflicts; drop candidates that are too close to the training set or carry unnecessary risk. Record why each filter eliminated each candidate.
7. Ranking: use a multitask model to estimate activity, stability, expression, and similar properties; use calibrated predictions along with uncertainty, and do not treat a single score as ground truth.
8. Diversity: cluster at the level of embeddings, sequence families, or structures; assign quotas by cluster so that the 1,000 selections are not near duplicates.
9. Selection: combine exploitation, uncertainty driven exploration, diversity, and experimental cost into an interpretable acquisition function. Reserve slots for random, control, and reference sequences.
10. Evaluation: offline, look at family held out ranking, calibration, top-k enrichment, and diversity; experimentally, look at hit rate, effect size, failure categories, and coverage.
11. Next round: record every usable result, including failures and assay QC; assess whether selection bias has distorted the model; compare the gain from the active loop against the random and diversity only baselines.

Follow-up Tree

One million candidates
      ├─ How are the candidates generated?
      │  ├─ Why not simple mutation?
      │  └─ How do you guarantee validity and novelty?
      ├─ How do you narrow down to 1,000?
      │  ├─ hard filters
      │  ├─ prediction and calibration
      │  └─ uncertainty and diversity
      ├─ How do you split the training set and the test set?
      └─ What if the entire first round fails?
         ├─ assay QC
         ├─ distribution shift
         └─ model and constraint diagnosis

Rubric (20 points): goal and constraints 3; data and split 3; baselines 2; generation and filtering 3; ranking, uncertainty, and diversity 4; experimental design 3; failure analysis 2.
Red Flags: opening with nothing but diffusion; taking the top 1,000 by predicted score; no controls; conflating novelty with effectiveness.

CS02: Building a Generative Model for a Target Protein

Tags: P0, problem formulation, generative modeling, 30–45 minutes
Evidence: A Generate:Biomedicines candidate reports being asked “how would you design a generative model for a target protein”; BigHat provides corroboration on Transformers and diffusion.

Prompt

We want to design new sequences for a target protein function. How would you define and train the generative model?

Structured Answer

1. Ask first what is being designed: a binder, an enzyme, stability, expression, or several objectives at once; and whether the input is a target sequence, a structure, a pocket, or an assay context.
2. Separate the hard constraints from the soft objectives. Length, fixed residues, symmetry, and chemistry can be hard constraints; affinity, stability, and similar properties are usually soft objectives carrying uncertainty.
3. Make the training unit and the data provenance explicit so that the same family or highly similar complexes do not straddle the split.
4. Establish non generative baselines: retrieval, motif grafting, local mutation, and a discriminative predictor plus search. A complex generative model has to beat these.
5. Choose the model based on the data modality and the decision: autoregressive models suit sequence likelihood; diffusion supports global and conditional denoising; a VAE offers latent search. The model is not chosen because it is “advanced.”
6. The training objective has to match the data. A sequence token loss is not equivalent to optimizing function, so consider adding a conditional objective, ranking, or property guidance, while staying alert to reward hacking.
7. Decompose generative evaluation into validity, novelty, diversity, constraint satisfaction, predicted properties, calibration, and ultimately experimental success.
8. Run ablations: remove the target conditioning, remove the structural information, swap in a simpler encoder, and vary the sampling temperature or guidance.
9. Design the wet-lab batch to include positives, known negatives, random generations, and diversity strata.
10. Build a taxonomy of failed candidates: no expression, misfolding, no function, assay failure. Use it to update the data and the objective.

Follow-up Tree: what if there is little data → pretraining, transfer, or simple search; what if there is no structure → condition on sequence, or predict the structure but propagate the uncertainty explicitly; the generations are valid but not novel → sampling, training distribution, evaluation; novel but nonfunctional → reward model shift, missing constraints.
Rubric (20 points): problem definition 4; data and leakage 3; baselines 2; rationale for the model choice 3; objective 2; evaluation 3; experiments and iteration 3.
Red Flags: using perplexity as the sole criterion of success; no discriminative or search baseline; treating a predicted structure as error free ground truth.

CS03: Antibody Design and Complex Assay Data

Tags: P0, antibody, multitask, noisy labels, 30–45 minutes
Evidence: A BigHat Biosciences candidate reports therapeutic design, complex biological assay data, and a discussion of diffusion and Transformer implementation.

Prompt

You are given data from several rounds of antibody design across multiple assays. The assays differ substantially in noise, missingness, and scale. How would you build the model and select the next round of candidates?

Structured Answer

1. Build an assay schema: what each column measures, its units, detection limits, replicates, batch, protocol version, and the reason for missingness.
2. Distinguish biological negatives, technical failures, and untested candidates; you cannot treat all missing values as negatives.
3. Stratify or group split by sequence lineage, campaign, target, and time so that the split simulates real generalization to the next round.
4. Start the baselines with per assay linear or tree models, simple sequence descriptors, and nearest neighbors.
5. A multitask model can share a representation, but each assay needs a matching head and loss; model count, binary, and continuous or censored labels separately.
6. Loss weighting should not follow sample counts alone; consider measurement noise, task importance, and gradient scale. A single task baseline is mandatory.
7. Use replicates to estimate the noise ceiling so that you are not asking the model to exceed label reproducibility.
8. When selecting the next round, weigh target activity, developability, uncertainty, diversity, manufacturability, and experimental cost together.
9. Evaluation is not just the average metric; also look at the high value operating region, top-k enrichment, calibration, and subgroups by assay and by family.
10. After closing the loop, check for selection bias: the new data was chosen by the old model and is not i.i.d. Retain exploration samples and random controls.

Follow-up Tree: only 5% of the sequences were measured in one assay → missingness and selection bias; the assays conflict with each other → Pareto front and business constraints; the model only learns batch → group split and adversarial probes; how to use uncertainty → acquisition and calibration.
Rubric (20 points): assay semantics 4; missingness and noise 3; split 2; baselines and multitask design 3; loss and evaluation 3; selection 3; bias and controls 2.
Red Flags: filling missing values with zero; using MSE for every task; reporting only the average RMSE; ignoring experiment provenance.

CS04: Closed-Loop Active Learning

Tags: P0, active learning, experimental loop, 30–45 minutes
Evidence: The official Lila job description explicitly names closed-loop discovery, and Lila Residency interview reports involve active learning and uncertainty. What follows is a transfer exercise for the current role, not a publicly reported question.

Prompt

You can run 200 candidates per round, with a total budget of five rounds. Design a closed loop that can determine whether active learning is genuinely working.

Structured Answer

1. Make the utility explicit: hit rate, best found value, Pareto coverage, knowledge gain, or the number of experiments needed to reach the goal.
2. Fix an evaluation or benchmark set that the acquisition function cannot influence; otherwise a model that improves on self selected data does not mean generalization improved.
3. The initial data must cover the candidate space, combining historical data, designed diversity, and controls.
4. Record the model version and data lineage for every round of training, and run calibration and OOD checks.
5. The acquisition function must be compared against, at minimum, random, uncertainty only, score only, diversity only, and combined methods.
6. Batch selection must remove redundancy and account for assay cost, feasibility, plate layout, and controls.
7. Preregister stopping and kill criteria, for example no improvement over random for two consecutive rounds, or uncertainty that is no longer calibrated.
8. Feed all experimental results back in an auditable form, including failures and QC status; keep technical failures separate from negative labels.
9. Analyze cumulative regret, best so far, hit rate, coverage, calibration, and cost per hit.
10. Evaluate the acquisition function with multiple seeds and retrospective replay in simulation, while being explicit about the limits of a historical counterfactual.

Follow-up Tree: the uncertainty is unreliable → ensembles, calibration, random exploration; the batch results take a month → asynchronous acquisition; the experimental failure rate is high → a success model and cost aware utility; the model only picks one family → diversity constraints.
Rubric (20 points): utility 3; unbiased evaluation 4; baselines 3; acquisition 3; experiments and QC 3; stopping 2; auditability 2.
Red Flags: no random baseline; watching only the training loss; treating assay failures as negatives; changing the metric arbitrarily each round.

CS05: Supervised Prediction on DNA or Protein Sequences

Tags: P0, sequence modeling, baseline, 30–45 minutes
Evidence: A Deep Genomics candidate reports modeling DNA sequence data and explaining the model, the loss, and the code; D. E. Shaw Research asks protein folding ML questions.

Prompt

Given sequences and functional labels, build a predictor that generalizes to new families. How would you approach it?

Structured Answer

1. Establish whether the label is sequence level, residue level, or pairwise; continuous, binary, count, or censored; and what the measurement error looks like.
2. Audit length, alphabet, duplicates, near duplicates, family, species, batch, and label prevalence.
3. Use a family or homology group split as the primary evaluation, and keep a random split as an “easy upper bound” diagnostic rather than as the final proof.
4. Baselines: class prevalence or mean, k-mer and handcrafted descriptors, nearest neighbor, linear or logistic regression, and a small MLP or CNN.
5. For a complex model, start with pretrained embeddings plus a linear probe before fine tuning, and compare data efficiency against compute.
6. Match the loss to the label and get the padding mask right; handle class imbalance with weights or sampling while keeping the true prevalence in the evaluation.
7. Metrics include the primary task metric, calibration, family stratified performance, confidence intervals, and concrete error examples.
8. Probe the confounders: length, species, batch, and nearest train similarity. If these simple variables explain most of the performance, revisit the task itself.
9. Use external validation or a prospective experiment to assess real generalization.

Follow-up Tree: the sequences are very long → truncation, a local and global architecture, compute; only 1,000 labels → frozen embeddings, simple models; performance collapses on new families → the split and the representation; explaining individual residues → saliency requires experimental or perturbation validation.
Rubric (20 points): label and data 3; family split 4; baselines 3; model and loss 3; metrics 3; confounders 2; prospective validation 2.
Red Flags: a random row split; fine tuning a huge model straight away; reporting only AUROC; not recording the padding mask.

CS06: Single-Cell RNA Count Data

Tags: P0, omics, statistics, 30–45 minutes
Evidence: An insitro candidate reports an RNA-seq scenario with questions on distributions and statistical testing; Cellarity directly asks about single-cell omics basics.

Prompt

Given a cell by gene count matrix spanning multiple donors, experimental batches, and treatments, predict treatment response and identify stable biomarkers.

Structured Answer

1. The decision unit is the donor or the sample, not the individual cell; treating large numbers of correlated cells as independent samples creates pseudoreplication.
2. QC covers low quality cells, doublets, library size, mitochondrial fraction, and gene filtering, but the thresholds must be set within the training data and recorded.
3. For count data, consider a negative binomial model with overdispersion or a defensible transformation; keep the raw counts available for count models.
4. The split must be by donor, and by site or time where extrapolation demands it; cells from the same donor cannot straddle train and test.
5. Start the baselines with donor level pseudobulk, cell type proportions, and a regularized linear model.
6. The model can represent cells hierarchically and then aggregate to the donor, or build features per cell type. Any complex single-cell encoder has to be compared fairly against the pseudobulk baseline.
7. Batch correction may only be fit on training information, and you must check whether it has removed the biological treatment signal.
8. Evaluation includes donor level prediction, calibration, performance across batches and sites, biomarker effect sizes, FDR, and stability.
9. A biomarker needs an independent cohort, a perturbation, or mechanistic evidence; it cannot be claimed on the basis of attention or saliency alone.

Follow-up Tree: the cell counts differ → weighted aggregation or subsampling; batch and treatment are completely confounded → the data cannot identify the effect and new experiments are required; there are many zeros → the observation model; the biomarker does not replicate → selection bias and multiple testing.
Rubric (20 points): unit of analysis 4; QC and distribution 3; donor split 3; baselines 2; model 2; batch and confounding 3; biomarker validation 3.
Red Flags: splitting cells at random; treating the number of cells as the sample size; batch correcting on the full data first; skipping multiple testing.

CS07: A Multimodal Model over Sequence, Structure, and Assay Data

Tags: P1, multimodal, missing modality, 30–45 minutes
Evidence: The official Lila job description explicitly names biological sequences, molecular structures, and multimodal experimental data. No candidate has publicly reported this exact case.

Prompt

The training data may contain sequences, predicted or experimental structures, assay values, and metadata, but not every record has every modality. How would you model it?

Structured Answer

1. Start from the prediction decision and determine which modalities are available at test time. Information visible during training but unavailable at deployment cannot be used directly as an input.
2. Build a modality availability table and analyze whether the missingness correlates with experimental success, target, or time; missingness itself can leak the label.
3. Give every modality a single modality baseline first: a sequence encoder, a structure descriptor or GNN, and a tabular model over assay values and metadata.
4. Compare late fusion, early fusion, and cross-attention or a shared latent space. With little data and heavy missingness, late fusion is usually easier to validate; only with ample fine grained alignment can you demonstrate the value of complex interaction.
5. For missing modalities, use modality dropout, mask tokens, mixture of experts, or partial encoders; do not fill zeros without marking them.
6. A predicted structure carries model error and cannot be pooled with an experimental structure as the same ground truth. The input should record the source and the confidence.
7. Split by entity, family, and time, and prevent different modalities from the same experiment being scattered across different splits.
8. Ablations cover each single modality, the fusion method, the missingness scenarios, and matched compute.
9. Report metrics separately for the complete modality case, each missingness pattern, family and OOD subsets, calibration, and experimental utility.

Follow-up Tree: structures exist only for the positives → missingness leakage; only sequences are available at test time → a teacher and student setup or a sequence only deployment; the modalities have very different dimensions → projection and normalization; fusion gives no gain → redundancy, data scale, alignment.
Rubric (20 points): deployment availability 3; missingness 4; single modality baseline 3; fusion choice 3; split 2; ablation 3; uncertainty 2.
Red Flags: concatenating everything directly; treating a predicted structure as ground truth; no evaluation with missing modalities.

CS08: Facing Scarce, Skewed, and Large All at Once

Tags: P0, data regimes, 30–40 minutes
Evidence: Flagship Pioneering directly asks about training edge cases such as very scarce, skewed, and large data.

Prompt

You have fifty million unlabeled sequences but only 3,000 experimental labels, of which 2% are positive. How would you design the project?

Structured Answer

1. Separate the three facts: the unlabeled set is very large, the labeled set is very small, and the positives are very few. Each calls for a different strategy.
2. Audit the quality, replicates, selection mechanism, and family coverage of the 3,000 labels first; label quality matters more than model size.
3. Split by family, time, or group, make sure the validation and test sets contain enough positives, and report bootstrap confidence intervals.
4. Baselines: prevalence, nearest neighbor, handcrafted or k-mer features with logistic regression or trees, and a frozen public or self-supervised embedding with a linear head.
5. The unlabeled data can support self-supervised pretraining, but compare its data and compute efficiency against a smaller model or an existing pretrained one.
6. Training may use class weights, balanced batches, or a ranking objective; the evaluation keeps the original prevalence and focuses on PR-AUC, recall at a given precision, calibration, and top-k enrichment.
7. Use learning curves to determine whether the bottleneck is labels, representation, or noise.
8. When designing active labeling, combine uncertainty, diversity, positive enrichment, and experimental cost, and keep a random baseline.
9. Scale up the big data engineering only after proving it is needed: streaming, deduplication, distributed training, mixed precision. Infrastructure cannot substitute for scientific design.

Follow-up Tree: no positives land in the test set → a stratified group split or a larger test set; pretraining gives no gain → objective or domain mismatch; the probabilities are off after class weighting → calibration; what to pick for 200 additional labels → acquisition.
Rubric (20 points): separating the regimes 3; label audit 3; split and metrics 4; baselines 3; pretraining rationale 2; active sampling 3; compute restraint 2.
Red Flags: training the largest available Transformer immediately; using accuracy; claiming real world performance from a balanced validation set.

CS09: Predicting the Likelihood of Vaccination

Tags: P1, classification, interpretability, ethics, 30–40 minutes
Evidence: A Merck Machine Learning Engineer Intern candidate directly reports a case on modeling demographics and vaccine usage and explaining the model.

Prompt

Given demographics and historical vaccine usage, predict who is likely to be vaccinated in the next three months, and explain the model.

Structured Answer

1. Clarify the use: resource planning, reminder outreach, or risk research. The use determines the cost of false positives and false negatives and whether individual level prediction is appropriate at all.
2. Define the index date, the prediction window, and the available features; future information, post vaccination records, and healthcare contact proxies can all leak.
3. Split by time and by person group so that one person’s history does not straddle the split.
4. Baselines: overall and stratified prevalence, logistic regression, and a regularized tree model.
5. Handle the imbalance, and evaluate PR-AUC, recall and precision at the operating point, calibration, and subgroup performance.
6. Interpretability must distinguish global association, local explanation, and causal effect. Feature importance does not prove that intervening on that feature would change vaccination.
7. Examine the fairness risks arising from missing data, access to care, geography, and demographic proxies.
8. If it is used for outreach, the better evaluation is an actual intervention trial rather than retrospective accuracy alone.

Follow-up Tree: the model’s most important feature is prior visits → an access proxy and leakage; calibration is poor for some groups → subgroup calibration and data; how to explain it → coefficients and SHAP, but not causally; how to deploy it → monitoring, consent, and an intervention test.
Rubric (20 points): use case 3; temporal target and leakage 4; baselines 2; metrics and calibration 3; interpretability 3; fairness 3; prospective test 2.
Red Flags: a random row split; boosting accuracy with sensitive attributes and no discussion of it; treating an explanation tool as causal analysis.

CS10: Strong Offline Metrics, Total Wet-Lab Failure

Tags: P0, failure analysis, experiment, 30–45 minutes
Evidence: A composite safe reconstruction from the BigHat assay record, the Latent design screening record, and the Lila closed-loop and research defense records; it is not a publicly reported question.

Prompt

The model clearly beats the baseline on held-out data, yet not one of the 100 top candidates in the first round succeeded experimentally. How would you investigate?

Structured Answer

1. Do not assume the model is at fault first. Verify assay QC, positive controls, plate layout, sample identity, expression and synthesis success, and protocol drift.
2. Audit the evaluation split for near duplicates, family leakage, batch shortcuts, selection bias, and preprocessing leakage.
3. Compare the 100 candidates against the train and validation distributions: similarity, uncertainty, length, motifs, structure, and manufacturability.
4. Check for objective misalignment: whether the offline label is only a proxy, and whether the score the model optimized agrees with actual experimental success.
5. Check top-k calibration and ranking, not just the AUROC or RMSE averaged over the whole distribution.
6. Check the filtering pipeline, units, normalization, feature order, checkpoint, train and eval mode, and the inference code.
7. Rerun the full inference to assay path with blinded known positives and negatives.
8. Build a taxonomy of the failures: not synthesized, not expressed, misfolded, no binding, assay failure. Different failures imply different fixes to the model or the process.
9. For the next round, add more conservative near distribution candidates plus random and diverse controls, and narrow each hypothesis down to a discriminating experiment.

Follow-up Tree: the controls also failed → the assay or the process; the controls succeeded but the designs failed → shift or objective; only one family failed → subgroups; a code mismatch → a reproducible inference test; how to select the next round → hypothesis driven strata.
Rubric (20 points): assay QC 3; leakage and split 3; distribution and objective 4; code pipeline 3; failure taxonomy 3; next experiment 4.
Red Flags: swapping in a different model immediately; retraining with all 100 failures labeled as ordinary negatives; no positive controls.

CS11: Transformer, VAE, Diffusion, or Simple Search?

Tags: P1/P2, model choice, 30–40 minutes
Evidence: Lila states that all three model classes “could” appear; BigHat mentions Transformers and diffusion; Latent uses diffusion; Generate provides the generative model case. The direct evidence for the VAE is the weakest.

Prompt

For a sequence design project, compare an autoregressive Transformer, a VAE, diffusion, and a discriminative predictor plus search. Make an initial choice and state what evidence would change it.

Structured Answer

1. List the constraints: data volume, conditioning information, whether globally coordinated mutations are needed, the number and speed of samples required, hard constraints, likelihood, latent optimization, and compute.
2. A predictor plus search is a mandatory baseline, especially when a good scaffold already exists, local mutations are permitted, or labels are scarce.
3. An autoregressive Transformer is a natural fit for sequence likelihood and conditional generation, but sampling is sequential and global constraints may need an additional mechanism.
4. A VAE offers a continuous latent space and fast sampling, but latent utility, reconstruction, and posterior collapse all need verification. Because the direct interview evidence in this study is weak, treat it as an extension rather than the default answer.
5. Diffusion supports global iterative refinement and conditioning, but the sampling cost and the noising process appropriate to the data type must both be justified.
6. Set up a matched data and matched compute comparison, and report validity, novelty, diversity, constraint satisfaction, top-k predicted and experimental success, and wall-clock time.
7. The initial choice should be a falsifiable hypothesis, for example “the local search baseline goes first; if it cannot satisfy the global multi-site constraints, then compare conditional autoregressive generation against diffusion.”
8. Evidence that would change the choice includes data efficiency curves, sampling bottlenecks, constraint violations, wet-lab enrichment, and the failure taxonomy.

Follow-up Tree: one million samples are needed → sampling speed; labels are scarce → pretraining or search; a fixed motif is required → conditional masking and inpainting; evaluating a generative model → not likelihood alone; how to make it fair → a matched budget.
Rubric (20 points): requirements 4; simple baseline 3; model trade-offs 6; evaluation 4; falsifiable decision 3.
Red Flags: choosing a model by popularity; claiming diffusion “always gives the highest quality”; no sampling cost and no wet-lab metric.

CS12: Materials Property Prediction and Candidate Discovery

Tags: P2, materials AI, transfer case, 30–45 minutes
Evidence: A Schrödinger Materials Science Applications Scientist candidate reports computational materials science and ML questions; Lila’s business spans both life sciences and materials science. This is not the main line for the current life sciences role.

Prompt

Given composition, structure, simulation, and experimental property data, build a model that finds new materials satisfying several performance targets.

Structured Answer

1. Pin down the properties, the measurement conditions, the units, the phase, and the synthesizable range of the candidates. In materials, “the same composition” does not necessarily mean the same structure or the same process.
2. Unify the provenance and distinguish simulation from experiment without mixing them unlabeled; record fidelity, temperature and pressure, and measurement uncertainty.
3. Split by chemical system, composition family, or time, so that closely related structures do not straddle the split.
4. Baselines: composition descriptors with linear or tree models; nearest neighbor; a physics or simulation baseline; a simple uncertainty model.
5. A structure model can use graphs or equivariant representations, but it must demonstrate the gain that structural information adds over the composition baseline.
6. Handle multiple objectives with a Pareto front or an explicit utility, incorporating stability, cost, toxicity, and synthesizability.
7. Use uncertainty and OOD signals to select new experiments, but calibrate them and compare against random and diversity baselines.
8. The prospective batch should include known references, exploratory regions, and synthesizable controls; a failed synthesis is also an important label, but it needs to be categorized.

Follow-up Tree: simulation bias → multifidelity modeling; the structure is unknown → a composition baseline and structure prediction uncertainty; very few experimental labels → transfer and physics priors; conflicting objectives → Pareto fronts and decision weights.
Rubric (20 points): scientific semantics 4; provenance and fidelity 3; split 3; baselines 3; model 2; multiobjective handling 2; experiment 3.
Red Flags: mixing simulation and experiment as the same label; a random split over crystals; ignoring synthesis.

CS13: Duplicate, Conflicting, and Dirty Experimental Records

Tags: P0, data quality, deduplication, 30–40 minutes
Evidence: An AstraZeneca Principal Associate AI Scientist candidate reports a data deduplication take-home based on a real problem the team faced; the assay conflicts added here are a scientific data practice reconstruction.

Prompt

After several legacy systems were merged, the same molecule or protein may appear under different names, with repeated experiments and mutually conflicting labels. How would you construct the training table?

Structured Answer

1. Define entity identity separately from observation identity. The same molecule or protein is not the same assay run, and repeated experiments cannot simply be deleted.
2. Build canonicalization rules while retaining the raw identifier, the source system, the conversion version, and the provenance.
3. Handle unambiguous duplicates with deterministic exact rules, then generate candidate pairs with similarity or record linkage; estimate precision and recall by manually auditing a sample.
4. For conflicting labels, first check units, assay protocol, conditions, time, batch, detection limits, and transcription errors.
5. Biological and technical replicates can be aggregated into a mean and variance or handled by an explicit hierarchical model; you cannot simply keep “the most recent” without justification.
6. Split after deduplication and grouping, guaranteeing that the same entity and the same replicate group do not straddle the split.
7. The data pipeline needs versioning, idempotence, and invariant tests, for example on entity count, unit ranges, duplicate rate, and conflict rate.
8. Run a sensitivity analysis: do different deduplication and aggregation rules change the model’s conclusions?
9. For conflicts that cannot be resolved, retain the uncertainty or record the reason for exclusion; do not fabricate a single ground truth.

Follow-up Tree: there is no unified primary key → a canonical graph and record linkage; fuzzy matching merges records incorrectly → thresholds and audits; should duplicates be weighted → the sampling process; most conflicts come from batch → a batch aware model; how to test it → golden pairs and invariants.
Rubric (20 points): identity semantics 4; provenance 3; deduplication method 3; conflict handling 3; split 3; pipeline tests 2; sensitivity 2.
Red Flags: stopping at drop_duplicates(); deciding entity identity from a string name; deduplicating only after the split; averaging conflicting labels arbitrarily.

General Answer Template for Case Studies

1. Clarify: the scientific goal, who makes what decision, the cost of success and failure
      2. Data: the unit of observation, the label generating process, provenance, missingness, noise
      3. Split: group, family, time, site, and the explicit leakage risks
      4. Baseline: trivial, classical, nearest neighbor, existing methods
      5. Model: why it is needed, its inputs and outputs, loss, constraints, compute
      6. Evaluation: primary metric, calibration, subgroups, uncertainty, confidence intervals
      7. Experiment: controls, batch selection, prospective validation
      8. Failure: which observations distinguish data, model, objective, code, and assay problems
      9. Loop: how new data flows back, and how you judge the method against random and baseline

Case Study Coverage Audit

• protein and antibody: CS01–CS03.
• closed-loop discovery: CS04.
• sequence and omics: CS05–CS06.
• multimodal: CS07.
• data regime: CS08.
• clinical and pharma classification: CS09.
• failure analysis: CS10.
• model selection: CS11.
• materials transfer: CS12.
• data engineering and deduplication: CS13.
• actual number of cases: 13.

RD01: The Technical Project You Are Proudest Of

Evidence: A direct Genentech Machine Learning Scientist question.
Prompt: Take five minutes to present the project you are proudest of, then take twenty minutes of follow-up questions.
Strong Answer Skeleton: the scientific and ML problem in one sentence; why it matters; the data and the constraints; the part you owned; the baseline; the key method; quantitative results with their uncertainty; the failures or limitations; the next step.
Likely Follow-ups: What makes that your contribution? What if the most complex component were removed? What would ten times more data change? Which result is least likely to replicate?
Scoring Rubric (10 points): problem 2; personal ownership 2; evidence 2; trade-offs 2; limitations 2.
Red Flags: long background with no decisions; saying only “we”; reporting only the best score.

RD02: From a Paper Claim Back to the Evidence

Evidence: The Genentech research back-and-forth, and Cellarity technical questions about research.
Prompt: Choose the most important claim in one of your papers and explain which experiments genuinely support it and which are only related to it.
Strong Answer Skeleton: define the claim precisely; the comparison that matches it; controls; ablations; statistical uncertainty; alternative explanations; external or prospective validation.
Likely Follow-ups: What if the main ablation shows no significant difference? Was the comparison matched on compute?
Scoring Rubric: precision of the claim 2; chain of evidence 3; controls 2; alternative explanations 2; honesty about the boundaries 1.

RD03: Code Ownership

Evidence: The Tempus take-home coding style review, the Deep Genomics code and design discussion, and the Genentech code samples.
Prompt: Pick a piece of your own training code, draw the data flow and the tensor shapes, and point out the three most dangerous bugs.
Strong Answer Skeleton: entry points and outputs; shape, dtype, and device at every boundary; configuration; randomness; checkpointing; metric aggregation; tests; failure recovery.
Likely Follow-ups: How would you prove that training and inference agree? Which part needs an integration test?
Scoring Rubric: data flow 2; shapes 2; risks 2; tests 2; design trade-offs 2.

RD04: Why Not the Simple Baseline?

Evidence: Deep Genomics follow-ups on design choices, and the BigHat and Flagship cases.
Prompt: Explain the strongest baseline you had before adopting a complex neural model, and the verifiable gain that the added complexity bought.
Strong Answer Skeleton: baseline selection; matched split, compute, and tuning; the hypothesis; ablation; learning curve; wall-clock; statistical uncertainty; the conditions under which it fails.
Likely Follow-ups: Was the baseline tuned fairly? Does the gain come from more data or more parameters?
Scoring Rubric: baseline 3; fairness 2; hypothesis 2; evidence 2; cost 1.

RD05: Describe a Failed Experiment

Evidence: A conservative reconstruction from the research defense and model debugging records; not a single observed question.
Prompt: Describe a failure, and how you separated data, objective, optimization, code, and evaluation as possible causes.
Strong Answer Skeleton: what you expected; what you observed; the candidate hypotheses; the smallest experiment that discriminates between them; the result; the fix; what remains uncertain; the reusable check you took away.
Likely Follow-ups: If you started over, what would you do on day one? Why did you not catch it earlier?
Scoring Rubric: defining the failure 2; hypotheses 2; discriminating tests 3; learning 2; ownership 1.

RD06: Transfer Between Your Research and the Role

Evidence: Flagship asks about research relevance directly, and Genentech asks about the technical experience named in the job description.
Prompt: Which parts of your work transfer to biological foundation models, and which do not?
Strong Answer Skeleton: the transferable formulation, code, and evaluation skills; the domain gap; the risk of a false analogy; a learning and collaboration plan; one low cost first experiment.
Likely Follow-ups: Why are you a fit without protein experience? Which assumption is most likely not to hold?
Scoring Rubric: transfer 3; honesty about the gap 2; a concrete bridge 3; first experiment 2.

RD07: Explaining a Model to an Experimental Scientist

Evidence: The Schrödinger customer needs discussion, and the cross disciplinary scientific team settings at Cellarity and Genentech.
Prompt: Explain the same model output and its uncertainty in two minutes to an ML scientist, then in two minutes to an experimental scientist.
Strong Answer Skeleton: for the ML audience, cover the training distribution, the metric, calibration, and shift; for the experimental audience, cover what decision the score does and does not support, and why this batch of candidates was chosen.
Likely Follow-ups: What do you do when the other side asks for one definite answer?
Scoring Rubric: accuracy 3; audience adaptation 2; uncertainty 2; actionability 2; boundaries 1.

RD08: Reproducibility and Auditability

Evidence: Tempus code style, AstraZeneca data engineering, and the shared requirements of closed loop discovery.
Prompt: Another researcher has to reproduce your central figure next week. What has to be frozen?
Strong Answer Skeleton: a raw data snapshot with provenance; split ids; the preprocessing version; environment and seed; config; checkpoint; evaluation script; the exact command; the expected artifacts; data access restrictions.
Likely Follow-ups: How do you handle GPU nondeterminism? What if the data cannot be shared?
Scoring Rubric: data 2; code and environment 2; config and seed 2; evaluation 2; limitations 2.

RD09: Design a Six Month Research Agenda

Evidence: The Flagship discussion built around a startup problem, and Lila’s official model design and closed loop responsibilities; the specific prompt has not been observed.
Prompt: Design a six month agenda around protein function prediction with few labels.
Strong Answer Skeleton: one falsifiable central hypothesis; a first two week baseline; a data audit; milestones; kill criteria; the highest risk experiment; compute and experimental dependencies; fallbacks; the final decision.
Likely Follow-ups: What if the budget is halved? What if the first experiment fails? How do you keep the agenda from being nothing but a list of models?
Scoring Rubric: hypothesis 2; prioritization 3; kill criteria 2; realism about resources 2; fallback 1.

RD10: The Ten Minute Compressed Version of a Research Talk

Evidence: The Genentech one hour presentation, and the Flagship seminar.
Prompt: Compress a full research talk into ten minutes while keeping one central story.
Strong Answer Skeleton: 1 minute problem and impact; 1 minute gap; 2 minutes data and setup; 2 minutes method; 2 minutes key results and controls; 1 minute failures and limitations; 1 minute relevance and next step.
Likely Follow-ups: If half the slides go, what stays? What is the one sentence an audience outside your field has to remember?
Scoring Rubric: story 2; technical clarity 2; evidence 2; time discipline 2; limitations and relevance 2.

What Can Be Stated Plainly

• These capabilities and question types come from verified candidate reports for closely adjacent roles, from Lila’s first party recruiting communication, or from the official scope of the role.
• MLPs, K-means, KNN, attention, training and debugging, metrics, regularization, statistics, scientific cases, and research talks have genuinely appeared at multiple companies.
• Advanced generative models do appear at adjacent protein roles, but the reasonable granularity for a half hour live coding session is a small component.

What Cannot Be Said

• “This is an actual Lila question” or “Lila will definitely test this.”
• Claiming that Transformers, VAEs, and diffusion are equally likely just because a recruiter said each of them “could” come up.
• Treating the frequencies in this question bank as the true probabilities of a company’s question types. The research sample comes from public self selected reports and carries platform and survivorship bias.
• Transferring computer vision question types directly into Lila’s main line. Lila has already said explicitly that computer vision is not tested.

Code Verification Status

• The 34 Python code blocks in this file were run locally, block by block and independently, and all pass in the current environment.
• The twelve PyTorch stable API pages listed at the top of this file were checked directly.
• Code using other APIs such as nn.Linear, nn.Embedding, one_hot, index_add_, logsumexp, masked_fill, and randn_like was actually run, but the official documentation page was not opened separately for each of these APIs. After a dependency upgrade, both the statements and the behavior should still be rechecked against the actual PyTorch version.
• The example tests are minimal sanity checks, not a full property based test suite or one that spans devices and dtypes.

Quantity Acceptance

This question bank and the evidence base are presented separately: the evidence base records what candidates actually encountered, and the question bank records what was reconstructed in order to train those capabilities. The two must not be blurred together, visually or in wording.