Interview experience | ML Systems | 🔥🔥🔥 | Meta
There were four rounds: two coding sessions, one ML system design, and one behavioral. The first coding round started a little slowly; the cache round went much better. Overall it was more systems-heavy than expected. The questions sounded reasonable at first, then more constraints kept getting added around latency, memory, and concurrency.
The design prompt had a name that may not be exact: next-action ranking for Meta AI. The basic idea was to suggest a few things a user might do after an interaction. That discussion went on for a while.
Round 1: coding / inference batching
The first problem was an inference request dispatcher. Each request looked like this:
{requestId, modelKey, tokenBudget, arrivalMs, deadlineMs}
The task was to batch requests for the same model version without exceeding a token limit or knowingly dispatching a request after its deadline.
The first pass used one queue per model key. A batch would be sealed when it reached the token budget or had waited for a short window. Then the interviewer started adding follow-ups:
- Requests may arrive slightly out of order.
- A request may be cancelled while waiting.
- Premium traffic has a shorter deadline, but normal traffic cannot starve.
- The queue needs a hard bound during a traffic spike.
A min-heap handled deadlines and lazy deletion handled cancelled requests. For priority traffic, the first suggestion was separate queues. The interviewer asked what would stop normal traffic from waiting forever. That took a beat, then quota and aging came in. Under overload, the design also needed to say which requests would be rejected instead of assuming every request would eventually run.
The waiting window was the part that slowed things down. It started as a fixed number, then the interviewer asked what happened when different model sizes had very different batching costs. That separated the latency and token constraints. The code itself was manageable; the hard part was revisiting decisions already made.
Round 2: coding / model-shard cache
This one initially looked like an LRU problem:
A serving worker loads model shards identified by
(model, version, shard). Implement a byte-bounded memory cache. A shard that is currently being used by an inference request cannot be evicted.
Used a hash map and doubly linked list, with byte size and a reference count on each entry. A request pins the shard until it releases it. Eviction starts from the least-recently-used side but skips pinned entries.
We did not spend long on the basic LRU mechanics. Most of the round was about concurrent loads and version changes:
- What if two workers miss the same shard at the same time?
- What if the download finishes but checksum verification fails?
- What happens to old requests during a new model rollout?
- Why could inference p99 get worse even if cache hit rate improves?
For concurrent loads, per-key single-flight. An entry being downloaded needs a loading state, and another request cannot treat it as usable until verification passes. For versions, the discussion separated which version new traffic should use from whether old shards were still in memory and pinned by active requests.
The p99 question was briefly counterintuitive because a higher hit rate sounds positive. Possible causes included larger shards in the new version, slower GPU transfer, and eviction churn pushing out genuinely hot shards. It felt like the interviewer mainly wanted to see whether the answer stopped at the hit-rate number.
This round felt good. I expected an LRU implementation, but it turned into a small model-distribution and rollout discussion instead.
Round 3: ML system design / next-action ranking
The design prompt was next-action ranking for Meta AI, or at least something close to that name. After an interaction, the product could suggest things like summarizing an image, drafting a reply, setting a reminder, or opening a related conversation. The question was how to pick a few that were actually useful without suggesting something obviously off.
The first question was what “useful” meant. A click can be accidental, so completed actions, immediate dismissals, and whether people used the action again mattered more than CTR alone. The interviewer kept coming back to that when asking about metrics.
The sketch was simple: get candidate actions from the context, run an eligibility or policy filter, then rank them. If there was enough budget and useful features, there could be a heavier stage afterward.
context
-> candidate generation
-> eligibility / policy filters
-> lightweight ranker
-> second-stage ranker
-> safety + freshness checks
-> top actions
The follow-ups were mostly about a slow feature store, cold start, offline NDCG improving while serving p99 got worse, CTR improving while completed actions fell, and a cross-region rollout.
For a slow feature store, the answer was not to wait forever: fall back to context and simpler rules. A new user would also start with context and population priors. Exploration traffic and IPS came up for position bias, but that part did not go very deep.
Near the end came a question about shipping when the aggregate metric was positive but one small slice got much worse. The average should not hide that. Start with shadow traffic and a small percentage, then stop if that slice crossed an agreed threshold. We did not get into calculating the exact threshold.
We did not spend much time on model architecture. It was more about the objective, biased data, and what to do when the online system got slow.
Round 4: behavioral / project deep dive
The main question was about a time model serving became faster but correctness was put at risk. The example was request coalescing: it reduced repeated computation, but some requests could reuse stale features.
The interviewer stayed on the same decision for several follow-ups: why launch it, who disagreed, and what actually happened during the first rollout.
The serving path was missing its latency target, so the work included the proposal, benchmark, and rollback plan. Shadow traffic came first, with metrics split by traffic type. The first rollout was paused because freshness looked wrong for one slice. The policy was adjusted before increasing traffic again.
There were also questions about a researcher who wanted more quality data while engineering wanted to reduce cost, and how to scope a request that only said “make the system cheaper.” Those stayed close to the project rather than going further into technical detail. At the end they asked briefly about using an AI coding assistant. Mostly used it to understand unfamiliar code paths and organize hypotheses, while reproduction, diff review, benchmarking, and the release decision stayed with me.
Overall
The main takeaway afterward was that it was easy enough to make the first version of an answer sound coherent. The harder part was getting pressed on failures, versions, and edge cases. The first coding round showed that; the cache round went better because more of those cases had been thought through beforehand.