Interview Experience | Post-Training / RL | π₯π₯ | Cohere
2026 (late Sep to early Oct) Member of Technical Staff, Integration/RL Team @ Cohere β New York β Virtual β 3 rounds | π Neutral | Medium-Hard
Integration/RL team. They own LLM post-training: SFT, offline preference, and large-scale RL. Phone screen plus two virtual onsite rounds.
Round 1 β Phone Coding
Preference Consistency. I got a list of pairwise preferences (winner_id, loser_id), each meaning one response beat another. Check whether the batch has a cycle; if it is a DAG, return one valid ranking of the response ids.
I built a directed graph and ran Kahn with indegrees. On the first pass I had the edge direction flipped. Their small example had two responses that each beat the other once, which should be a cycle, but I returned a partial order. After the fix, a cycle returned an empty list and a DAG returned a topo order.
Follow-up: whether duplicate (a,b) pairs should be weighted. I only said treat them as a single edge for indegree. Did not implement a weighted version.
Round 2 β Coding
Offline Preference Batching. Input is a list of preferences, each with prompt_id / chosen_id / rejected_id, plus a batch size B. Pack them into batches of at most B so that the same prompt_id never appears twice inside one batch.
I went left to right and kept a set of prompts already used in the current batch. If the next pair fit, it went in; otherwise I opened a new batch. A trailing short batch stayed.
They added a case where the same prompt showed up with chosen and rejected swapped. My first version only deduped by prompt and ignored contradictory pairs. After a hint I merged on (prompt, frozenset({chosen, rejected})), dropped contradictions, then packed. Majority vote across annotators was only sketched verbally and not finished.
Round 3 β System Design
Design a post-training eval harness for Integration/RL. After each SFT / preference / RL checkpoint, run a fixed eval suite and decide whether the run can advance to the next training stage or be handed to researchers.
I drew a checkpoint registry, an eval worker pool, suite configs, and a results store. Suites mix automatic graders (code, tool-use) with a smaller preference / LLM-judge slice. Promotion was βprimary metrics do not drop past a threshold, and the safety suite is all green.β
Follow-up: during an RL run the policy already moved on, but in-flight rollouts still come back with rewards from the old policy. I said stamp policy_version on each trajectory and aggregate per version. Whether old-version rows should enter the offline preference pool was only a one-liner β default no β and the filter rules were left open.