Staff Machine Learning Engineer, LLM Inference Optimization

<div class="content-intro"><p><strong>About Nebius:</strong></p> <p>Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.</p> <p>Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.</p> <p>Listed on Nasdaq (NBIS) and headquartered in Amsterdam, we have a global footprint with R&D hubs across Europe, the UK, North America and Israel. Our team of 1,500+ includes hundreds of engineers with deep expertise across hardware, software and AI R&D.</p></div><div class="_3iDTPbvLZJ renderer-heading-wrapper" data-testid="renderer-heading-wrapper" data-level="2"> <h2 id="Role-Summary" data-local-id="ad213d8fe4e2" data-renderer-start-pos="1929">Role Summary</h2> </div> <p data-renderer-start-pos="1943" data-local-id="4daa64b4c380">Nebius Token Factory is building production-grade inference for open-source, customer, and internally optimized models. This role owns the path from model artifact to fast, reliable, cost-efficient endpoint. The work sits at the intersection of model internals, inference engines, serving architecture, benchmarking, and production deployment.</p> <div class="_3iDTPbvLZJ renderer-heading-wrapper" data-testid="renderer-heading-wrapper" data-level="2"> <h2 data-local-id="6b105a288faf" data-renderer-start-pos="2508"> </h2> <h2 id="What-You-Will-Do" data-local-id="6b105a288faf" data-renderer-start-pos="2508">What You Will Do</h2> </div> <ul class="ak-ul" data-local-id="ebbff5d80805" data-indent-level="1"> <li> <p data-renderer-start-pos="2528" data-local-id="c5115fb223c0">Own a platform-level optimization pillar such as quantization, speculative decoding, <span data-highlighted="true" data-vc="highlighted-text">KV</span>-cache optimization, engine selection, or long-context serving.</p> </li> <li> <p data-renderer-start-pos="2682" data-local-id="0ffc3414d75f">Create reusable evaluation, benchmarking, and rollout frameworks across multiple model families.</p> </li> <li> <p data-renderer-start-pos="2782" data-local-id="b1093d4c9089">Provide technical direction to MLEs, Applied Scientists, GPU kernel engineers, and infrastructure engineers on shared performance goals.</p> </li> <li> <p data-renderer-start-pos="2922" data-local-id="ba8442a0b3db">Optimize LLM and <span data-highlighted="true" data-vc="highlighted-text"><span class="_1mT5xdRAKS"><span class="_0nmozVaHVx _2V7sgO2M4S _3BoeIDwZ7G _30zrXRyxdf _0ReiIH7zJH" data-testid="definition-highlighter">VLM</span></span></span> endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.</p> </li> <li> <p data-renderer-start-pos="3046" data-local-id="295f060f2f89">Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, <span data-highlighted="true" data-vc="highlighted-text">NVIDIA</span> Dynamo, or similar systems.</p> </li> <li> <p data-renderer-start-pos="3204" data-local-id="a92060bc9227">Build and productionize model-compression workflows, including quantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery.</p> </li> <li> <p data-renderer-start-pos="3367" data-local-id="fc94ad87ea7d">Implement or integrate speculative decoding, draft-model approaches, <span data-highlighted="true" data-vc="highlighted-text">KV</span>-cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated prefill/decode serving.</p> </li> <li> <p data-renderer-start-pos="3558" data-local-id="767163fd69ef">Build reproducible benchmark harnesses for <span data-highlighted="true" data-vc="highlighted-text">TTFT</span>, <span data-highlighted="true" data-vc="highlighted-text">TPOT</span>, tokens per s