Jobs/Senior Machine Learning Engineer, Evals

Senior Machine Learning Engineer, Evals

ProsusAmsterdam
Full Time

About the Role

<p style="min-height:1.5em"><strong>Who We Are</strong></p><p style="min-height:1.5em">You've ordered dinner through Just Eat Takeaway. You've bought and sold on OLX. You've paid with PayU. Chances are, you've already used a Prosus company today.</p><p style="min-height:1.5em">We are Prosus, a global technology company powering some of the world's leading lifestyle ecommerce businesses. Every day, billions of people across India, Latin America, Europe and beyond use our products and services to buy, sell, pay, connect and discover.</p><p style="min-height:1.5em">Our AI-driven ecosystem spans 100+ companies across ecommerce, food delivery, fintech and experiences, serving billions of consumers worldwide. AI powers how we operate and innovate, with more than 60,000 active AI agents helping our teams build, scale and move faster. Through our investment team, we also back the founders and technologies shaping the future: From robotics and autonomous systems to synthetic biology and frontier AI.</p><p style="min-height:1.5em">Guided by our five values <strong>Innovation, Results, People, Impact, and Entrepreneurship,</strong> culture is our competitive advantage. We thrive with experimentation; we build, we test, we learn, we iterate.</p><p style="min-height:1.5em">Join us in shaping the future of technology at global scale.</p><h2><strong>About the team</strong></h2><p style="min-height:1.5em">Join the AI team at Prosus, a global technology group with businesses in food delivery, e-commerce, classifieds, travel and payments. We build LCM, the Large Commerce Model: our own foundation model, trained on billions of transactions and interactions from proprietary data across the portfolio. LCM already powers search, recommendations and marketing in production at iFood. We are extending it across our companies around the world and to businesses outside the group, while broadening its capabilities in agents and real-time prediction tasks. Your work will shape what LCM learns next and feed directly into products used by millions of people.</p><p style="min-height:1.5em"></p><h2><strong>About the role</strong></h2><p style="min-height:1.5em">Evaluation is the core of everything we do. We build from the evals: we define how success will be measured before developing a capability, and use the results to guide training, model selection and deployment. We already have hundreds of evaluations spanning domain knowledge, commerce and gaming, with plans to expand their coverage, depth and scale.</p><p style="min-height:1.5em">This work builds on ProLLM, the public benchmarking platform we ran for real-world use cases until our coding assistant benchmarks became saturated. Since then it has been a private benchmarking platform for Prosus companies. We plan to open it again to publish our commerce benchmarks, covering both agentic and decision-making tasks.</p><p style="min-height:1.5em">We’re looking for a Senior Machine Learning Engineer to own the development of these evaluations and the platform that runs them. The work involves defining meaningful tasks, curating evaluation data, implementing reliable scoring, investigating model behaviour and making results useful for research and product decisions.</p><p style="min-height:1.5em">In our flat team of around 30 engineers, you’ll work directly with applied scientists, training engineers and domain experts. You’ll help teams contribute evaluations and build the infrastructure and automation needed to run them throughout model development.</p><p style="min-height:1.5em">In the first three months, we expect you to take technical ownership of part of the evaluation platform, deliver a set of new or improved evaluations with curated data and calibrated difficulty, and work with scientists to use them in a training or release decision.</p><p style="min-height:1.5em"></p><h2><strong>What the role involves</strong></h2><ul style="min-height:1.5em"><li><p style="min-height:1.5em">Design evaluations with scientists and domain experts, translating intended capabilities into representative tasks, datasets and success criteria. Cover current applications and capabilities we want future models to acquire.</p></li><li><p style="min-height:1.5em">Curate evaluation data and calibrate task difficulty, so that evaluations are neither saturated nor out of reach and still separate models of different quality. Work with domain experts on sourcing, annotation and review, within data provenance, permitted use across companies and privacy requirements.</p></li><li><p style="min-height:1.5em">Own the development and operation of the evaluation platform across training checkpoints, deployed models and external baselines. Support distributed execution, reproducible configurations, trace inspection and comparison of results, while managing runtime and cost.</p></li><li><p style="min-height:1.5em">Develop programmatic checks, outcome-based scoring and LLM-based graders. Calibrate automated judgments agains
Apply on Company Site

Apply for this Position

Fill out the form below to apply for this role

PDF or Word document, max 10MB

By submitting this application, you agree to your information being shared with the hiring team.

About the Company

Prosus

Job Details

Job Type

Full Time

Location

Amsterdam

Reference

EXT-MWKN5DLE-40F2

Posted

17 Nov 2026

Views

7 views

Applications

0 applications