Senior AI Researcher- Pre-training (f/m/d)

<h2><strong>Our Mission</strong></h2><p style="min-height:1.5em">Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released <a target="_blank" rel="noopener noreferrer" href="https://huggingface.co/Aleph-Alpha/Kolibri-1">Kolibri-1 model</a>. Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We're building that in Heidelberg.</p><p style="min-height:1.5em">We are hiring a Senior AI Researcher to join our Pre-training team and to advance the architecture and training of our next generation of foundation models. If you are excited about designing inference-efficient architectures, optimizing training recipes that scale reliably, and training models on a large scale cluster (thousands of NVIDIA Blackwell GPUs), we would love to hear from you.<br /></p><h2><strong>Team Culture</strong></h2><p style="min-height:1.5em">We foster a culture built on ownership, autonomy, and empowerment. Teams and individual contributors are trusted to take responsibility for their work and drive meaningful impact. We maintain a flat organisational structure with efficient, supportive management that enables quick decision-making, open communication, and a strong sense of shared purpose. We collaborate closely on complex technical problems, working in pairs or using mob programming to resolve challenging issues.</p><h2><strong>About the Role</strong></h2><p style="min-height:1.5em">As a <strong>Senior AI Researcher in Pre-training (f/m/d)</strong>, you will own the critical technical levers that determine the success of our next-generation models: architecture, optimization, stability, and scaling. </p><p style="min-height:1.5em">Working at the high-leverage intersection of research and engineering, you will translate mathematical reasoning and empirical observations into principled training decisions - from small-scale proxy experiments to multi-thousand-GPU runs. </p><p style="min-height:1.5em">We are looking for an expert who can combine rigorous experimental design with high-quality production code, directly influencing model quality, run reliability, and the efficiency of the models we ship.</p><p style="min-height:1.5em"></p><h2><strong>Your Responsibilities</strong></h2><ul style="min-height:1.5em"><li><p style="min-height:1.5em"><strong>Recipe &amp; Architecture Optimization:</strong> Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.</p></li><li><p style="min-height:1.5em"><strong>Scaling Strategy &amp; Predictability:</strong> Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions.</p></li><li><p style="min-height:1.5em"><strong>Stability, Diagnostics &amp; Debugging:</strong> Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors.</p></li><li><p style="min-height:1.5em"><strong>System-Model Co-Design:</strong> Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.</p></li></ul><h2><strong>Core Qualifications</strong></h2><ul style="min-height:1.5em"><li><p style="min-height:1.5em">You are proficient in Python and deeply familiar with PyTorch-based training workflows.</p></li><li><p style="min-height:1.5em">You have a strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.</p></li><li><p style="min-height:1.5em">You have a strong mathematical foundation and are comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics.</p></li><li><p style="min-height:1.5em">You deeply understand transformer training dynamics, optimisation, and the behaviour of large distributed training jobs.</p></li><li><p style="min-height:1.5em">You can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.</p></li><li><p style="min-height:1.5em">Hands-on experience pre-training large models <strong>(e.g., 7B+ parameters)</strong> on substantial infrastructure <strong>(e.g., 100+ GPU clusters).</strong></p></li></ul><ul style="min-height:1.5em"><li><p style="min-height:1.5em">You apply strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.</p></li><li><p style="min-height:1.5em">You are able to implement complex model