Development Architect (f/m/d) - AI Foundation Model Training and Serving

We help the world run better. At SAP, we keep it simple: you bring your best to us, and we'll bring out the best in you. We're builders touching over 20 industries and 80% of global commerce, and we need your unique talents to help shape what's next. The work is challenging – but it matters. You'll find a place where you can be yourself, prioritize your wellbeing, and truly belong.

What you`ll do:

At SAP Business AI, we are exploring the next frontier of AI on structured business data like graphs, tables and more. Foundation Models on structured data, like SAP-RPT-1, have ushered in the GenAI era for business relevant scenarios. Now, we are pushing further. We want to enable companies across all industries to collaboratively train foundation models and build knowledge graphs. As an architect for the AI side of an open data platform for foundation model training, working alongside our lead Platform Architect, you will design and guide implementation of the AI stack — distributed training, federated training across customer sites, model lifecycle (fine-tuning, distillation, evaluation), deployment and serving of multi-modal foundation models. Your work will complement existing AI platforms and overcome gaps when it comes to bringing together current foundation model training approaches, symbolic approaches, and enterprise data in a distributed, heterogeneous environment.

The Role:

• Drive technical architecture and design topics of the AI training, evaluation, and serving stack within the open platform for industries – partnering with the lead Platform Architect on cross-cutting design
• Architect distributed and federated training infrastructure: cluster orchestration, parallelism strategies, checkpointing, fault tolerance, and secure aggregation across customer data sites
• Design the model lifecycle: fine-tuning, post-training, evaluation and benchmarking pipelines, model registries
• Architect inference and serving for industrial scale: multi-tenant deployment, dynamic batching, KV caching, quantization, and latency/cost trade-offs across cloud and on-prem environments
• Ensure secure, policy-compliant handling of proprietary industrial datasets and models – authentication, lineage, provenance, licensing metadata, and data residency for federated training
• Coordinate effectively with technical, product, and management stakeholders across multiple involved teams and ensure alignment with the overarching technical concepts and project vision
• Support delivery teams with your deep technical guidance on AI model training, evaluation and serving

What you bring:

• 8+ years of experience architecting AI / foundation-model ML/AI solutions
• Hands-on experience designing distributed training systems for large models – cluster orchestration, parallelism (DDP/FSDP/TP/PP), checkpointing, fault tolerance – and ideally with federated training across organizational boundaries
• Hands-on knowledge on foundation model fine-tuning (incl. RLHF/DPO/distillation), evaluation pipelines, and serving at scale and serving
• Strong hands-on software engineering skills, proficiency in Python and at least one other programming language
• Working knowledge of inference serving systems (e.g., vLLM) and serving trade-offs (batching, KV cache, quantization, latency vs. cost) is a strong plus
• Ability to translate business and research requirements into robust, scalable technical platform designs
• Experienced in collaborating effectively with product, consulting and customer teams, as well as other stakeholders
• Strong communication and documentation skills for technical stakeholders (engineers, data scientists, architects)
• Being comfortable working in a complex, evolving environment with multiple partners, technologies and dependencies, iterating the architecture as components evolve

Meet your team:

SAP's AI organization is dedicated to seamlessly infusing AI into all enterprise applications, enabling customers, partners, and developers to enhance business processes and generate remarkable business value. Join our international AI team where innovation thrives, opportunities for personal development abound, and exceptional colleagues collaborate globally.