In this seminar, Mark Russinovich, Azure CTO and Technical Fellow, gave an underthehood look at Microsoft’s AI architecture, including the largescale supercomputers that train foundational models and the infrastructure that efficiently serves small and large pretrained and finetuned models. He covered everything from how the company designs servers, to the AIaware resource management service that schedules training and inference, to the AIspecific techniques it’s developed for maximizing GPU usage. He also gave insight into advancements and opportunities in trending AI research and confidential AI.

This HAI Seminar took place on March 6, 2024. Upcoming HAI Events can be found here: https://hai.stanford.edu/events