Microsoft's Core AI Vision: Jay Periq on AI Infrastructure, Security & the Future of Work

Meta Title: Microsoft's Core AI Vision: Jay Periq on AI Infrastructure, Security & the Future of Work Keywords: AI infrastructure, AI security, Microsoft Core AI, GPU utilization, model efficiency, open source vs closed source AI, AI hacking, AI investment, cultural transformation, AI deployment

An interview with Jay Periq, Executive Vice President of Core AI at Microsoft, discussing AI infrastructure, security, and the impact of AI on enterprise workflows and culture.

Here's my interview with Jay Periq the executive vice president of core AI at Microsoft. In this interview we talk about AI infrastructure whether there are any dark GPUs whether energy is a limiter of course given his experience in security we talk about AI hacking and AI security in general and why his team is returning to office fulltime. Enjoy.

🛡️ SOURCE: Interview with Jay Periq, Executive Vice President of Core AI at Microsoft.

🚀 KEY TAKEAWAYS:

• Microsoft's Core AI team is focused on providing a comprehensive platform for developers and enterprises to build and deploy AI applications.
• Model efficiency and data center optimization are critical for scaling AI deployments and managing energy consumption.
• Security and trust are paramount, with AI agents requiring robust identity management and compliance frameworks.

1. The Vision for Microsoft's Core AI Team

We created the core AI team early this year and in May this year at Microsoft Build the conference we rolled out this product strategy this vision, right?

🧠 WHY: To think about in the new world of AI what is it going to take to help builders help developers help enterprises be successful with this technology.

There's just a lot of different pieces. There's all these different models there's different sort of frameworks out there now we have different protocols like MCP and then how do you put all this together in a new stack? And I I say stack kind of in in quotes because it's evolving it's not something that we have oh hey here's the blueprint we know how to build it you know it's just cookie cutter thing right now so it's changing kind of every week. It's a different way of working it's a different way of engaging with our customers and so what we're focused on at the top layer is reinventing re-imagining all the tools that you need to build software a different way in this AI era that sits on top of a platform that we call Foundry or our agent factory that is where these agents these AI applications are going to be built they're going to be deployed they're going to be the places where you can observe those those entities those agents in the enterprise workplace in that in that space. And then for organizations big or small security and trust for AI has to be baked in from the start, right? Because these are things that these agents these AI applications are not as deterministic as software as we've built in the past where we could go through kind of a compliance checklist or a set of kind of security policies to go and make sure that you set up correctly. Here you've got some bit of thinking planning reasoning calling out to different tools different access to different data so these things are going to do more complex tasks in the enterprise and so security and trustworthy AI has to be there from the start. And then for our enterprises we want to give them a flexible deployment strategy so a lot of this stuff today runs in the cloud but as we think about different types of sectors different types of geos in the world some of these agentic applications are going to run in the edge in edge devices, right? so that programming model of what we're building has to span all four of those areas.

📊 STAT: 11,000 models in Foundry today.

2. Empowering Builders and Developers in the AI Era

The goal the end goal for us is to serve builders serve developers, right? So everything has to acrue to serving this persona developer but even the notion of a developer today is changing, right? And that's why I refer to folks builders as as builders and so everything we do in terms of our product the way we we work together the way we measure our progress the way we engage with the end user with the companies that we work with is about really looking through the entire tech stack to like make sure this stuff all adds up, right? In a way where it's going to be easy for developers to get their arms around this technology to be able to really use this to really ramp up creativity and collaboration and not be stumbling around with a lot of things that don't connect or they're hard or they're insecure or they're not observable or they're just missing parts, right? That's why this like in some ways is a vertically integrated organization that's end goal is to and our focus is to empower everyone every builder out there to shape the future with AI and these are the components that I think are necessary to put together to accomplish that focus.

💡 TIP: Stay curious, be experimenting all the time, and be openly sharing what's working, what's not working, what do we need to tweak, what are we going to do more of if we like this new pattern, let's continue on changing work.

Aspect	Description
Model Router	A Microsoft capability that helps enterprises choose the right AI models for their applications based on cost, performance, and quality preferences.

📷 IMAGE_PROMPT: A visual representation of the AI development stack, with layers representing tools, platforms, security, and deployment options (cloud, edge).

AI Development Stack

AI Model Deployment: Pros/Cons

👍 Pros

• Smaller, targeted models can be more efficient for specific tasks.
• Enterprise data can be used to fine-tune and improve model performance.

👎 Cons

• Larger, more complex models may be needed for certain use cases.
• Managing a diverse range of models can be challenging.

3. Navigating the AI Infrastructure Landscape: GPUs, Power, and Efficiency

I think it depends in terms of what people's ambitions and plans are, right? And I think there is a lot of supply chain scaling up that's happening whether it be power land whether it be kind of the big heavy equipment transformers those types of things that go into running a hypers scale or a big scale data center, right? And those are all things that the industry is obviously rallying on and you know people are scaling filling up their manufacturing and we're all teaming up to kind of tackle those changes. And it depends on kind of the geo in the in the in the world as well, right? and what might be the constraints here in the US versus what might be true in Europe there are some countries in the world that have put you know moratoriums on not building any more data centers because they rode that boom you know for a really long time for a while and now there's a pause for a while, right? whereas others are you know open for business and want to invest and build and scale more more rapidly, right? I think this is not atypical of some like infrastructure um I guess acceleration boom right now that we're seeing but it's exciting because I think a lot of really interesting engineering challenges are being sorted out in terms of how to do these things cheaper better faster. There's another aspect of this which is the actual hardware kind of what these AI systems required last year versus this year versus next year is changing, right? Even if you look at say Nvidia's hardware roadmap, right? you look at the generational differences and what they will do to the entire system from cooling to power to what that network looks like. And now even with these more advance advanced agents that we're building and starting to deploy in the enterprises it's interesting because these agents actually make a lot of tool calls they talk to a lot of other systems and that then drives up the amount of normal compute and storage and network not GPU load, right? So the more these agents become cap more capable in the enterprise, right? and they're able to tackle higher what I would say higher ROI workflows or tasks or programs then they do need to interact with a lot of these other enterprise systems Those enterprise systems are all the things we've been building for decades, right? but now you're able to get a lot more throughput and it drives up the utilization of the conventional stuff too. So we have a whole kind of system scaling problem here and as much as there is a lot of like money and focus on kind of the GPU and AI data center it is a system that is evolving and and growing and advancing at a very rapid rate beyond just you know the the the chip the GPU or a particular AI data center.

⚠️ Warning: Constraints may exist regarding power availability and supply chain scaling for data centers, impacting the pace of AI infrastructure development.

4. Safety and Security Considerations for AI Deployment

I think the attack vectors that most security people worried about is the ones you know that you don't know yet, right? And I think the ones that you know you can put in mitigations to prevent those or to catch those and to mitigate them quickly. I think we're in a place just like we talk about what AI can help you do to create and to solve problems that humankind may not have been able to solve for decades in the past now those things are reachable, right? with and and discoveries but the flip side is is okay well what can this technology do to break down a lot of the security conventions we've had in the past, right? because one it can operate really really fast two it's advancing really really fast, right? and three like it's learning along the way too. So what we're doing and and it's throughout the the company but even here at Ignite everything we do has to hook into the overall security of the enterprise of the organization, right? So for example an agent created in our platform will get an ID that ID is tracked in entra it has policy and compliance and you can track it and you can grant it access or not and if it's doing something that it's not supposed to you can deactivate this, right? so these aren't things that are an afterthought for us they're designed kind of from the get-go when we're building a new part of the platform or a new tool and providing that understanding of what it is that observability of what it is and and making sure it adheres to your company's compliance or governance or security guidelines that you have your expectations that you have and then being able to act on these things, right? Being able to go and say "Hey I want to go and trace through what this agent did yesterday in this customer support case." And being able to see line by line everything it did all the tools it called all the data it access if there was a human in the loop what did they approve or not approve and being able to really learn from how these things are operating and and that but security is got to be there from the start for us.

5. The Evolving Culture of Work in the Age of AI

I believe and I think we're excited to have people be able to collaborate be able to create and be able to mentor coach learn from each other in person, right? because in some ways so much of being able to use these AI tools is learning from each other, right? because the stuff is changing so fast, right? And somebody may find a unique way to use co-pilot to solve this type of task and then you want to broadcast you want other people to share, right? or it's like hey we should really push this AI system to do a more complex thing let's do it as a team let's strategize for how we might you know prompt this thing or provide more context or build some other scaffolding that might be missing but let's do that as a team. So one the technology is just changing so so so fast and I think being in person enables us to learn faster so that we can really stay along that exponential trajectory that this technology is on right now. We need to be progressing as fast as the technology is progressing, right? And Microsoft the organization covers a lot of different products it covers a lot of different enduser personas whether it be somebody in human resources or in the finance team or in an engineering team or a security op center we serve all of those different personas and we're trying to build our products in a way that bring AI as a superpower to every one of these different departments every one of these end users. For example for us is in corei we have this focus on a program called nge thrive inside of nge thrive this program there's three pillars here but a big part of it Matt is what we're trying to do is understand how we're spending our time and what can we do to free up kind of run the business and other time so that it can be reallocated to creative time where we could be making products improving improving our products delivering more value to our customers to our partners but we can use AI to actually be more time efficient, right? with run the business stuff with administrative work that we're doing.

FAQ

How does the Microsoft Core AI team view the role of open source versus closed source AI models?

We want to support choice because we know one the space is advancing super fast, right? and to be dogmatic about one or the other isn't what builders and developers around the world want.