Azure AI Foundry

Azure AI Foundry is the place you build, deploy, evaluate, and operate AI applications and agents on Azure. It is two things wearing one name: a portal and SDK you develop in, and an ARM resource that hosts your model deployments, your agents, and the connections between them. If Azure OpenAI gave you a model endpoint, Foundry gives you the workspace around it.

Names — read this first, because there are four of them. The service began as Azure AI Studio, was renamed Azure AI Foundry in 2024, and at Ignite on 18 November 2025 was renamed again to Microsoft Foundry — dropping "Azure" to signal it now spans more than the Azure portal. Underneath, Azure AI Services (itself formerly Cognitive Services) has been rebranded Foundry Tools, and Azure OpenAI Service is now Azure OpenAI in Foundry Models. The built-in RBAC roles were renamed too — Azure AI User → Foundry User, Azure AI Project Manager → Foundry Project Manager, and so on — and Microsoft says both sets of names will appear in the portal while the rename rolls out. This topic uses Azure AI Foundry as the heading because that is what the resource provider, most tooling, and most of the ecosystem still say, and flags the current name wherever it matters. ⚠️ Verify the current portal branding and role names against current Azure docs — this one is moving.

What it is and where it fits

Strip away the portal and Foundry is a container for AI capability inside your subscription. You create one Foundry resource; inside it you create projects; inside a project you create model deployments (a name you call, pointing at gpt-4.1 or Llama or Mistral at some capacity), connections (credentials to a search index, a storage account, a Bing grounding resource), agents (a model plus instructions plus tools), and evaluations (how you know any of it works). Everything else — the playground, the model catalogue, the tracing UI — is a view over those objects.

The problem it solves is the one every team hits about six weeks into an LLM project. The model call itself was easy. What was hard was: who is allowed to call it, where the prompt logs go, how you swap gpt-4o for gpt-4.1 without a redeploy, how you attach a retrieval index without hand-rolling an orchestrator, how you keep thread state, how you prove quality didn't regress, and how you get all of that through a network review. Foundry is Microsoft's answer to that list, as one resource with one RBAC surface.

There are two architectures, and knowing which one you are looking at is the single most useful thing on this page. The Foundry (new) architecture puts everything under Microsoft.CognitiveServices/accounts with kind = AIServices, with projects as child resources — no mandatory storage account, no mandatory key vault, one RBAC model. The hub-based (classic) architecture puts an AI Hub on Microsoft.MachineLearningServices/workspaces with projects beneath it, and requires a storage account and a Key Vault as siblings. New capability is going to Foundry projects; hub projects remain for workloads that need the Azure Machine Learning surface (managed compute, prompt flow, managed network isolation of the AML kind). Tutorials, Terraform resources, and blog posts from 2024–early 2025 are almost all hub-shaped, which is why so much of it does not match what you see in the portal today.

The neighbours in the catalogue, separated in one line each:

• Azure Machine Learning — the training and MLOps platform: datasets, compute clusters, pipelines, the model registry, managed online endpoints. Choose it when you are building models. Choose Foundry when you are consuming them. The hub-based Foundry architecture is literally built on AML, which is why the two blur.
• Foundry Tools (formerly Azure AI Services / Cognitive Services) — the prebuilt task APIs: Speech, Vision, Document Intelligence, Language, Translator, Content Safety. These are inside Foundry now; a Foundry resource of kind AIServices exposes them on the same endpoint and the same key.
• Azure OpenAI (in Foundry Models) — not a separate product any more so much as a model family inside Foundry. A resource of kind = OpenAI still exists and still works; kind = AIServices is the superset you should create new.
• Copilot Studio / Microsoft 365 Agents — low-code agents living in the Microsoft 365 tenant. Choose Foundry when the agent is a component of your application, with your code, your network, and your telemetry.
• Azure AI Search — not a competitor: the retrieval half of nearly every Foundry RAG design, wired in as a connection.

The honest summary: Foundry is the right default for anything generative on Azure, and the wrong place to look for classic model training, feature engineering, or a managed compute cluster.

Key facts at a glance

Category	AI / ML — model hosting, agent runtime, and AI application platform
Resource provider	Foundry (new): Microsoft.CognitiveServices/accounts with kind = AIServices and project management enabled; projects are Microsoft.CognitiveServices/accounts/projects; deployments are .../accounts/deployments. Hub-based (classic): Microsoft.MachineLearningServices/workspaces with kind = Hub / kind = Project
Scope	Regional resource in a resource group, with a globally unique subdomain (<name>.services.ai.azure.com). Model availability is per-region and differs sharply between regions
SKU / tier axis	The account SKU is almost always S0 and tells you nothing. The axis that matters is the deployment type, chosen per model deployment: Standard (regional), DataZoneStandard, GlobalStandard, Batch, and the provisioned family — ProvisionedManaged / GlobalProvisionedManaged / DataZoneProvisionedManaged sold in PTUs. This choice sets your data-residency boundary, your throughput, your latency profile, and whether you pay for idle
Unit of billing	Standard: per 1,000 tokens, priced separately for input and output, with a discount for cached input. Provisioned: per PTU per hour, running or not (reservations cut this). Batch: discounted, asynchronous. Plus whatever the agent's dependencies cost — AI Search, Cosmos DB, Storage ⚠️ verify current pricing
Capacity unit	TPM (tokens per minute) quota for standard deployments, and PTU for provisioned — both counted per subscription, per region, per model, per deployment type
SLA posture	Standard deployments carry an availability SLA on the endpoint, not on latency; provisioned adds a latency dimension. Preview features and preview models carry no SLA ⚠️ verify against current Azure docs
Usual companions	Azure AI Search, Cosmos DB, Blob Storage, Key Vault, Application Insights / Azure Monitor, API Management, Private Link, managed identity
Primary alternative	Azure Machine Learning managed endpoints (for your own models), Copilot Studio (for low-code agents), or calling a model provider directly (for the fastest possible start and none of the governance)
AWS analogue	Amazon Bedrock, plus a slice of SageMaker Studio. The analogy holds for the model-catalogue-behind-one-endpoint idea and breaks on structure: Bedrock has no "project", no per-deployment named endpoint, and no capacity object you provision by name

When to use it

• Anything generative you intend to keep. A chat feature, a summariser, a classifier over free text, a document Q&A tool. The governance you get for free — RBAC, private endpoints, diagnostic logs, content filters — is work you would otherwise do yourself.
• RAG over your own data. The Foundry + AI Search + Storage triangle is the most-travelled path on Azure, and the connection model means credentials live in one place.
• Agents that call tools. Foundry Agent Service gives you threads, runs, tool invocation, and state without you owning an orchestration loop — and it does it behind Entra ID rather than an API key.
• Model comparison and evaluation. The catalogue plus the evaluation harness lets you swap models behind a stable deployment name and measure the difference instead of guessing.
• Regulated or network-restricted environments. Private endpoints, customer-managed keys, data-zone deployments, and Azure Policy are the reason many organisations use Foundry rather than a model API directly.

When not to use it

• Training or fine-tuning at scale as your main job. Fine-tuning exists here, but if your day is datasets, distributed training, and GPU clusters, that is Azure Machine Learning.
• A single hard-coded model call in a prototype. If you will throw it away next week, a bare kind = OpenAI resource — or nothing at all — is less machinery.
• Deterministic, structured work that isn't a language problem. A regex, a rules engine, or a gradient-boosted tree is cheaper, faster, and auditable. "Because we have Foundry" is not a design.
• Ultra-low-latency inference on your own small model. A container on Container Apps or AKS with the model in-process will beat a network hop to a shared endpoint.
• Workloads that cannot tolerate regional model gaps. Not every model is in every region, and the region you standardised on for everything else may not have the one you want. Check before you design.
• On-device or disconnected scenarios. That is Foundry Local's territory, not the cloud service.

What this topic covers

Sub-topic	What it covers
What & Why	The problem Foundry kills, the four-name rename, where it sits against Azure ML and Foundry Tools, the Bedrock analogy and where it breaks, and the honest anti-patterns
Core Concepts	Account, project, model deployment, deployment type, connection, agent, thread, capability host, evaluation — and the SKU axis that is really a per-deployment choice
Architecture	What happens on an inference call and on an agent run, control plane vs. data plane and the three RBAC surfaces, quota and throttling mechanics, and the failure modes
Getting Started	One Foundry resource, one project, one model deployment, one chat completion — portal, az CLI, and a minimal Terraform snippet — plus teardown and the purge trap
Deployment	A parameterised Terraform module with azurerm + azapi, remote state, an Ansible playbook, the Bicep equivalent, OIDC CI/CD, environments, model-version rollback, and drift
Integrations	AI Search, Cosmos DB, Storage, Key Vault, API Management as an AI gateway, Monitor, Content Safety, Logic Apps and Functions as tools, and the two glue mechanisms that recur everywhere
Production	Security, cost, quota and scaling scopes, observability, and reliability — including the multi-region pattern everyone eventually builds
Interview Questions	Three tiers with answer keys, from "what is a deployment" to "someone changed the model version by hand, now what"
Glossary & Cheatsheet	Every term in one line, the commands you will actually type, the resource ID shapes, and the limits worth knowing

Three ideas worth carrying into every other page

A deployment is a name, not a model. When you deploy gpt-4.1 you choose a deployment name, and your application calls that name forever. Behind it you can change the model version, the capacity, and sometimes the model itself. This indirection is the most useful thing Foundry gives you and the most commonly wasted: teams name the deployment gpt-4o-2024-08-06, hard-code it, and throw the indirection away. Name it for the job — chat-default, summarise-cheap — and you can change models without a release.

Capacity is a quota you were granted, not a resource you scaled. There is no autoscale here. A standard deployment has a TPM allocation drawn from a pool held per subscription, per region, per model, per deployment type, and when you exceed it you get 429 with a Retry-After. Growth is a quota conversation, a second region, or provisioned throughput — never a slider that solves it silently.

The control plane and the data plane have different doors, different roles, and different auth. Being Owner on the Foundry resource does not let you call the model; that needs a data-plane role like Cognitive Services OpenAI User or Foundry User (formerly Azure AI User). Conversely, a data-plane role does not let you create a deployment. This split accounts for a large share of the "it works in the portal but 401s from my app" tickets on this service.

Resource groups, subscriptions, ARM, Entra ID, and RBAC inheritance are explained once in Foundations rather than repeated here.

Reading paths

New to Foundry — What & Why → Core Concepts → Getting Started. Deploy one model, call it, delete the resource group, then come back for Architecture.

Coming from Bedrock or the OpenAI API — read the account/project/deployment section of Core Concepts first; the indirection is the part with no equivalent in what you already know. Then Architecture for quota and the RBAC split.

Need to ship this quarter — Deployment then Production. Getting Started is deliberately throwaway.

Building an agent — Core Concepts for threads, tools, and the capability host, then Integrations for the dependencies a standard agent setup drags in.

Interview or AI-102 prep — Core Concepts, Architecture, Interview Questions. Deployment types and the control-plane/data-plane split come up almost every time.

Chasing a cost surprise — the cost section of Production, then the deployment type table in Core Concepts. An idle PTU deployment and an agent quietly running a Cosmos DB and an AI Search tier are the two usual culprits.

---

Next: What & Why →

← Back to the article