8. Interview Questions

A regional Azure resource — plus a portal and SDK over it — that hosts model deployments, agents, connections, and evaluations, giving generative AI work a home inside your subscription with Azure's identity, networking, and governance already attached.

The problem is not "how do I call a model" — that is four lines of code. It is everything after: who may call it, where the logs go, how you change model versions without a redeploy, how you attach your own data, how you prove quality has not regressed, and how it passes a network review. Foundry answers that list as one resource with one RBAC surface.

Bonus marks for naming the rename chain — Azure AI Studio → Azure AI Foundry → Microsoft Foundry (Ignite, November 2025), with Azure AI Services becoming Foundry Tools — while noting the resource provider is still Microsoft.CognitiveServices.

Your code calls a model deployment by name. That deployment is a child of a Foundry account (Microsoft.CognitiveServices/accounts, kind = AIServices), which also contains projects — child resources scoping connections, agents, evaluations, and role assignments. The account lives in a resource group, in a subscription, in a tenant.

The key subtlety: projects share the account's model deployments, network configuration, and quota. They isolate people and artefacts, not capacity or blast radius. Needing real isolation means needing a separate account, and usually a separate subscription.

The classic (hub-based) alternative puts an AI Hub on Microsoft.MachineLearningServices/workspaces with projects beneath it, and requires a storage account and Key Vault as siblings.

A deployment binds a model + version + deployment type + capacity under a name you choose. Your application calls the name, not the model.

It matters because the name is an indirection you can change behind the caller's back: swap the model version, change the capacity, move to provisioned throughput, or point at a different model entirely — all without a code change. It is also what makes multi-region routing trivial, because identical deployment names in two regions mean the router only has to change a hostname.

The anti-pattern is naming the deployment after the model (gpt-4o-2024-08-06) and hard-coding it, which throws the indirection away. Name it for the job: chat-default, summarise-cheap, embed-default.

Foundry when you are consuming and orchestrating models: generative applications, RAG, agents, prompt engineering, evaluation of model outputs. Azure ML when you are building them: datasets, training runs, compute clusters, custom models, the model registry, and classic MLOps.

They overlap by design — the hub-based Foundry architecture is built on Azure ML, which is why the vocabulary bleeds. If your artefact is a training run or a .pkl, that is AML. If your artefact is a prompt, an agent, or an index, that is Foundry.

Three meters. Tokens on standard deployments, priced separately for input and output, with a discount for cached input — no traffic, no bill. PTU-hours on provisioned deployments, charged from creation to deletion whether or not a single request arrives. And the dependencies — AI Search, Cosmos DB, Storage, Log Analytics ingestion — which run continuously and, on a low-traffic agent, routinely exceed the token bill.

The idle-billing answer is the provisioned deployment, plus the standard agent setup's Cosmos DB and Search. Also worth naming: a purchased PTU reservation is a billing commitment that survives deleting the deployment.

Token acquisition from Entra ID → DNS resolution of <account>.services.ai.azure.com (private IP if a private endpoint and the matching Private DNS zone exist) → data-plane authorisation against a role assignment → network ACL evaluation → input content filter → routing to capacity for that deployment's type, with TPM/RPM measured against your allocation and 429 + Retry-After if over → generation, with time-to-first-token driven by prompt length and total time by output length → output content filter, in chunks if streaming → telemetry to whatever diagnostic destinations you configured.

The thing that is not in the path: any autoscaling. Capacity was fixed when the deployment was created and when your quota was granted.

Control plane is management.azure.com: create/update/delete accounts, projects, and deployments; set quota allocation, networking, and keys. Governed by Cognitive Services Contributor, Foundry Account Owner, Owner.

Data plane splits in two: model inference at <account>.services.ai.azure.com (governed by Cognitive Services OpenAI User / Contributor, or an API key if local auth is enabled), and the project data plane at .../api/projects/<project> for agents, threads, and evaluations (governed by Foundry User — formerly Azure AI User — and Foundry Project Manager).

The classic mistake: you are Owner, you created the deployment, and your application gets 403 calling it. Owner is control plane. Inference needs a data-plane role on the account or project. The portal hides this by auto-assigning the role when a human clicks through — which is why it works for you and fails in the pipeline, where the SDK and CLI assign nothing.

Second-order point: Cognitive Services Contributor can read the keys, so while local auth is enabled it is effectively a data-plane grant. Setting disableLocalAuth = true closes that door.

Standard (regional) — inference stays in the region; tightest residency story; smallest capacity pools and the fewest models. DataZone Standard — anywhere within a Microsoft data zone (US, EU, APAC); more capacity, a compliance boundary that is regional-ish but not country-level. Global Standard — anywhere in the fleet; the largest pools, newest models first, usually the best price, and the weakest residency guarantee. Batch — asynchronous, discounted, not interactive. Provisioned (PTU) — reserved capacity, predictable latency, billed per hour regardless of traffic.

Moving between them is not a resize. Deployment type is part of the deployment's SKU, so changing it means delete and recreate — the name is briefly unavailable and callers get 404s. Create the replacement under a temporary name and cut over, or use two names behind a router. Moving also moves you to a different quota pool, which may not have room, and may change your data-residency posture, which may need sign-off rather than a pull request.

Set disableLocalAuth = true so keys stop working entirely, and enforce it with Azure Policy. Give the application a managed identity and assign Cognitive Services OpenAI User at the account (or project) scope — nothing broader. Give the pipeline a separate, OIDC-federated identity with Cognitive Services Contributor scoped to the resource group, and a different one per environment.

For everything Foundry itself reaches — AI Search, Storage, Cosmos DB — use the account's or project's managed identity with data-plane roles on those targets, so connections are identity-based rather than key-based. Add a private endpoint on the account sub-resource with publicNetworkAccess disabled and the Private DNS zones linked, and customer-managed keys if the data classification requires them.

Then remember the two operational details: automation must assign data-plane roles explicitly because only the portal does it for you, and agent tools must authorise the end user, not just trust the agent's identity.

There is no autoscale. Scaling is one of four deliberate acts: raise a deployment's capacity within your pool, request more quota, add a region, or buy provisioned throughput.

The ceiling is TPM quota, counted per subscription, per region, per model, per deployment type — four dimensions, and people routinely forget the last two. PTU quota is per subscription, per region, per deployment type. You allocate the pool across deployments yourself, which means a forgotten dev deployment in the same subscription and region can prevent a production deployment from being created at all. That is the argument for one subscription per environment.

Over-consumption returns 429 with Retry-After. RPM can bind before TPM on many-small-call workloads. Quota increases are a request with lead time and can be refused in a hot region, so capacity belongs on the project plan, not in the runbook.

Changing the account's kind, its custom subdomain, or its location forces replacement. So does disabling project management on an account that has it (unless kind changes to OpenAI).

Replacement is expensive here in a specific way: a new endpoint hostname, every model deployment gone and needing recreation against fresh quota, every data-plane role assignment gone, connections gone, and — because Cognitive Services accounts are soft-deleted — the old globally unique name still reserved and blocking the recreate until it is purged (az cognitiveservices account purge, or the purge_soft_delete_on_destroy provider feature). If there is a CanNotDelete resource lock, the apply fails partway with what looks like a permissions error.

Read every # forces replacement in a Foundry plan carefully. On this resource it is never a small change.

Diagnose: check the throttled-request metric and the token consumption rate against your allocation first — intermittent-at-peak plus slow-otherwise is the signature of a shared standard deployment at its TPM ceiling with contention below it. Confirm in the logs whether the failures are 429 and whether Retry-After is being honoured; check whether RPM rather than TPM is binding, which happens on many short calls. Separate model latency from tool and retrieval latency using the end-to-end trace — "slow" is frequently retrieval, not inference.

Fix, cheapest first: honour Retry-After with jitter and stop hammering. Reduce input tokens — retrieval returning ten chunks where three would do is a permanent multiplier. Split interactive and batch traffic onto different deployments so they cannot starve each other. Then, structurally: request more quota, add a second region with matching deployment names behind APIM with a load-balancing policy and circuit breakers, and consider provisioned throughput for the steady base load with spillover to standard for bursts. Add per-consumer token limits so one team cannot do this to everyone else again.

Residency first, because it constrains everything else. GlobalStandard is off the table if the obligation is that inference stays in Europe; DataZoneStandard keeps processing in the EU data zone, and regional Standard keeps it in one region — but check that the models you need are offered that way, because the newest models reach Global first. Confirm whether the obligation is EU-level or country-level; "EU" is a data zone of many countries and that distinction has killed designs late.

Network: private endpoints on the account sub-resource, publicNetworkAccess disabled, networkAcls default Deny, Private DNS zones linked to every VNet including the build agents', and outbound access restricted. Everything Foundry touches — Search, Storage, Cosmos — behind private endpoints too.

Identity: local auth disabled, managed identities throughout, Entra ID with Conditional Access for humans, and data-plane roles assigned explicitly in IaC.

Headroom: two EU regions, matching deployment names, APIM in front with per-consumer token limits, backend pools, and circuit breakers. Provisioned capacity for the predictable base once you have several weeks of measured p95, standard for burst. Separate subscriptions per environment so dev never eats prod's quota. Request quota in both regions early — it is a lead-time item.

Data: RAG over an EU-resident AI Search index with security trimming in the index so retrieval respects user permissions; agent state in your own Cosmos DB (standard setup) rather than Microsoft-managed; customer-managed keys in a Key Vault with purge protection.

Governance: Azure Policy at the management group denying local auth, denying public network access, and deploying diagnostic settings; content filter configuration change-controlled; evaluation suite in the pipeline; retention on RequestResponse logs set deliberately, because they contain user text.

First, identify which layer changed, because they roll back at wildly different speeds. A prompt or agent definition: redeploy the previous artefact, seconds. A model version: set it back, minutes — unless the old version has retired, in which case there is no back, only forward, and you should have found that in staging. Capacity: change it back, minutes, though increasing may fail on quota. Deployment type: that is a delete-and-recreate, so the name goes away and callers 404 — create under a temporary name and cut over rather than deleting in place. Account kind, subdomain, or location: there is no rollback; that is a replacement, and you are rebuilding deployments, role assignments, and connections against a soft-deleted name that is still blocking you.

Blast radius: the account is the boundary for endpoint, network, keys, and quota consumption — everything inside shares them. The project is not a blast-radius boundary. So the damage from a bad account-level change is every project and every workload on that account.

The Azure-specific traps to name: soft delete means a destroyed account holds its globally unique name until purged; a CanNotDelete lock makes the apply fail in a way that reads like a permissions bug; and a re-run in complete deployment mode will delete resources in the resource group that are not in the template — which on a Foundry resource group typically means the Cosmos DB and AI Search holding your agent state. Always what-if with the mode you intend to use.

Find out: terraform plan shows the diff immediately if the deployment is in state — this is why a scheduled nightly plan that fails on any non-empty result is the single most effective drift control on this service. For who and when, the activity log, filtered to Microsoft.CognitiveServices/accounts/deployments/write; Change Analysis gives the before-and-after. An alert on that operation in prod turns this from archaeology into a notification.

Get clean: decide whether the human or the code was right. If the incident change should stick, update the .tfvars capacity and apply so code and reality agree. If it should not, apply the code as written — but understand you are re-imposing the pre-incident capacity, so confirm the incident is actually over. Either way the outcome is an empty plan, and the change is now in git with a reason.

Then close the hole. Foundry drifts more than most services because the portal is designed to be worked in. Give humans Reader plus a break-glass path in prod rather than standing write access; keep connections and content filter configurations in IaC so they show up in the plan; and add the Azure Policy compliance checks for the settings you never want changed by hand. Note the one that will not show up in terraform plan at all: a connection created in the Foundry portal that was never in state. Import it or delete it — do not leave it.