8. Interview Questions

A managed, regional key-value store for the settings and feature flags an application reads at runtime, reached over HTTP at https://<store>.azconfig.io.

The problem it solves is lifecycle coupling. Configuration naturally changes on a different schedule from code — urgently, operationally, often by someone who isn't the author. If it lives in the deployment artifact (appsettings.json, a Helm values file, a pipeline variable), then every config change inherits the full cost and latency of a deployment. App Configuration decouples them: a setting change is a PUT to an endpoint, and within one client refresh interval every instance of every consuming app has the new value with no restart and no deployment.

Three things it adds beyond "a shared dictionary": labels (one key, a different value per environment or release), revision history and point-in-time restore (an audit trail and a real undo), and feature flags as typed objects with client-side filters for percentage and targeted rollout.

Strong candidates also name what it is not: not a secret store (that's Key Vault, referenced from here), and not a runtime database (values are read at startup and cached in memory, not read per request).

Bottom up: a key-value is the atomic item, identified by the pair (key, label) — not by key alone. Key-values live inside a configuration store, an ARM resource of type Microsoft.AppConfiguration/configurationStores. Also inside the store: feature flags (which are just key-values under the reserved .appconfig.featureflag/ prefix with a special content type), snapshots (immutable named compositions of key-values), revisions (the write history), and replicas (regional copies, each with its own endpoint).

The store sits in a resource group, in a subscription, in a tenant — the standard Azure scope chain. It is regional and its name is globally unique, because the name becomes a DNS label in the data-plane endpoint.

Two things a strong answer adds. First, there is no real hierarchy inside the store: the : in Api:Timeout is a naming convention that .NET's configuration binder maps to nested objects; the service sees one flat string and does prefix matching. Second, the store is reached through two different endpoints — ARM for the resource, azconfig.io for the contents — with separate RBAC.

Within a single replica: read-after-write consistency for your own writes. Conditional operations on the etag give optimistic concurrency, so two writers to the same key can be made to conflict loudly rather than last-write-wins silently.

Across replicas: eventual. Geo-replication is asynchronous, so writing to West Europe and reading immediately from East US may return the old value. Convergence is quick but not instant — the practical rule is never write to one region and validate from another.

Durability: platform replication within the region, with zone redundancy on the higher paid tiers in regions that support zones. Revisions give point-in-time restore of key-values within a tier-dependent retention window; soft delete protects the store resource itself and holds its globally unique name during retention. The Free tier has no SLA. ⚠️ Specific retention windows, zone-redundancy behaviour and SLA figures should be verified against current Azure docs.

The mature addition: none of that is a backup. The authoritative copy of your configuration belongs in git, applied by a pipeline, with the store treated as a projection of the repository. Revisions expire; a deleted subscription takes everything.

Versus Key Vault: it isn't a choice, it's a division of labour. Key Vault is for secrets — per-secret RBAC, HSM-backed key options, expiry, rotation events, certificate lifecycle, and correspondingly tighter throttling and higher per-read cost. App Configuration is for non-secret settings, and is cheaper and faster to read in bulk. The intended design is both: secret in Key Vault, Key Vault reference in App Configuration, one lookup namespace for the application. Putting a password directly into a key-value is the clearest misuse of the service — not least because it enters the immutable revision history, so you can't un-leak it by editing the value.

Versus App Service application settings: app settings are free, already deployed, injected by the platform, and completely adequate for one app with a handful of settings. They break down at the second consumer (nothing is shared), the second environment (no label dimension), the first audit question (no history), and the first feature flag (no concept of one). App Configuration is what you graduate to — and the one setting that should stay in app settings is the endpoint of your store, because there is always exactly one bootstrap value you can't keep inside the store.

Versus a dedicated feature-flag platform: App Configuration's feature management is real (percentage and targeting filters, time windows, variants, telemetry) and thinner than a product built for experimentation. If experimentation is a discipline with metrics pipelines and significance testing, evaluate the specialists honestly.

Two meters, and neither one is per key:

• A flat daily charge per store on paid tiers — charged per replica, so two replicas cost roughly twice one store.
• Request overage beyond the tier's included daily allowance, charged per block of requests.

Keys and values are free; storage is capped by tier rather than metered. The Free tier has no daily charge and a hard daily request cap that throttles you for the rest of the UTC day when hit. ⚠️ Verify current figures against Azure pricing.

What bills while idle: the store's daily charge. It's small, so the surprise is never the idle cost — it's discovering you have eleven stores because every project created its own.

The variable line is driven entirely by client behaviour, and this is where a good answer goes. Requests = refresh interval × watched keys × instance count. A 5-second interval, 20 individually registered keys, 60 instances is roughly 20 million requests a day — all cheap 304 Not Modified responses, all billable. One sentinel key and a 30-second interval reduces the same fleet to about 170,000 a day. Identical functionality, two orders of magnitude apart.

1. The app resolves the endpoint from somewhere that isn't App Configuration — an env var or an application setting. There is always one bootstrap value outside the store.
2. The client acquires an Entra token for the App Configuration audience, via managed identity in Azure or developer credentials locally (DefaultAzureCredential walks its chain).
3. The SDK performs replica discovery from the origin endpoint and builds an ordered failover list. Hand-rolled HTTP clients skip this and therefore have no failover.
4. It issues one paginated request per configured (key filter, label filter) pair — typically two: no-label defaults, then the environment label. Not one request total.
5. The data plane authorises against Azure RBAC assignments scoped to the store — App Configuration Data Reader/Data Owner, not the control-plane roles.
6. Key Vault references are resolved by the client, not the store: it reads the reference content type, extracts the secret URI, and makes a second call to Key Vault with its own identity. A missing Key Vault Secrets User role fails startup here — and the error names Key Vault, which sends people to debug the wrong service.
7. Values are layered (later selects override earlier by key), prefixes trimmed, : mapped to nested objects, JSON parsed, and merged into the app's native configuration system.
8. Everything is cached in memory with per-key etags. From here, no request your app serves touches App Configuration.
9. A background refresh polls on the interval using conditional requests — 304 Not Modified when unchanged, which is cheap but still a billable request.

The shape to state explicitly: two round trips at startup, nothing per request, a cheap poll on a timer. Any deviation is a design bug that shows up as 429s.

There is nothing to scale — no instances, no throughput units, no partition key. The levers are the tier, the replica count, and, dominantly, client behaviour.

The ceiling you actually hit is the request quota, counted per store (and per replica). The Free tier's daily request cap is also per store. Distinct from that, "how many Free-tier stores may I have" is counted per subscription, per region — that's usually the first quota anyone meets. Storage capacity is per store; the size cap on a single key-value is per key-value (key + value

• label + metadata combined, single-digit kilobytes).

⚠️ All specific numbers vary by region and subscription type and change over time; verify against current Azure docs. The interview-relevant part is that every limit needs its scope stated — a number without a scope is useless in Azure.

Hard vs. soft: the per-key-value size cap is hard (a design needing a 200 KB value needs Blob Storage for the document and App Configuration for the URL). Store counts and some allowances are soft and raiseable. Tier limits aren't raiseable at all — the remedy is the next tier.

And the real answer to "we hit the ceiling" is almost never a quota increase: it's fixing the refresh configuration, stopping a per-request read, moving off Free, or adding a replica to split load across endpoints.

The tier gates storage capacity, request allowance, private endpoints, geo-replication, customer-managed keys, soft-delete retention, zone redundancy, and whether there's an SLA at all. There are four — Free, Developer, Standard, Premium — and ⚠️ the names and inclusions have changed and should be verified against current Azure docs.

Free is the trap, in two directions. It's right for a demo and wrong for anything a team depends on: a hard daily request cap (not a soft overage — you get 429s for the rest of the UTC day), no SLA, no private endpoints, no replication, and a small per-subscription-per-region store limit. The second half of the trap is subtler: teams pick Free for a "temporary" store, wire three services to it, and then find the tier change and the networking features they now need are awkward.

What moving costs you. Moving up is generally an in-place update. Moving down can fail outright if you exceed the lower tier's limits, and transitions involving Free are the least forgiving — you may be looking at a new store and a migration, which means a new globally unique name and a new endpoint for every consumer. ⚠️ Verify the current supported tier-change matrix before planning one.

The counter-point worth making: this is a cheap service. The top tier is a rounding error next to one App Service Plan. Don't under-provision to save pennies — but equally, don't buy geo-replication for a single-region workload, because each replica is another daily charge and another endpoint to fail over between.

1. Disable local authentication — local_auth_enabled = false / disableLocalAuth: true. This is the load-bearing step, for the reason in the next question.
2. Managed identity on every consumer, system- or user-assigned, with App Configuration Data Reader scoped to the store. Nothing else.
3. App Configuration Data Owner only for the configuration pipeline and platform engineers.
4. App Configuration Contributor for the infrastructure pipeline — which, once local auth is disabled, grants no data access at all.
5. Secrets in Key Vault, surfaced as Key Vault references. Remember the consumer needs a second role — Key Vault Secrets User on the vault — because the client resolves the reference.
6. Network isolation: private endpoint on the configurationStores sub-resource plus the privatelink.azconfig.io private DNS zone, and publicNetworkAccess: Disabled. Note there is no service endpoint for this service — it's public-with-IP-rules or private.
7. Azure Policy to enforce all of it: deny disableLocalAuth: false, deny free SKU in production subscriptions, deployIfNotExists for the diagnostic setting.

Where the built-in roles are too broad: Data Owner is read-write over the entire store — every label. There's no built-in "read-write but only under label dev" role, so store-per-environment is how you get environment-scoped permissions today. ⚠️ Key/label-scoped conditions have been improving; verify what the current implementation supports. For on-call flag flipping without full write access, the practical pattern is a small automation holding Data Owner and exposing only "toggle flag X", with humans permitted to run the automation rather than to write to the store.

Also worth saying: Entra-only auth is a stronger and cheaper control than network isolation for this service. Doing only the network half while leaving keys enabled is the worst combination — it looks like a control while a long-lived credential still exists and still works from anywhere inside the perimeter.

This is the flagship question on this service. Two endpoints, two RBAC models:

• Control plane — ARM. Creates and configures the store: tier, network rules, identity, encryption, replicas, soft-delete policy, and the access keys. Roles: App Configuration Contributor, Contributor, Owner, Reader. Audited in the activity log.
• Data plane — https://<store>.azconfig.io. Reads and writes key-values, feature flags, snapshots, revisions. Roles: App Configuration Data Reader, App Configuration Data Owner. Audited only via diagnostic settings, which are off by default.

The classic mistake, level one: granting an application Contributor on the store and expecting it to read configuration. It cannot. Being Owner of the resource gives you zero data-plane access. The inverse also surprises people: an identity with Data Reader and nothing else can read every setting and cannot see the resource in the portal's resource list.

The classic mistake, level two — and the answer that distinguishes a senior candidate: the separation leaks. Access keys are a control-plane property, and Microsoft.AppConfiguration/configurationStores/ListKeys/action is included in Contributor and App Configuration Contributor. So a Contributor can list the keys and then sign data-plane requests as a full data owner. The plane split is real for Entra-based access and completely bypassed by key-based access. The fix is one property — disableLocalAuth: true — and it should be enforced by policy, not by good intentions.

The consequence to name: until local auth is disabled, any resource-group Contributor is effectively a data owner on every store in the group, and your configuration is only as protected as your loosest Contributor assignment.

Most properties — tier (upward), network rules, identity, replicas, soft-delete retention — update in place. The ones that force replacement:

• name — it's the DNS label in the endpoint. A rename is a migration.
• location — there is no move operation; it's destroy-and-recreate.
• resource_group_name — same.
• Some tier transitions, particularly downward and anything involving Free. ⚠️ Verify the current matrix.

What it costs you is the part people miss: key-values are child resources of the store, not of your Terraform state. Replacing the store destroys every key-value, feature flag, snapshot, and revision in it. If your configuration is all in git and applied by a pipeline, that's an inconvenience. If it was typed into the portal, that's data loss with no undo. Consumers also get a new endpoint and a new identity principal ID, so every role assignment and every consumer's bootstrap setting has to change.

Two Azure-specific traps compound it:

• Soft delete. The destroyed store is recoverable, which means it still holds its globally unique name, so the recreate fails with a name-unavailable error that looks nothing like the cause. The azurerm provider has a dedicated features { app_configuration { purge_soft_delete_on_destroy, recover_soft_deleted } } block for exactly this — and where purge protection is on, you cannot purge early at all and the name is held for the full retention window.
• Resource locks. A CanNotDelete or ReadOnly lock makes the apply fail with something that reads exactly like a missing role assignment. Check az lock list before debugging RBAC.

The operational answer: read the plan. # forces replacement on an App Configuration store is never routine.

Because azurerm_app_configuration_key writes through the data plane, and the identity running apply has no data-plane permission just because it created the resource. It needs App Configuration Data Owner.

Two distinct problems, and a complete answer names both:

1. Missing dependency edge. The key resource references the store, not the role assignment, so Terraform has no reason to order them. You get 403 on the first apply and success on the second — "flaky pipeline" that is actually a missing depends_on.

2. RBAC propagation is eventually consistent. Even with the edge, the assignment may not be live when the write happens. A time_sleep helps; it's a workaround, not a fix.

The sturdy answers, in increasing order of maturity:

• Grant the pipeline identity App Configuration Data Owner once, out of band, at the resource group or subscription scope — so it is never part of the plan that depends on it.
• Split the pipelines. Terraform owns the store, its network, RBAC, and the existence of flags; a separate configuration pipeline owns the contents via az appconfig kv import from a reviewed file in git. Different cadence, different tool, different blast radius. This is the structurally right answer for this service, because the thing you deploy most often is the contents, not the resource.
• Use Bicep for the key-values — ARM writes keyValues as child resources through the control plane, so the data-plane RBAC ordering problem doesn't exist. For a config-only deployment that's a genuine simplification, and it's a good moment to show you'd pick the non-default tool when it fits.

One more thing a strong answer adds: don't let Terraform own live feature-flag state. Put lifecycle { ignore_changes = [enabled, ...] } on the flag resources. Otherwise the first on-call kill-switch flip becomes drift, and the next apply silently re-enables the thing that broke production.

Confirm and scope it first. ThrottledHttpRequestCount and HttpIncomingRequestCount metrics on the store, bucketed at 15 minutes, tell you when it started — which usually correlates with a deployment. Then find the caller from the HttpRequest diagnostic logs:

AppConfigurationHttpRequest
| where TimeGenerated > ago(1d)
| summarize Requests = count(), Throttled = countif(StatusCode == 429)
    by ClientIPAddress
| order by Requests desc

⚠️ Table and column names vary with the collection mode; check your workspace schema.

Then work the causes in likelihood order:

1. Refresh interval too aggressive, multiplied by instance count. Requests = interval × watched keys × instances. A 5-second interval, 20 registered keys, 60 instances is ~20 million requests a day, all cheap 304s, all billable.
2. Individual keys registered for refresh instead of a sentinel. Register one sentinel with refreshAll: true — 20× fewer polls, and it fixes torn reads as a bonus.
3. Reading on the request path. Someone constructing a client or calling the store per HTTP request rather than reading cached IConfiguration. This is the one that scales with traffic and so appears suddenly.
4. A client per operation instead of one provider per process — re-does discovery and re-fetches every time.
5. A Free-tier store in something that isn't a demo, hitting the hard daily cap mid-afternoon. The giveaway is throttling that starts at a consistent time of day and clears at UTC midnight.
6. A CI job importing key-values in a loop of az appconfig kv set instead of one batched az appconfig kv import.

Fix, then prevent. Set the interval to tens of seconds, one sentinel, verify no per-request reads, move off Free if applicable. Then: an alert on sustained throttled requests > 0, and — because this recurs every time a new service onboards — a documented default refresh configuration in your service template. Note explicitly that raising the quota is almost never the right fix here.

Topology: one store per environment, not one store with environment labels. The reason isn't aesthetic — labels are not an access-control boundary. Built-in data-plane roles scope to the store, so you cannot grant a team read on dev without granting prod. One store also means one blast radius: one --strict import against the wrong label, one network change, one throttling incident, and all three environments feel it. Separate resource groups per environment; separate subscriptions if the platform already works that way, since the subscription is Azure's natural quota and policy boundary.

Within each store, use labels for what they're good at: the service name as a key prefix (svc-orders:*), and labels for release version or regional override. Shared platform settings go under the null label and are layered underneath the service's selects, so a service reads Select("shared:*", null) then Select("svc-orders:*", region).

Tier: paid tier everywhere, Premium for prod (private endpoints, geo-replication, zone redundancy). Deny the free SKU in production subscriptions by policy. ⚠️ Verify current tier inclusions.

Two regions: geo-replicate the prod store to the second region and let the client libraries do discovery and failover. Accept eventual cross-replica consistency and never write in one region to validate in another. Budget for it: each replica bills as a store.

Compliance / no public access: private endpoint on the configurationStores sub-resource, private DNS zone privatelink.azconfig.io linked to every VNet that needs it (including peered spokes — a missing DNS link is the most common private-endpoint failure and presents as a timeout, not a 403), and publicNetworkAccess: Disabled on prod only. Keep dev public so laptops and hosted runners work, and say out loud that this is a deliberate asymmetry.

The consequence to plan for, not discover: hosted CI runners can't reach prod. You need a self-hosted runner, a Container Apps job, or a managed DevOps pool on the VNet — and portal access needs a network path too.

Security baseline, enforced by policy at the management-group scope: disableLocalAuth: true (deny), diagnostic settings to the environment workspace (deployIfNotExists), required Environment and owner tags, free SKU denied in prod. Every service gets a user-assigned managed identity with App Configuration Data Reader on its environment's store, and Key Vault Secrets User on the vault if it uses references.

Delivery: two pipelines. Terraform owns stores, network, RBAC, and flag existence — rare changes, reviewers required. A configuration pipeline owns contents via az appconfig kv import --strict from config/<env>/<service>.yaml in git, with --dry-run on the PR and a sentinel bump after apply. One label, one owning file, one pipeline — --strict will happily delete another team's keys otherwise. Lighter approval gates for the config pipeline than the infra one, or you've recreated the deployment latency you adopted this to remove.

Client contract, standardised in the service template: one sentinel per service with refreshAll, a 30-second interval, unversioned Key Vault reference URIs, optional: true with a baked -in appsettings.json fallback, and emitted telemetry for "which config version am I on" and "when did I last refresh successfully". At 30 services, the client contract matters more than the topology.

Rollback: snapshots per release for the settings that should move with a deploy, revisions and point-in-time restore for accidental edits, and git as the real backup. Plus a scheduled az appconfig kv export to blob, because revision retention expires.

Identify which of four things broke, because the undo differs:

az appconfig revision list -n "$STORE" --auth-mode login --key "Api:Timeout" --label prod -o table
az appconfig kv restore   -n "$STORE" --auth-mode login --label prod --datetime "2026-07-30T09:00:00Z" --yes

Then bump the sentinel. This is the step that gets forgotten and it's why half of all rollbacks on this service appear not to work — the restore changed the store, not the in-memory caches. Nothing propagates until clients notice, and they notice via the sentinel.

Blast-radius reasoning, and the point that matters most: a broken store does not take down running applications, because values are cached in memory. It takes down every process that tries to start. So the profile is: nothing happens for hours, then your next deployment, next autoscale event, or next pod restart fails, apparently unrelated to a config store nobody has touched since Tuesday. That delayed coupling is why the store's availability posture must match the most critical thing that reads it — and it's why the store being "fine" is not evidence that you're fine.

Also assess: which labels did the change touch (one environment or several — the argument for store-per-environment), which services select those keys, and whether any of them cached a torn set because refresh was registered per-key instead of via a sentinel.

If it had been an ARM/Bicep deployment in complete mode: complete mode deletes every resource in the resource group that the template does not declare. For this service that has a specific and nasty edge: if the template declares the store but not its keyValues children, a complete-mode deployment can remove key-values written by your configuration pipeline — silently, as a side effect of an infrastructure deploy. Complete mode and the two-pipeline split are in direct contradiction.

The remedy: stay on incremental (the default) for anything sharing a resource group with pipeline-managed data, and use deployment stacks if you want managed deletion semantics without complete mode's bluntness. Always run az deployment group what-if first — it shows the deletions. And if it has already happened, the recovery is point-in-time restore from revisions plus a re-import from git, which is exactly why git is the real backup.

First, a framing point that matters: for this service, hand-editing is the feature, not the bug. The whole value proposition is changing configuration without a deployment. So the goal isn't zero drift — it's knowing which surfaces must never drift and watching only those.

How you find out — three sources, and you need all three:

1. az appconfig revision list tells you what the value was and when it changed.
2. The HttpRequest/Audit diagnostic logs tell you which identity wrote it — and only if the diagnostic setting existed before the change. There is no retroactive fix; this is the single most common Azure observability gap.
3. Azure Policy compliance state catches drift out of standard — local auth re-enabled, a diagnostic setting removed, a tag missing — including on stores nobody told you about.

For control-plane changes specifically, the activity log is retained by default and will name the caller.

Getting back to clean:

• If the value was wrong: re-run the configuration pipeline's --strict import from git. The store is a projection of the repository; making it match is one command, and --strict deletes the hand-added keys that shouldn't exist.
• If the value was right and the file is stale — which is the common case, because the person editing at 3 a.m. was correct — backport it to git and let the pipeline re-apply it. Don't revert a correct fix in the name of tidiness; that's how people learn to distrust the pipeline and stop using it.
• For infrastructure drift: terraform plan, read it, and either apply to reconcile or terraform import if someone created something real out of band.
• If Terraform is fighting a live flag state: that's a design error on your side, not the engineer's. Add lifecycle { ignore_changes = [enabled, percentage_filter_value, targeting_filter] } so Terraform owns the flag's existence and humans own whether it's on.

Prevent the recurrence by making ownership explicit rather than by locking people out: one label, one owning file, one pipeline; --strict only where the pipeline is the sole writer; a ReadOnly resource lock on the store's infrastructure (which notably does not block the data plane — another argument for the two-pipeline split); and Event Grid → a Teams message on every production key-value write, so a hand edit is visible within seconds instead of at the next drift run.