8. Interview Questions

Azure VMs are Infrastructure as a Service: a virtualised server — CPU, memory, disks, a network interface — running an OS you have root or Administrator access to, billed by the second while it holds hardware.

The problem they kill is capacity as a capital decision. Before the cloud, adding a server meant a purchase order, a six-week lead time, and a three-year depreciation schedule based on a traffic guess made a year early. A VM turns that into an API call you can undo in a minute.

Two second-order effects matter more than the cost: infrastructure becomes a text file you can review in a pull request and recreate identically, and failure becomes cheap — a dead VM is replaced by re-running the template rather than by a hardware incident with a lead time.

The trade-off is the IaaS bargain. Azure owns the datacentre, the host, the hypervisor, and host patching. You own the guest OS, its patching and hardening, the application, the data, and the network rules. Every other Azure compute service moves that line upward, which is why they're cheaper operationally and why a VM should be the choice you make when something rules the others out — licensing, drivers, kernel access, lift-and-shift — not the choice you default to.

Two hierarchies, and both matter.

The scope hierarchy: tenant → management group → subscription → resource group → resource. The subscription is the billing and quota boundary; the resource group is the lifecycle boundary — things in one should share a fate, because az group delete removes all of them.

The resource constellation: "a VM" is not one resource. Creating one produces at least a Microsoft.Compute/virtualMachines, one or more Microsoft.Compute/disks for the OS and data disks, a Microsoft.Network/networkInterfaces, usually a Microsoft.Network/publicIPAddresses, and usually a Microsoft.Network/networkSecurityGroups. Each has its own ARM resource ID, its own lifecycle, and its own line on the bill.

Almost everything surprising about Azure VMs follows from that second point: orphaned disks that keep billing after the VM is deleted, an NSG that survives a rebuild and blocks the new VM, a public IP that can't be released because a NIC still references it. AWS hides most of this; Azure makes you hold it, which is more work up front and considerably less mystery later.

A managed disk is backed by Azure Storage with three synchronous replicas within the region (LRS), quorum-committed before a write is acknowledged. ZRS disk options spread those replicas across availability zones in supported regions. Encryption at rest with platform-managed keys is always on.

What that protects against: disk hardware failure, host failure, and — with ZRS — a zone failure.

What it does not protect against: deletion, corruption, ransomware, or a bad deploy. The replication is synchronous and faithful; it replicates your rm -rf instantly and in triplicate. Replication is not backup, and this distinction is the single most common misconception about cloud storage generally.

Backup is a separate decision: Azure Backup into a Recovery Services vault, with soft delete enabled on the vault so a compromised admin account can't destroy the recovery points. And an untested restore is a belief rather than a capability — the restore drill is what turns it into an RTO number you can actually quote.

The honest rule: a VM is right when something specific rules the others out.

Choose a VM for a lift-and-shift migration where the app has an installer and nobody will touch it; for licensed commercial software certified against a specific OS build (SAP, Oracle, ISV appliances); for anything needing kernel or driver control (GPU drivers, custom kernel modules); for long-running high-utilisation compute where a reservation or savings plan beats per-request pricing; for host-level compliance requirements; and for self-hosted infrastructure like build agents and domain controllers.

Choose something else when it's a plain web app or API (App Service), a container workload (Container Apps, or AKS if you actually want the Kubernetes API), or short bursty event-driven work (Functions).

Two clarifications worth adding: AKS is not an alternative to understanding VMs — its node pools are VM Scale Sets, so sizing, disks, zones, and quota behave exactly as described here. And "use a VM and install Docker" is almost never right — it looks cheaper than Container Apps until you count the node patching, registry auth, restart policy, and monitoring you now own.

The strongest anti-pattern: choosing a VM because it's familiar. That's a permanent operational tax paid for nothing. And a genuine architectural criterion people skip — if no team will own guest patching, the correct decision is a PaaS service, because an unpatched internet-facing VM is the most common route into a compromised Azure subscription.

Five meters, and only one of them stops when you shut the machine down:

• Compute, per second, while the VM is allocated.
• Managed disks, per provisioned GiB per month, whether the VM runs or not — and after the VM is deleted, if the disk was orphaned.
• Public IP, per hour for Standard SKU, attached or not.
• Egress, per GB out of the region.
• OS licence, included in the Windows/RHEL/SLES rate unless you apply Azure Hybrid Benefit.

The critical distinction: stopping is not deallocating. A shutdown -h now inside the guest, or az vm stop, leaves the VM allocated — Azure is still holding CPU and RAM for you and still billing the compute meter. Only deallocation releases the hardware and stops that charge. The portal's Stop button deallocates; az vm stop does not; Stop-AzVM in PowerShell does by default. Three tools, two behaviours — which is exactly how the mistake happens.

The three cost traps in order of money wasted: VMs stopped-but-allocated, orphaned disks after a VM delete without the delete options set, and dev VMs allocated 168 hours a week for 40 hours of use. The last one is fixed by auto-shutdown in about five minutes.

1. The client authenticates to Microsoft Entra ID and gets a token. Portal, az, Terraform, and Bicep all do exactly this — they're all clients of the same REST API.
2. The request hits ARM at management.azure.com as a PUT on the resource ID.
3. ARM authorises it against Azure RBAC at that scope, then evaluates Azure Policy. A deny policy stops it here, before anything is created — which is why policy failures look different from permission failures in the activity log.
4. ARM dispatches to the resource providers: Microsoft.Network creates the NIC and public IP, Microsoft.Compute creates the disks from the image, and the VM resource is created last because it references the others. ARM derives that ordering from the references, which is why templates rarely need explicit depends_on.
5. The Compute RP asks the fabric controller for a placement — a physical host in the right region and zone, with the right hardware for the size, and free capacity. This is where allocation failures happen.
6. The host provisions: OS disk attached from managed storage, virtual NIC plugged into the software-defined network, hypervisor boots the guest.
7. The guest agent starts, reports Ready, and applies hostname, admin user, SSH keys, and cloud-init.
8. Extensions install in sequence, each a separate child-resource deployment that can fail independently.

Two consequences worth stating: provisioning state is not application state — ARM reporting Succeeded means the resource exists, not that your app is up, and a VM can be Succeeded with a failed extension and therefore no monitoring. And a failed create leaves resources behind — if step 6 fails, the NIC, disks, and public IP from step 4 already exist and still bill.

Vertically, by changing the size. Easy within a hardware family; crossing families or generations often requires the VM to be deallocated so the fabric can place it on different hardware. Always costs downtime. The ceiling is the largest size available in that region, and it's hard.

Horizontally, with a scale set. Autoscale rules react to metrics or a schedule.

The ceilings, and this is the part with the Azure-specific twist — a number without a scope is useless here:

• Total regional vCPUs: per subscription, per region.
• Per-family vCPUs (e.g. the DSv5 family): per subscription, per region, separately from the total. Both must have room, and GPU families are often zero by default.
• Spot vCPUs: a third, separate quota, same scope.
• Instances per scale set: per scale set.
• VMs per availability set: per availability set.

The failure mode people meet: autoscale silently stops at the regional vCPU quota. There's no obvious error — the system just doesn't grow — so if your autoscale max is 20 and your quota allows 12, you have a ceiling of 12 and no alert about it.

Critically, quota is not capacity. Quota is an accounting limit on your subscription and is usually raised self-service in minutes. An allocation failure means Azure physically doesn't have that hardware in that region or zone right now, and the fix is a different zone, a different size, a different region, or a capacity reservation. The errors read similarly; the remedies have nothing in common.

Also worth naming: autoscale reacts to trends, not spikes. Metrics lag minutes and boot takes minutes more, so the instance floor — not the scale-out rule — is what absorbs a spike.

An availability set spreads VMs across fault domains (separate racks, power, network) and update domains (separate host-patching batches) within a single datacentre. It protects against rack failure and against maintenance rebooting everything at once. It does not protect against a datacentre outage.

Availability zones are physically separate datacentres within the region, with independent power, cooling, and networking. Spreading instances across zones protects against a datacentre-level failure, and carries the highest VM SLA tier.

Zones are strictly stronger wherever they're available. The costs of zones are inter-zone data transfer charges and slightly higher latency between instances — usually irrelevant, occasionally decisive for very chatty tiers, which is what proximity placement groups exist for (at the price of more allocation failures and giving up zone spread).

What moving costs you: a rebuild. Zone assignment and availability set membership are both fixed at creation. A VM cannot be in both, and neither can be added, removed, or changed afterwards. So "we'll add zones later" means destroying and recreating every VM, which is why this belongs in the design conversation rather than the deployment one.

The SLA tiering is worth knowing as a shape: highest for multi-zone, middle for an availability set, lowest for a single VM — and that single-VM SLA is conditional on using premium or ultra disks for every disk. A single VM on standard disks has no availability commitment at all, which regularly surprises people who chose Standard SSD to save money on a production box. ⚠️ The exact percentages move; verify against current docs.

Identity for the VM: a managed identity — system-assigned or user-assigned — plus narrowly scoped Azure RBAC role assignments. The VM fetches tokens from IMDS at 169.254.169.254; every Azure SDK's DefaultAzureCredential uses this automatically, so the same code runs on a laptop under az login and on the VM under the managed identity with no branching. That removes every connection string, storage key, and client secret from config.

Identity for humans: Entra login via the extension, plus Virtual Machine User Login by default and Virtual Machine Administrator Login granted just-in-time through Privileged Identity Management. This inherits MFA and Conditional Access, and access is revoked centrally at offboarding — unlike a shared SSH key in a password manager, which is unrevokable and survives everyone leaving.

Network: no public IP on the VM, no inbound 22 or 3389 from the internet at all — not "restricted to the office IP", none. Access via Azure Bastion; egress via NAT Gateway; NSGs on the subnet with Application Security Groups so rules read asg-web → asg-app:8080; private endpoints for PaaS dependencies with the matching private DNS zone linked to the VNet.

Secrets: Key Vault, reached with the managed identity and the Key Vault Secrets User data-plane role. For TLS certificates, the Key Vault VM extension refreshes them into the guest store so rotation needs no deployment.

Guest: Trusted Launch on, encryption at host on, Azure Update Manager owning patching, Defender for Servers for EDR, and a CIS-hardened golden image from a Compute Gallery so hardening is a build step rather than a recurring task.

Control-plane RBAC — the part people miss: scope Virtual Machine Contributor to a resource group at most, because (see the next question) it is effectively root on the guest.

The control plane is ARM. It governs the VM as a resource: create, resize, start, stop, deallocate, attach disks, install extensions. Authorisation is Azure RBAC — Virtual Machine Contributor, Contributor, Owner.

The data plane is the guest OS. It governs the filesystem, processes, and application. Authorisation is the OS's own — SSH keys, local Administrator, domain credentials — or, with Entra login enabled, the Virtual Machine Administrator Login and Virtual Machine User Login RBAC roles.

The classic mistake is assuming these are separated on a VM the way they are everywhere else.

For storage, Key Vault, and Cosmos DB, the separation is real and strict: Owner on a storage account doesn't let you read a blob, and Key Vault Contributor doesn't let you read a secret. People learn that lesson and then apply the instinct to VMs — where it is exactly backwards.

On a VM, control-plane access is data-plane access. Anyone who can install an extension or invoke Run Command executes arbitrary code inside the guest as root or SYSTEM — with no SSH key, no guest credential, and no inbound network path at all.

Virtual Machine Contributor sounds like an infrastructure-only role. It is remote code execution on every VM in its scope. Three practical consequences: scope it to a resource group and never subscription-wide as a convenience; audit extension and runCommand operations in the activity log, because they're the quietest route to a compromised guest; and use Azure Policy or deny assignments to restrict which extension types can be installed in a regulated environment.

If an interviewer asks how to give someone the ability to restart VMs without giving them the guest, the answer is a custom role with read, start, restart, and deallocate — and specifically without .../extensions/write, .../runCommand/action, and .../write.

Replacement means the VM is destroyed and recreated — everything not on a persistent disk is gone, the temp disk is wiped, and any dynamic IP changes.

Forces replacement:

• Changing the OS disk image reference.
• Changing the admin username.
• Changing availability zone or availability set membership — both are fixed at creation.
• Changing the subnet on an existing NIC (replaces the NIC, and therefore disrupts the VM).
• Some changes to os_disk.storage_account_type, depending on provider version. ⚠️ Verify against your own plan output rather than trusting a general claim.

Updates in place, but with a restart:

• VM size — and if the target hardware isn't in the current cluster, the VM must be deallocated first, which means the temp disk is lost and a fresh placement (and therefore a possible allocation failure) is involved.

Updates in place, no disruption: tags, NSG rules, data disk attach/detach, extensions.

Practically: Terraform annotates these with # forces replacement in the plan, and that string is the most important thing to grep for in a VM plan. Posting terraform show tfplan as a plan.txt artifact on the pull request is what makes it reviewable — a reviewer who sees "1 to add, 1 to destroy" on a stateful VM should block the merge.

The mitigation is architectural rather than procedural: keep state off the VM. If the VM is stateless and the data lives on a separate managed disk, in a database, or in Blob Storage, then replacement is a rolling upgrade rather than an incident. That's the whole argument for immutable infrastructure with golden images.

Low CPU with poor performance points away from compute, so work through the other ceilings in order.

First, the two disk metrics — and the distinction between them is the whole answer. Compare Data Disk IOPS Consumed Percentage against VM Uncached IOPS Consumed Percentage:

• If the disk metric is pegged, the disk SKU is the constraint. A Premium SSD's performance is tied to its size tier, so the classic case is a 128 GiB P10 doing work that needs P30 IOPS. The fix is a bigger disk, or Premium SSD v2 which decouples IOPS from capacity.
• If the VM metric is pegged, the VM size is the constraint. A bigger disk changes nothing — you need a larger size, because every size has its own cached and uncached IOPS caps.

Getting these the wrong way round means spending money on a disk upgrade that doesn't help, which is common enough to be worth naming explicitly.

Second, memory. Not visible without the Azure Monitor Agent, which is exactly why so many VMs have no memory telemetry at all. A guest swapping presents as low CPU and terrible latency.

Third, the B-series trap. If the size has a B, check whether CPU credits are exhausted. A depleted burstable VM throttles to a baseline that can be a fraction of a vCPU, and the symptom looks like a mysterious application slowdown, not a quota error. This is a very common cause of "it was fine for a month and then got slow."

Fourth, network. Check whether the size's bandwidth cap is being hit, and whether accelerated networking is enabled — it's free and materially reduces latency and jitter.

Fifth, caching. Confirm host caching matches the workload: ReadOnly for read-heavy data disks, None for database log disks (ReadWrite there can violate write-ordering guarantees the database depends on).

Sixth, inside the guest. Now it's an ordinary Linux/Windows investigation — iostat, vmstat, application logs — plus the VM Insights dependency map to see whether the real latency is in a downstream call rather than on this host.

Resilience:

• A Flexible VM Scale Set across three availability zones, minimum three instances so a zone loss leaves capacity. Zones, not an availability set — sets only protect against rack failure.
• Automatic instance repair enabled with a grace period, so an unhealthy instance is replaced without a human.
• Zone-redundant frontend: Application Gateway (WAF SKU) or Front Door, both zone-redundant.
• State off the VMs entirely — a zone-redundant database, Blob Storage, Azure Files. If instances are disposable, replacement is routine rather than an incident.
• Azure Backup for anything stateful, with soft delete on the vault, and a tested restore.
• For regional failure: Azure Site Recovery with a recovery plan and non-disruptive test failovers, or active/active across regions behind Front Door if the RTO justifies the roughly doubled cost.

Scale to 10x:

• Autoscale on CPU and a queue-depth or request-count metric, scaling out fast and in slow — asymmetric windows (5 min out, 10 min in) and step sizes (2 out, 1 in) to prevent flapping.
• Set the floor high enough to absorb the spike, because metrics lag minutes and boot takes minutes more. Autoscale answers trends; the floor answers spikes.
• Check regional and per-family vCPU quota against the autoscale maximum, and alert on quota headroom. Autoscale hitting the quota ceiling is silent — this is the failure people don't design for.
• Pre-baked golden images from a Compute Gallery so boot is fast and deterministic; long cloud-init or Ansible runs at boot destroy your scale-out time.
• Ephemeral OS disks for stateless instances — faster boot and reimage, free, and the fact that they're lost on deallocate doesn't matter if there's no state.
• Termination notification enabled with connection draining, so scale-in doesn't drop live requests.

The honest caveat: if the workload is stateless HTTP, this design is a well-engineered version of something Container Apps or App Service gives you with far less to own. Say so. The VM version is right when something specific — licensing, drivers, a legacy runtime — rules those out, and a good interview answer names that condition rather than assuming it.

First, establish what actually changed. terraform state list and the ARM deployment history for the resource group; the activity log tells you what succeeded and what failed, and in what order. ARM's dependency ordering means a partial failure usually leaves the network resources created and the compute resource missing or in a Failed state.

Rollback options, in increasing pain:

1. Re-apply the previous commit. The default. Revert in git, let the pipeline plan and apply. Read the plan — the question is always whether the revert updates in place or forces replacement, and on a stateful VM those are very different events.
2. Roll back the image version. With a scale set on a pinned image version, changing the version back and letting the rolling upgrade run is a genuine fast rollback. This is the strongest argument for the immutable-image approach.
3. Redeploy the previous ARM deployment from the resource group's deployment history — useful when the change wasn't made through Terraform.
4. Restore from Azure Backup. Slowest, and the only option that recovers data.

Blast radius: what forces replacement — image reference, admin username, zone or availability set membership, NIC subnet. On a stateful VM, replacement means data loss for anything not on a persistent disk.

The complete-mode question, which is what's really being asked:

az deployment group create defaults to incremental mode — resources in the template are created or updated, and anything else in the resource group is left alone. Complete mode (--mode Complete) deletes every resource in the resource group that isn't in the template.

So a complete-mode redeploy of a VM template into a resource group that also holds, say, a storage account someone created by hand, a data disk added later, or another team's resources will delete all of them. No extra confirmation beyond the standard prompt. Always run az deployment group what-if first and read the Delete section specifically — it's the only warning you get.

Two Azure-specific traps that turn a rollback into an outage:

• Soft delete and purge protection. A destroyed Key Vault holding your disk-encryption key is soft-deleted, still holds its name, and blocks recreating a vault with that name. With purge protection on, it can't be purged at all until retention expires. A destroy-then-apply cycle fails on the vault with an error that doesn't obviously mention soft delete.
• Resource locks. CanNotDelete and ReadOnly locks aren't visible in a Terraform plan, and an apply that hits one fails with a message that reads like a permissions problem — sending people to RBAC for an hour. az lock list -g <rg> should be step one of any failed-apply investigation.

Prevention, which is the better half of the answer: separate resource groups per environment and per lifecycle, never complete mode without what-if, prevent_deletion_if_contains_resources = true in the Terraform features {} block, a CanNotDelete lock on stateful production resources, and the plan file posted to the pull request so replacements are visible before merge.

Finding out. Three complementary mechanisms, and each answers a different question:

• A scheduled terraform plan -detailed-exitcode in CI, nightly. Exit code 2 means drift. This answers what differs, and it's the primary mechanism.

• The activity log answers who and when, which terraform plan cannot:

  AzureActivity
  | where TimeGenerated > ago(7d)
  | where OperationNameValue has "Microsoft.Compute/virtualMachines"
  | where OperationNameValue has_any ("write", "restart", "deallocate", "runCommand", "extensions")
  | project TimeGenerated, Caller, OperationNameValue, ActivityStatusValue, _ResourceId
  | order by TimeGenerated desc

• Azure Policy compliance state catches drift Terraform can't see, because it evaluates every resource including ones created entirely outside the state file. An "allowed VM SKUs" policy would have flagged this resize immediately — or, set to Deny, prevented it.

Getting back to clean — the decision, which should be agreed in advance rather than during the incident:

1. If the manual change was wrong, re-apply. Terraform reverts it. Read the plan first: a size change is an update-with-restart, so the revert is another restart, and that needs a window.
2. If the manual change was right — and at 3 a.m. during an incident it often is — bring it into code. Update the vm_size variable, open a pull request, plan, apply. The plan should then be empty. This is the more common correct outcome, and treating it as a violation rather than as information is how teams end up with an emergency-change process nobody follows.
3. Either way, terraform plan should show no changes by end of day. Tolerated drift compounds until nobody trusts plan output, and at that point you don't have infrastructure as code — you have a text file that resembles your infrastructure.

For resources created by hand that should be managed, terraform import (or an import block) brings them into state without recreating them. Verify with a plan showing no changes afterwards.

Reducing recurrence — the systemic answer, which is what separates a senior response:

• Azure Policy with allowed SKUs, set to Deny in prod, so the portal path simply fails.
• Scope Virtual Machine Contributor narrowly, and put production elevation behind PIM so 3 a.m. changes require an activation with a justification, which is itself an audit record.
• Make the paved path faster than the portal — a pipeline that can ship an emergency size change in ten minutes with an approval gate. If the correct route is slower than the portal during an incident, people will use the portal, and no amount of policy documentation changes that. This is a process design problem more than a technical one.