2. Core Concepts

Fifteen or so terms carry the whole platform. Learn them in this order and the studio stops looking like a menu of unrelated features.

[Image Prompt: 2D minimalistic labelled hierarchy diagram of Azure Machine Learning, showing a resource group containing a workspace with its four dependent resources — storage account, key vault, container registry, application insights — attached at the side, and inside the workspace three columns: compute targets, versioned assets comprising data, environments, components and models, and endpoints containing deployments, flat design, clean vector art style, white background]

The workspace

Analogy: a project folder that also owns a budget, a guest list, and a filing cabinet.

Technically: an ARM resource of type Microsoft.MachineLearningServices/workspaces, regional, in a resource group. It is the security boundary, the naming scope for every asset inside it, and the anchor for the data-plane endpoint your tools talk to. Everything in this topic lives in a workspace.

The workspace has a kind, and the value changes what the resource means:

kind	What it is
Default	A normal Azure ML workspace. This topic.
Hub	An Azure AI Foundry hub (classic architecture) — same resource type, different product.
Project	A Foundry project beneath a hub.
FeatureStore	A managed feature store — a workspace specialised for feature sets and retrieval.

That one field is why Foundry and Azure ML keep colliding in ARM listings, Terraform state, and policy assignments. If a Microsoft.MachineLearningServices/workspaces resource appears in a subscription you thought had no ML, check its kind before deleting anything.

The dependent resources

A workspace does not stand alone. Four Azure resources sit beside it:

Resource	Role	Notes
Storage account	The default datastore — job outputs, snapshots, logs, uploaded data	Mandatory. Also where the workspaceblobstore and workspacefilestore datastores point
Key Vault	Secrets: datastore credentials, connection secrets	Conventionally required; treat it as required
Container Registry (ACR)	Stores environment images built for jobs and deployments	Often created lazily — the first time an image needs building. Which is why it sometimes appears in your resource group days after you created the workspace
Application Insights	Telemetry for online endpoints and pipelines	Conventionally attached; endpoint logs land here

⚠️ Newer API versions have relaxed which of these are strictly required at creation. Verify against current Azure docs before writing a Terraform module that omits one.

The practical consequence: create these yourself, with your names, in your Terraform. If you let the portal create them, you get mlws1234567890 and you will live with it. And note the deletion asymmetry: deleting a workspace does not delete any of the four. See Deployment.

Compute — the axis that is really the SKU

Azure ML's workspace sku field says Basic and tells you nothing. Compute is the tier axis. Four kinds, and choosing wrong is the most expensive mistake on the platform.

Compute type	What it is	Bills	Use it for	The trap
Compute Instance	A single-user managed VM with Jupyter, VS Code server, and the SDK pre-installed	Per hour while running, regardless of activity	Authoring, debugging, exploration	Leaving it on. Always set idle shutdown. This is the #1 line on surprise bills
Compute Cluster (AmlCompute)	A managed, autoscaling node pool that runs jobs	Per node-hour, only for nodes that exist	Training, pipelines, batch scoring	min_instances > 0. Set it to 0 unless you have a measured reason
Serverless compute	Azure ML picks and manages the nodes; you specify size and count on the job	Per node-hour of the job	Teams who don't want to administer clusters at all	Less control over networking and image caching; check it supports your isolation requirements
Kubernetes compute	Your own AKS or Arc-enabled cluster, registered via the Azure ML extension	You pay for the cluster, always	Existing K8s investment, on-prem/edge, tight control	You now own a Kubernetes cluster's operational life. Choose deliberately

Two sub-dials cut across all of them:

• Dedicated vs. Spot (low-priority). Spot nodes cost dramatically less and can be evicted mid-job. Perfect for hyperparameter sweeps and anything checkpointed; wrong for a single long job with no checkpointing. Quota is tracked separately for the two. ⚠️ Discount varies by region and family — verify current pricing.
• VM family and size. Standard_DS3_v2 for CPU work, the NC/ND families for GPU. Quota is granted per family, per subscription, per region, and GPU families commonly start at zero.

Datastores and data assets

Datastore — a registered connection to a storage service (Blob, ADLS Gen2, File, and a few others), holding the auth method so your code never holds a key. Two auth modes matter:

• Credential-based — an account key or SAS in Key Vault. Simple, and a standing secret.
• Identity-based — the job's managed identity (or the submitting user) is authorised on the storage account directly with Storage Blob Data Reader/Contributor. Preferred. The catch that bites everyone: the workspace identity having access does not mean the compute's identity has access. Grant the role to whichever identity actually runs the job.

Data asset — a named, versioned pointer to data, of one of three types:

Type	Points at	Use when
uri_file	A single file	One CSV, one parquet file
uri_folder	A directory	An image dataset, a partitioned folder
mltable	A folder containing an MLTable spec that describes how to load it	Schema, delimiters, subsetting, and time-series partitions matter

A data asset is a pointer, not a copy — versioning it does not snapshot the underlying bytes. If the blob changes, my-data:3 now means something different. If you need genuine immutability, write to immutable paths (date-partitioned prefixes) or turn on blob versioning/immutability policies. This is the most commonly misunderstood thing on the page.

Environments

Analogy: a Dockerfile with a version number and a memory.

Technically: a versioned asset describing the runtime — a base image plus a conda or pip specification, or a full custom Docker context. Azure ML materialises it as an image in the workspace's ACR the first time it's needed and caches it thereafter (which is why the first job on a new environment is slow and the rest are not).

Curated environments are Microsoft-maintained, prebuilt, and already in the registry — fast to start, and you don't control what's in them. Custom environments are yours. The rule: prototype on curated, pin a custom one before anything reaches staging.

Jobs

A job is a unit of tracked execution. The types you will meet:

Job type	What it does
Command	Runs one command in one environment on one compute target. The workhorse
Sweep	Runs a command many times over a hyperparameter search space, with an early-termination policy
Pipeline	A DAG of steps, each step a component. Steps can run on different computes and can be individually cached
AutoML	Azure ML searches models and hyperparameters for you across tabular, vision, and NLP tasks
Spark	A Spark job, on serverless Spark or an attached Synapse pool

Jobs live inside an experiment — which is just a name you group runs under, nothing more. Metrics, parameters, and artifacts are logged through MLflow: the workspace is an MLflow tracking server, so mlflow.log_metric("auc", 0.91) works with no Azure imports. Set the tracking URI to the workspace and your existing MLflow code runs unchanged.

Components and pipelines

A component is a job step packaged as a reusable, versioned asset: a named interface of typed inputs and outputs, plus the command and environment that implement it. A pipeline job wires components into a DAG.

Why bother instead of one long script: step-level caching and step-level compute. If the featurise step's inputs haven't changed, Azure ML reuses the previous output instead of recomputing it — and the featurise step can run on cheap CPU while the training step runs on a GPU node. On a pipeline you rerun daily, that is the difference between a coffee and an afternoon.

Models and registries

A model is a versioned artifact plus metadata. Three flavours:

• custom_model — bytes you'll load yourself in a scoring script.
• mlflow_model — an MLflow-format model with its signature and dependencies. Prefer this: Azure ML can generate the scoring script and environment for you, so no-code deployment becomes possible.
• triton_model — packaged for the Triton inference server, for high-throughput GPU serving.

A registry (Microsoft.MachineLearningServices/registries) is a separate ARM resource, not a workspace, that holds models, environments, components, and data shared across workspaces and regions. This is the promotion mechanism: train in ws-dev, push the model to the registry, and have ws-prod deploy the byte-identical artifact — no rebuild, no re-upload, one lineage. If your MLOps story involves copying model files between workspaces, a registry is the thing you were missing.

Endpoints and deployments

The three-level model, and the reason it exists:

Endpoint  (stable HTTPS address + auth mode)   my-scorer
   ├── Deployment "blue"   → model:7,  env:3, 2 × Standard_DS3_v2   → 90% traffic
   └── Deployment "green"  → model:8,  env:4, 2 × Standard_DS3_v2   → 10% traffic

Your caller only ever knows the endpoint. A deployment pins a model version, an environment, a scoring script, an instance type, and an instance count. Traffic is split across deployments by percentage, adjustable in one command. Rollback is --traffic "blue=100 green=0".

Three kinds of endpoint:

Kind	Shape	Bills	For
Managed online	Always-on HTTPS, Azure manages the VMs	Per instance-hour, traffic or not	Real-time, low-latency scoring
Kubernetes online	Same API, running on your registered AKS/Arc cluster	Your cluster	Existing K8s, on-prem, custom scaling
Batch	Submit a job against a dataset; it runs on a compute cluster and writes outputs	Per node-hour of the run	Bulk scoring, nightly runs, large files

There is no scale-to-zero on managed online endpoints. That single fact drives most serving-cost decisions: spiky, latency-tolerant traffic is often cheaper on a batch endpoint or a Container App.

Auth on an online endpoint is key, aml_token (short-lived, Azure ML-issued), or aad_token (Entra ID). Key auth is the easy start and the standing secret; Entra ID is what production should use.

Control plane vs. data plane

Say this out loud once per service, and here it matters more than most because there are three addresses.

Plane	Endpoint	What it governs	Typical roles
Control plane	management.azure.com (ARM)	Create/delete workspace, compute, endpoints; role assignments; networking; tags; locks	Owner, Contributor, Reader, AzureML Compute Operator (start/stop compute without full contributor)
Workspace data plane	<region>.api.azureml.ms (discovered from the workspace)	Submit jobs, register and read assets, read metrics, manage deployments' data-side	AzureML Data Scientist — broad rights over assets and jobs, without the right to change workspace-level settings
Inference data plane	<endpoint>.<region>.inference.ml.azure.com	Actually scoring a request	Needs the endpoint /score/action permission, or a key, or an aml_token

The consequences you will hit:

• Owner on the resource group lets you create an endpoint and does not by itself mean your app's token can score against it. Grant the scoring permission explicitly.
• AzureML Data Scientist lets someone run jobs and burn money on compute that already exists, but not create new compute — that is a control-plane action. This is usually the split you want.
• The job's identity is not your identity. A job reading a datastore uses the compute's system- or user-assigned managed identity. Authorising yourself on the storage account fixes the studio preview and not the job.

⚠️ Exact built-in role names and their contained actions change; verify AzureML Data Scientist, AzureML Compute Operator, and AzureML Registry User against current Azure docs before writing policy.

Networking, in one paragraph

Two dials. Public network access on the workspace, and the managed virtual network — Azure ML provisions and runs a VNet for your compute so you don't build one. Its isolation mode is either AllowInternetOutbound (compute can reach the internet; inbound is locked down) or AllowOnlyApprovedOutbound (egress only to FQDNs and private endpoints you list — which provisions a managed firewall that you pay for). Add private endpoints for the workspace and for each dependent resource, and remember that a private workspace whose storage account is still public has solved half a problem. Full treatment in Production.

Terms in one line

Term	One line
Workspace	The regional resource that owns everything; check its kind
Datastore	A saved connection to storage, with the auth attached
Data asset	A named, versioned pointer to data — not a copy
Environment	A versioned image + dependency spec, cached in ACR
Job	One tracked execution: command, sweep, pipeline, AutoML, or Spark
Experiment	A name you group jobs under. Nothing more
Component	A reusable, versioned pipeline step with a typed interface
Model	A versioned artifact; prefer mlflow_model
Registry	A separate resource sharing assets across workspaces and regions
Endpoint	A stable address with an auth mode
Deployment	Model + environment + script + instances, sitting behind an endpoint
Traffic split	Percentages across deployments — your blue/green dial
Compute instance	A personal dev VM. Set idle shutdown
Compute cluster	An autoscaling job pool. Set min_instances = 0
Managed VNet	Azure-run network isolation for your compute

---

Next: Architecture →

← Back to the Azure Machine Learning overview · ← Previous: What & Why