Backup & Retention

In a Nutshell

Backups are copies of data you can restore from when the original is lost or corrupted; retention is the policy governing how long you keep data (and its backups) before deleting it. Together they answer two opposing pressures: keep data long enough to recover from disasters and meet legal obligations, but don't keep it so long that you waste storage, increase breach exposure, and violate privacy rules. This is the storage-and-lifecycle view of a topic touched in Backup & Data Durability — here the focus is on policy: what to keep, for how long, where, and when to delete, balancing recoverability, cost, compliance, and privacy.

How It Actually Works

Backup Fundamentals

Backups protect against logical loss (accidental deletes, corruption, ransomware, bad deploys) that replication can't — because replication faithfully copies your mistakes (see Backup & Data Durability):

Backup Type	Captures	Trade-off
Full	Everything	Fast restore, most storage
Incremental	Changes since last backup	Least storage, slower restore (chain)
Differential	Changes since last full	Middle ground
Snapshot	Point-in-time image (copy-on-write)	Efficient, fast
Continuous (PITR)	Every change via log	Restore to any moment

The 3-2-1 Rule

The canonical backup guideline:

3 copies of the data
2 different media/storage types
1 copy offsite (different location/provider)

Modern addition: 1 copy IMMUTABLE/offline — to survive ransomware that
                 encrypts or deletes your online backups.

Retention Policies: How Long to Keep

Retention is a policy decision balancing several forces:

Driver	Pushes Retention...
Disaster recovery	Longer (recover from old corruption)
Legal/compliance	Fixed (regulations mandate periods)
Cost	Shorter (storage isn't free)
Privacy (GDPR)	Shorter (storage limitation — don't keep PII forever)
Business/operational	Varies (how far back is useful)

Example tiered retention:
  Daily backups   → keep 30 days
  Weekly backups  → keep 12 weeks
  Monthly backups → keep 12 months
  Yearly backups  → keep 7 years (compliance)
  → recent granularity for operational recovery, sparse long-term for
    compliance, old data expired to control cost and privacy risk.

The Compliance vs Privacy Tension

A subtle but important conflict: some regulations require you to keep data (financial records for 7 years), while privacy laws require you to delete personal data when no longer needed (GDPR storage limitation) or on request (right to erasure). Retention policy must reconcile both — often by keeping compliance-required records while minimizing and anonymizing personal data (see Data Privacy).

Keep: transaction records (legal) — but pseudonymized where possible.
Delete: raw personal data past its purpose (privacy).
Honor: erasure requests — while retaining legally-mandated minimal records.

Storage Tiering for Cost

Old backups rarely need fast access, so lifecycle policies move them to progressively cheaper, slower tiers:

Hot (frequent access)  → Warm (infrequent) → Cold (archive) → Deep archive
   $$$$                    $$$                 $$              $
   recent backups          older              archival        long-term
                                                               compliance
Automated lifecycle rules transition + expire objects (see object-storage).

The Rules That Make Backup/Retention Actually Work

• Test restores — an untested backup is a guess; drill regularly and measure RTO/RPO.
• Automate the policy — schedules, tiering, and expiry should be automatic, not manual.
• Document what's kept and why — for audits and to justify retention periods.
• Encrypt backups — they contain the same sensitive data as production.
• Protect against ransomware — immutable/offline copies that malware can't reach.
• Align deletion with privacy — expiry must actually remove data, including from backups per policy.

Seeing It in Action

Scenario: A fintech company's backup and retention strategy.

Backup strategy (3-2-1 + immutability):
  - Database: continuous PITR (base backup + WAL) → restore to any second.
  - 3 copies: primary region, second region, and an immutable archive.
  - 2 media/services: managed DB backups + object-storage exports.
  - 1 offsite + immutable: cross-region, write-once (WORM) bucket that
    ransomware can't encrypt or delete.

Tiered retention policy (automated lifecycle):
  - PITR window:        35 days (operational recovery, RPO ~seconds)
  - Daily snapshots:    kept 90 days   (hot → warm)
  - Monthly snapshots:  kept 7 years   (cold archive — financial compliance)
  - Transaction records: 7 years       (regulatory requirement)
  - Raw analytics logs:  90 days, then deleted (privacy — not needed longer)

Compliance vs privacy reconciliation:
  - MUST keep: transaction/audit records 7 yrs (financial regulation).
  - MUST minimize/delete: customers' raw personal data past its purpose,
    and honor erasure requests.
  - Resolution: retain compliance records in pseudonymized form (reference
    customers by token); delete/anonymize raw PII on schedule and on request,
    while preserving the minimal legally-required record.

Operational discipline:
  - Automated MONTHLY restore drills to a scratch environment; measure RTO.
  - Backups encrypted at rest; access audited.
  - Retention rules enforced by lifecycle automation, not manual cleanup.

The failure this prevents:
  ✗ A bad migration corrupts the DB → replicas faithfully replicated the
    corruption → without PITR backups, data is lost.
  ✓ PITR restores to the second before the migration. Immutable offsite
    copy survives even a ransomware attack that hits the primary backups.

Why backup and retention must be designed as policy, not left to chance: the two failure modes are symmetric and both costly. Keep too little (or never test restores) and a corruption, ransomware attack, or bad deploy becomes unrecoverable data loss — replication won't save you because it copies the damage. Keep too much and you waste money on storage, expand the blast radius of any breach, and violate privacy laws that require deleting personal data you no longer need. A good strategy resolves this with tiered retention — fine-grained recent backups for operational recovery, sparse long-term copies for compliance, automatic expiry of everything past its useful and lawful life — layered on a 3-2-1 foundation with an immutable offsite copy to survive ransomware. The uniquely hard part is reconciling compliance (which mandates keeping certain records for years) with privacy (which mandates deleting personal data promptly), typically solved by retaining minimal, pseudonymized compliance records while aggressively minimizing and expiring raw personal data. And the discipline that makes all of it real is testing restores — because a backup you've never restored is not a backup, it's a hope. Backup and retention done well is what lets a company survive disasters, satisfy regulators, control cost, and respect privacy all at once.

Interview Questions

1. Q: What is the 3-2-1 backup rule, and what modern addition addresses ransomware? Hint: 3 copies of the data, on 2 different media/storage types, with 1 offsite (different location/provider). The modern addition is 1 immutable/offline copy — write-once or air-gapped — so ransomware that encrypts or deletes your online backups can't reach it. This ensures a clean recovery point survives even a full compromise of your primary environment and its online backups.

2. Q: How do you decide retention periods, and what forces are in tension? Hint: Balance disaster recovery (keep longer to recover from old corruption), legal/compliance (fixed mandated periods), cost (storage isn't free → shorter), privacy (GDPR storage limitation → delete PII promptly), and business usefulness. Typically tiered: fine-grained recent backups for operations, sparse long-term copies for compliance, and automatic expiry of everything past its useful/lawful life to control cost and privacy risk.

3. Q: How do you reconcile compliance requirements to keep data with privacy requirements to delete it? Hint: They genuinely conflict — regulations may require keeping records (e.g., financial data 7 years) while privacy law requires deleting personal data when no longer needed or on request. Reconcile by retaining the minimal legally-required records in pseudonymized/anonymized form (reference people by token, strip unnecessary PII), while deleting/expiring raw personal data on schedule and honoring erasure requests for everything not legally mandated.

4. Q: Why is testing restores essential, and what should the test measure? Hint: A backup you've never restored is unverified — it may be corrupt, incomplete, missing dependencies (schema, keys, config), or too slow to meet objectives. Regular restore drills prove backups actually work and measure real RTO (how fast you recover) and RPO (how much data you'd lose) against targets. "We have backups" means nothing without demonstrated, timed recovery.

5. Q: Why is replication not a substitute for backups in a retention strategy? Hint: Replication protects against hardware/physical loss by keeping identical live copies — but it replicates everything, including accidental deletes, corruption, bad migrations, and ransomware, to all replicas instantly. Backups are point-in-time copies (ideally with PITR) that let you restore to before a logical error occurred. Retention policy governs how long those recovery points are kept; replication provides none of that historical recoverability.

References

• Backup & Data Durability (Topic 06) — durability and PITR foundations
• AWS Backup & lifecycle policies — automated backup and retention
• CISA: 3-2-1 backup guidance — canonical backup rule

Dive Deeper

• Google SRE Book: Data Integrity — backups, retention, and why restores fail
• GDPR storage limitation & retention — the privacy side of retention
• Object storage lifecycle policies (S3) — automated tiering and expiry