---
title: "Reliability, upgrades, and disaster recovery"
chapter: "14"
---

# Reliability, upgrades, and disaster recovery

GKE control-plane availability is only one part of application reliability.

## Regional baseline

Use regional clusters for important production workloads, spread Pods and
capacity across zones, keep multiple replicas, define disruption budgets, and
remove single-zone storage or downstream dependencies when the requirement
demands it.

## Upgrades

Use release channels, maintenance windows, and exclusions. GKE upgrades all
clusters over time; you can control timing but cannot stay unsupported forever.
Test new versions and APIs in lower environments or canary clusters. Monitor
deprecated APIs and ensure Pod disruption budgets do not make maintenance
impossible.

## Failure design

Test Pod deletion, node drain, zone capacity loss, dependency timeouts, DNS
failure, quota exhaustion, image-pull failure, and bad configuration. Use
topology spread, graceful termination, safe retries, and capacity headroom.

## Disaster recovery

Start from RTO and RPO. Preserve:

- clusters and cloud infrastructure as code;
- manifests and policy in version control;
- images and provenance in durable registries;
- workload resources and volumes with Backup for GKE where appropriate;
- external database backups and replication;
- DNS, secrets, certificates, and recovery permissions.

Restore into a predesigned target, validate data and traffic, and practice. A
backup plan that has never restored is a hope, not evidence.