IT‑IS‑Networking
Projects › Local government

Disaster Recovery Validation and Application Tiering

Criticality bound to recovery objectives in an instrument rather than a narrative, and a definition of "tested" that the estate could not yet meet — reported rather than softened.

By October 2020 the continuity plan was being revised against NIST 800 guidance, with application criticality expressed as a tier table rather than as prose. Each tier ties to a recovery time objective, a recovery point objective, a maximum tolerable downtime, a site posture and the mechanism required to achieve it.

  • Tier 0, continuous — zero and zero, active/active, cross-site clustering on synchronous replication
  • Tier 1, highly critical — zero to one hour and zero to fifteen seconds, active/passive, automated recovery on continuous asynchronous replication
  • Tier 2, critical — one to four hours, manual recovery to the alternate site

The table carried its own warning: an application tier rating is not to be transposed with a data center tier rating. That is the standard error in availability planning, because it lets an organization believe a Tier III building confers Tier 0 recovery on the software inside it. A facility rating describes power and cooling paths. It says nothing about whether an application can start somewhere else.

Three legs, not one

The validation regime imposed the same October defined tested as three things rather than one: replication configured to the recovery site, cutover actually performed, and somebody on a keyboard using the system successfully once it was running there.

That rules out the three things which ordinarily pass for recovery evidence — that replication exists, that a runbook exists, and that a failover was once performed without anyone using the result.

Applied honestly it produced an uncomfortable finding, and the finding was reported rather than softened: for lack of evidence, no applications or servers could be called ready for failover.

It also surfaced what only appears when replication coverage is checked against the application inventory rather than against the server list. A quality assurance database carrying live-adjacent workloads had no replication at all while its production counterpart was covered. Public safety web applications shared servers with unrelated sites that would have moved incidentally during a failover, so the blast radius of a recovery action did not match the plan’s stated scope.

The regime was collaborative and its ownership shared, driven by infrastructure management alongside architecture. The contribution here is real rather than sole. The value was in establishing, against evidence, which of the redundancy already paid for actually worked.

Outcome

An evidence-based answer to which of the redundancy already paid for actually worked.