Regional data boundaries and egress gates

Last updated: 6 Oct 20265 min read

tutorial

A regional data boundary defines where classified inputs, derived outputs, processing jobs and diagnostic copies may exist; an egress gate checks each movement against that policy.

Inventory more than the main table

A warehouse table may stay in an approved region while query results, temporary spill files, error queues, traces, backups or a disaster-recovery replica cross the boundary. List every storage and processing location in the data product manifest. Attach classification and allowed region set to the dataset, then propagate the restriction to derived data unless an approved transformation changes its classification. Classification begins the decision; it does not enforce placement.

Check movement before bytes move

Validate destination region, service purpose, encryption key boundary and recipient authorization before a copy job starts. A nightly transfer that fails a policy check should not create a partial object at the destination. Apply the gate to backfills and emergency failover as well as steady-state replication. Recovery targets cannot be met by silently violating a placement contract.

Treat processing as a location

A dataset stored in one region can still be processed elsewhere by an export, managed query service or diagnostic tool. Keep the execution location in the release manifest and test the path a real operator uses. An encryption key in an approved region does not by itself prove that plaintext processing stayed there. Distinguish storage, transit, processing and support-access boundaries.

Record exceptions with expiry

Some products allow de-identified aggregates to leave the source region. State the transformation, re-identification risk, owner approval, receiving region and exception expiry. Reject an exception whose scope is simply all customer data. When the job changes schema or joins a new identifier, rerun classification before reusing the exception.

Exercise denied and allowed routes

Run a permitted intra-region materialization and a prohibited cross-region export in a test environment. The permitted path should publish one generation with location evidence; the prohibited path should write no destination artifact and emit a policy denial. Inspect backup and error paths too. A policy document without a failing deployment test is only an intention.

Implementation

datasets = {
    "customer_receipts": {"classification": "restricted", "allowed": {"region-west"}},
    "daily_counts": {"classification": "aggregate", "allowed": {"region-west", "region-east"}},
}

def permit_copy(catalog, dataset_name, destination):
    if dataset_name not in catalog:
        raise KeyError("unclassified dataset")
    return destination in catalog[dataset_name]["allowed"]

assert permit_copy(datasets, "daily_counts", "region-east")
assert not permit_copy(datasets, "customer_receipts", "region-east")

Performance and operating cost

The catalog lookup and set membership are O(1) expected time and space per decision, with O(D + P) policy storage for D datasets and P allowed placements. Real gates also check service identities, keys and lineage before transfer; those calls add latency and must fail closed when policy state is unavailable. The cost of regional copies includes network egress, storage and duplicate retention.

Common Mistakes

Do not check only the primary table while ignoring logs, spill and backup copies.
Do not treat encryption key location as proof of processing location.
Do not let an emergency recovery path bypass the egress gate.