Data classification and access boundaries

Last updated: 6 Oct 20265 min read

tutorial

Classification labels data by sensitivity and allowed use; access boundaries turn those labels into enforceable permissions at each pipeline stage.

Classify the fields actually stored

A payments pipeline may carry account ID, merchant ID, card token, postal code and derived risk score. Classify raw values, joinable pseudonyms and aggregates separately. A hashed account ID may still identify someone when the mapping is available or the input space is small. Record the purpose, retention and authorized readers for every sensitive field, not only the table name.

Keep raw and serving zones distinct

Landing stores the minimum source payload needed for replay under restricted roles. Transformation workers receive only the fields needed for their task; an analytics mart publishes approved columns and aggregates. Raw landing should not become a broad query playground. Limit export, temporary tables, logs and failed-record queues too, since those paths often copy sensitive values.

Bind permission to identity and purpose

Grant access to service identities and named job roles, then map human users to reviewed groups. Avoid shared credentials that erase accountability. A pipeline operator may need to inspect health and row counts without reading full payment identifiers. A data steward may approve a new use but should not have to run the compute job. Separation makes incident review possible.

Propagate labels through derivations

A joined table inherits the sensitivity of its inputs unless the derivation provably removes it for the authorized use. Small group counts can reveal an individual even when direct identifiers are absent. Keep lineage from source fields to output fields, and require a new access review when a new join or export makes previously separate attributes linkable. Lineage provides the dependency map.

Verify denial as well as approval

Test that an approved report role can read its aggregate, that an analyst role cannot read raw account IDs, and that an operations role sees logs with values redacted. Record which identity ran the test and which policy version applied. A permission policy is incomplete until the denied paths are exercised.

Implementation

FIELD_POLICY = {
    "account_id": {"class": "restricted", "roles": {"risk_service"}},
    "merchant_region": {"class": "internal", "roles": {"risk_service", "analyst"}},
    "daily_count": {"class": "aggregate", "roles": {"risk_service", "analyst"}},
}

def permitted_fields(role, requested):
    denied = [name for name in requested if role not in FIELD_POLICY[name]["roles"]]
    if denied:
        raise PermissionError("requested fields exceed role policy")
    return requested

assert permitted_fields("analyst", ["merchant_region", "daily_count"])
try:
    permitted_fields("analyst", ["account_id"])
except PermissionError:
    pass
else:
    raise AssertionError("restricted field was exposed")

Performance and operating cost

A set-membership check takes O(F) time for F requested fields and O(F) policy metadata. Real access controls add identity lookup, policy evaluation, audit writes and possibly query rewriting. Denying an unauthorized scan early is cheaper than filtering sensitive records after they have already moved to a less trusted stage.

Common Mistakes

Do not treat a hash as automatically anonymous.
Do not forget logs, exports and quarantine tables when classifying data.
Do not rely on a role name without testing both allowed and denied queries.