Project: select a model and audit warehouse segments

Frame two outputs

The predictive output estimates clearance hours at first sorting scan. The unsupervised output groups warehouses by predeclared backlog and handoff features to inform a staffing discussion. A prediction can be scored against future clearance hours; a cluster has no ground-truth label by default. Keep separate success criteria, partitions and release claims. Model selection defines the predictive test boundary.

Select and inspect the predictive model

Compare a training-only baseline with candidate ridge settings on time-aware development folds. Fit each scaler inside its fold, choose a penalty and freeze the resulting pipeline. Inspect future MAE and costly underestimates. If the final model performs poorly, do not tune against the final test and report it as untouched. Use a development holdout for permutation importance and record correlated-feature caveats. Feature inspection has a narrower meaning than causality.

Build and challenge the segments

Specify numeric features and scaling, then fit K-means with declared K values and multiple initializations. Report inertia, membership stability, smallest cluster support and site profiles. Fit PCA only as a diagnostic projection or a separately tested transformation; do not choose clusters solely because a two-dimensional plot looks tidy. Scale choice, centroid fitting and PCA are distinct decisions.

Demand an operational test

A segment label is useful only if it changes a staffing or routing decision that can be evaluated. Compare the proposed segment-based action to the current policy on a later period or a controlled rollout, accounting for capacity. Avoid naming one cluster "bad warehouses" from a correlation with delays. Store feature definitions and centroid version so a later scoring run means the same thing.

Deliver a reviewable packet

The code checks for evidence artifacts, not model quality. Include the fold plan, all candidate scores, final test record, feature-importance protocol, scale manifest, cluster restarts, PCA purpose and intended action. If the scale was fitted on future sites or test outcomes influenced model selection, stop and rebuild the evaluation. The pipeline lesson covers that failure.

Implementation

def selection_and_segment_gate(packet):
    required = {"fold_calendar", "candidate_scores", "final_test_record",
                "training_transform_manifest", "permutation_protocol",
                "segment_features", "cluster_restarts", "pca_purpose",
                "staffing_action"}
    missing = sorted(required - set(packet))
    if missing:
        raise ValueError("missing review evidence: " + ", ".join(missing))
    if not packet["candidate_scores"] or not packet["cluster_restarts"]:
        raise ValueError("selection or segment evidence absent")
    if packet["final_test_record"]["shipments"] <= 0:
        raise ValueError("empty final test")
    return {"packet_complete": True,
            "candidates": len(packet["candidate_scores"]),
            "restarts": len(packet["cluster_restarts"])}

packet = {"fold_calendar": ["spring", "summer", "autumn"],
          "candidate_scores": {0: 3.5, 20: 3.1, 80: 3.2},
          "final_test_record": {"shipments": 57, "mae": 3.3},
          "training_transform_manifest": "warehouse-scale-v2",
          "permutation_protocol": "development holdout, three seeds",
          "segment_features": ["backlog", "handoff_minutes"],
          "cluster_restarts": [23, 47, 61],
          "pca_purpose": "diagnostic projection only",
          "staffing_action": "compare shift allocation against current roster"}
review = selection_and_segment_gate(packet)
assert review == {"packet_complete": True, "candidates": 3,
                  "restarts": 3}

Performance and operating cost

The artifact gate is O(A) expected time for A packet keys and O(A) sorting space for missing keys. Candidate fitting, repeated clustering, site review and the staffing evaluation are the substantial costs.

Common Mistakes

Do not call cluster membership a validated outcome label.
Do not tune the predictive model on the final future test.
Do not fit scaling or PCA on later sites while claiming a frozen transform.