Shift-Left Data Quality & Observability for Customer-Facing Product Publishing

Scenario

A production, customer-facing product configuration experience was repeatedly publishing incorrect product data, leading to revenue-impacting errors and avoidable customer dissatisfaction.

• Sales errors: customers configured the wrong products due to incorrect bundles/attributes.
• Publishing failures: products failed to appear on the live site, reducing sell-through and revenue capture.
• Marcom & compliance risk: product communications and disclosures were incorrect or non-compliant.
• Reactive detection: issues surfaced only after customer complaints or order cancellations.

Initial Framing (and Why It Didn’t Scale)

The issue was initially framed as a human-in-the-loop QA problem. The mitigation was a weekly “scrub call” where large cross-functional groups manually clicked through a sandbox environment before pushing to production.

• High coordination cost: participation ballooned to 100+ people weekly.
• Low reliability: results depended on attendance and what pages people happened to inspect.
• High operational toil: ~3+ hours/week of repetitive manual validation.

This approach treated symptoms in the UI rather than addressing upstream data defects that were propagating downstream.

Problem Framing

I reframed the issue as a data quality and pipeline integrity problem: if incorrect information reaches production, then upstream systems are producing or transforming incorrect data without enforceable validation rules.

Product data was due from development teams at T-3 weeks before launch — meaning the data already existed in upstream databases long before it reached the live site. That enabled a shift-left validation strategy: validate the source-of-truth data early, continuously, and automatically.

Solution

Built a lightweight data quality + release readiness workflow that: (1) curated a set of data products representing high-risk product configurations, (2) enforced data contracts and business-rule validations, (3) generated exceptions for triage, (4) automated remediation for known defect patterns, and (5) produced launch-ready artifacts for downstream web publishing.

Architecture (Visual)

A simple end-to-end view of the flow: curate → validate → remediate → re-validate → publish readiness → monitor.

Implementation Approach

1) Build the Data Products (Curated Tables)

Modeled the product experiences most frequently broken on the website as a curated dataset (a “data product”), capturing key identifiers plus audit metadata to support traceability and change monitoring.

Data Product: product_software_mapping

Columns:
- product_id
- product_name
- software_id
- software_name
- last_modified_timestamp
- source_system / channel (optional)
- effective_date (optional)

2) Encode Business Rules as Data Contracts

Translated business expectations into explicit validation rules (a lightweight “data contract”) enforced at the database layer so the “publishable” dataset matched intended product configuration logic.

Example contract rules:
- product_id = 12345 must include software_ids {345, 678, 891}
- software_id = 345 must map to software_name = "Microsoft"
- software_id = 678 must map to software_name = "McAfee"
- software_id = 891 must map to software_name = "CrowdStrike"

3) Automated Validation Pipeline (Python + SQLAlchemy)

Implemented a validation runner that queried curated tables, executed rule checks, and wrote results to an exceptions table for downstream consumption — creating a repeatable, auditable quality gate that could be scheduled.

Validation Output Tables:
- dq_results_summary (run_id, timestamp, pass_count, fail_count)
- dq_exceptions (run_id, product_id, software_id, rule_id, status, reason_code, last_modified_timestamp)

Classification:
IF product_id + software_id + software_name match contract
  THEN PASS
ELSE
  FAIL with reason_code:
    - missing_mapping
    - unexpected_mapping
    - name_mismatch
    - stale_data / drift_detected

4) Power BI Dashboards (Direct Connection)

Built operational dashboards in Power BI using a direct connection to curated tables and exception outputs so product managers and developers could monitor release readiness in near-real time.

• Release readiness: pass/fail by launch date, product line, and experience
• Exception backlog: failures ranked by severity and business impact
• Drift monitoring: recent changes to high-risk products (CDC-style visibility)
• Root cause support: audit metadata to trace defects back upstream

Automation & Release Enablement

In addition to validation and monitoring, I implemented automation workflows to close the loop between data quality detection and downstream execution — reducing rework and making release readiness explicit.

Automated Remediation & Enrichment

When validation checks surfaced failures, a secondary Python-based remediation process applied deterministic correction and enrichment logic to resolve known, well-understood defect patterns upstream.

• Rule-based correction of missing or incorrect product attributes
• Deterministic mapping to authoritative reference values (“golden record” behavior)
• Standardization of product/software metadata for consistent downstream consumption
• Re-validation of remediated records to ensure contract compliance

Remediation Control Loop:
1. Detect validation failure
2. Classify failure type (missing, mismatch, drift)
3. Apply deterministic correction / enrichment rules
4. Re-run validation checks
5. Persist corrected record + audit metadata

Product Readiness Artifact Generation

Once records passed validation, the pipeline generated product readiness artifacts in standardized, contract-compliant formats required by the web development team. These artifacts served as a clean handoff boundary between data workflows and application publishing workflows.

• Structured files (e.g., CSV / JSON) aligned to downstream schemas
• Explicit inclusion of only “release-approved” configurations
• Consistent typing/formatting to support direct ingestion (no manual cleanup)
• Versioned outputs to support traceability and rollback

Release Artifact Characteristics:
- Schema-aligned
- Versioned
- Source-of-truth backed
- Launch-ready only (no partial / invalid records)

Pre-Release Quality Gate

Together, validation, remediation, and artifact generation formed a pre-release quality gate ensuring only compliant, fully validated product data was eligible for publication. This shifted defect detection upstream while reducing manual coordination and downstream rework.

Impact

• Reduced manual QA toil: replaced weekly “scrub calls” with automated checks + exception triage.
• Shifted defect detection left: surfaced issues at T-3 weeks rather than after production release.
• Improved cross-functional execution: enabled PMs and developers to collaborate using a shared operational view.
• De-risked customer experience: fewer configuration errors, fewer cancellations, improved compliance confidence.