Data Quality Engineer (Databricks)

Full-time Other-General

Overview

Develop Delta Lake metric aggregation structures supporting enterprise quality reporting. Calculate and publish: Data Quality Index (DQI) scores; dimension-level quality metrics; rule pass/fail rates; dataset compliance scores; SLA adherence indicators. Provide curated outputs to support Power BI quality dashboards and executive reporting.

Monitoring, Alerting and Predictive Quality Management

Configure automated quality monitoring and alerting mechanisms. Implement threshold-based notifications using Databricks SQL Alerts, Azure Monitor integrations. Develop predictive risk scoring models to identify datasets at risk of future quality degradation. Support proactive quality management and operational intervention activities.

Root Cause Analysis and Remediation Support

Apply Databricks machine learning and pattern analysis techniques to profiling and rule execution outputs. Support AI-assisted root cause analysis across established remediat...