Lead Data Engineer

W-2 Computer Occupations

<div>Lead end-to-end delivery of large-scale data engineering initiatives, ensuring high-quality and timely outcomes.</div> <div>Architect, design, and implement scalable, cloud-native data platforms using <b data-ogsc="">Databricks, Apache Spark (Scala), and AWS services</b>.</div> <div>Develop and maintain Bronze, Silver, and Gold layer implementations following <b data-ogsc="">Databricks Medallion Architecture</b> best practices.</div> <div>Design and build high-performance batch and streaming data pipelines with strong emphasis on scalability, reliability, and maintainability.</div> <div>Drive Spark performance tuning and optimization, including query optimization, partitioning strategies, memory management, and cluster utilization.</div> <div>Develop and enhance <b data-ogsc="">metadata-driven ingestion and processing frameworks</b> to accelerate...