Lead Data Engineer – PySpark & Databricks

Full-time Other-General

Experience:- 8-10 Years
Work Type- Hybrid Duration:6-12 Months
Shift Timing: 1:00 PM 10:00 PM

Job Locations:--Bengaluru, India / Pune, India / Noida, India / Gurgaon, India

Skills Required

Primary Skills

PySpark and Databricks

Job Description

Shift Timing: 1:00 PM 10:00 PM

Key Responsibilities A. Data Engineering & Pipeline Development

Design, develop, and optimize large-scale ETL/ELT pipelines using PySpark on Databricks, processing structured and unstructured data at scale.

Build and maintain Lakehouse architecture (Bronze/Silver/Gold medallion layers) using Delta Lake, ensuring reliability, scalability, and schema evolution support.

Develop reusable, parameterized, metadata-driven pipeline frameworks for ingestion, transformation, and curation of data from diverse source systems.

Optimize Spark jobs for performance and cost (partitioning...