Job Description
Overall Responsibilities
Design and develop scalable, reliable data pipelines on Azure to collect, process, and store large datasets
Implement efficient data models and data warehousing solutions aligning with business requirements
Collaborate with data scientists, analysts, and application teams to integrate data workflows
Optimize data pipeline performance, reliability, and security
Support data governance, compliance, and data privacy standards
Monitor operational health of data solutions and troubleshoot issues proactively
Document architecture, workflows, and best practices for data ingestion and transformation
Stay updated with the latest Azure data platform innovations and incorporate improvements
Software Requirements
Proficiency in SQL and Python for data processing and scripting