Job Description
Roles & Responsibilities
- Architect and implement robust, scalable data pipelines on Azure using Databricks, ensuring efficient data ingestion, transformation, and loading.
- Design and manage real-time data streaming solutions with Kafka, enabling low-latency data processing and event-driven architectures.
- Lead the development of data models and schemas optimized for performance and analytical querying within the Azure ecosystem.
- Oversee the deployment, monitoring, and optimization of data infrastructure, ensuring high availability and cost-effectiveness.
Desired Candidate Profile
- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
- 5+ years of progressive experience in data engineering, with a strong focus on cloud platforms.
- Proven expertise in Azure services, particularly Azure Databricks and Azure Data Factory.
- Deep understanding of Kafka for real-time data streaming and event processing.