Data Engineer/Senior Data Engineer - PySpark/Azure Databricks

Description :

- Implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake and Databricks components

- Develop and optimize large-scale distributed data processing systems

- Create and manage workflows using Databricks Workflows or orchestration tools

- Optimize Spark jobs for performance, scalability, and cost efficiency

- Implement data quality, governance, monitoring, and observability frameworks

- Build event-driven and streaming pipelines using Kafka or similar technologies

- Collaborate with cross-functional teams and global customers to deliver high-quality solutions

- Contribute to architecture discussions, modernization initiatives, and engineering best practices

- Ensure reliability, security, and scalability of data platforms

Required Skills & Qualifications :

- 3- 8 years of experience in Data Engineering

- Strong expertise in Databricks, PySpark, SQL, and distributed data systems

- Hands-on experience with ETL/ELT pipeline development and data modeling

- Good understanding of Kafka or other event streaming technologies

- Experience with Azure data ecosystem (ADLS, Azure Services, etc.)

- Strong understanding of scalable distributed systems and event-driven architectures

- Experience working in Agile environments

- Client-facing experience with global customers (US/EU preferred)

- Strong analytical, communication, and problem-solving skills

- Ownership mindset with ability to work independently

- Exposure to AI-assisted development tools is a plus