We are looking for a highly skilled Senior Data Engineer with strong expertise in Python, PySpark, SQL, Databricks, and Spark SQL.

The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and data platforms that support analytics, reporting, and data-driven decision-making across the organization.

Key Responsibilities :

- Design, develop, and maintain scalable and efficient data pipelines using Python and PySpark.

- Build and optimize ETL/ELT workflows on Databricks.

- Write complex and high-performance queries using SQL and Spark SQL.

- Process and transform large-scale structured and semi-structured datasets.

- Ensure data quality, reliability, and performance across data pipelines.

- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.

- Optimize data processing jobs for performance and cost efficiency.

- Implement best practices for data engineering, including version control, testing, and documentation.

- Troubleshoot and resolve data-related issues in production environments.

- Participate in code reviews and mentor junior data engineers.

Required Skills & Qualifications :

- Strong hands-on experience with Python for data engineering.

- Extensive experience in PySpark and Apache Spark.

- Advanced knowledge of SQL and Spark SQL.

- Proven experience working with Databricks platform.

- Strong understanding of data warehousing concepts and data modeling.

- Experience handling large-scale data processing and distributed systems.

- Familiarity with cloud platforms (AWS / Azure / GCP) is a plus.

- Experience with workflow orchestration tools (Airflow, Azure Data Factory, etc.) is an advantage.

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities