KPI Partners is a global consulting and technology services firm specializing in data, analytics, and digital transformation solutions. With deep expertise in cloud platforms such as Microsoft Azure, Snowflake, and Databricks, KPI Partners helps organizations modernize their data ecosystems and drive business insights. The company focuses on delivering scalable data engineering, business intelligence, and AI-driven solutions across industries, enabling clients to make faster, data-driven decisions and achieve measurable business outcomes.

Job Summary :

We are looking for a Data Engineer with strong experience in Databricks, Airflow, Python, and SQL to build and maintain scalable data pipelines and data processing solutions. The ideal candidate will have expertise in ETL/ELT development, workflow orchestration, data modeling, and performance optimization.

Key Responsibilities :

- Develop and optimize data pipelines using Databricks, Python, and SQL.

- Design and manage workflow orchestration using Apache Airflow.

- Build scalable ETL/ELT processes for data ingestion and transformation.

- Monitor, troubleshoot, and improve data pipeline performance.

- Implement data quality checks and validation frameworks.

- Collaborate with cross-functional teams to deliver data-driven solutions.

Required Skills :

- Strong hands-on experience with Databricks and PySpark.

- Expertise in Apache Airflow DAG development and orchestration.

- Proficiency in Python and advanced SQL.

- Experience with data warehousing and cloud-based data platforms.

- Knowledge of performance tuning, data modeling, and ETL best practices.

Preferred Skills :

- Experience with AWS, Azure, or GCP.

- Familiarity with Delta Lake, CI/CD, and Agile methodologies.

Experience :

- 6 - 10 years experience