Role : Data Engineer

Experience : 4 - 6 years

Location : Pune & Gurugram

Role and responsibilities :

- Design, build, and maintain robust, scalable, and high-performance ETL/ELT pipelines.

- Develop and optimize data processing workflows on Databricks.

- Partnering with data architects, analysts, and business stakeholders to translate data requirements into reliable, production-grade pipelines.

- Drive end-to-end data engineering across project phases - ingestion, transformation, quality, and delivery, ensuring alignment with business objectives and data governance.

- Implement and enforce data quality checks, validation frameworks, and monitoring/alerting for pipeline health.

- Adhere to defined data engineering standards, guidelines, and best practices across projects.

Required Experience & Skills :

- 4 - 6 years of hands-on experience designing and building enterprise-scale data pipelines and ETL/ELT solutions.

- Strong hands-on expertise with Databricks : notebooks, jobs/workflows, delta tables, unity catalog.

- Deep proficiency in Apache Spark (PySpark and/or Spark SQL) for data processing.

- Strong programming skills in Python and advanced SQL for data transformation and analysis.

- Solid understanding of data warehousing concepts, dimensional modeling, and the medallion (bronze/silver/gold) lakehouse architecture.

- Demonstrated ability to troubleshoot, tune, and optimize jobs for performance and cost benefits.

- Experience working in agile environments with CI/CD for data pipelines.

- Excellent communication and documentation skills.

Tech Stack & Expertise :

- Databricks platform : Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks SQL, Workflows.

- ETL/ELT and orchestration.

- Data formats and storage : Parquet, JSON, Delta.

- Data quality frameworks.

- Cost optimization of data workloads : cluster right-sizing, job scheduling, and compute.