Job Title : Data Engineer (Databricks & PySpark)

Experience : 5-8 Years

Location : Bangalore (Hybrid)

Employment Type : Full-time

Job Description :

We are looking for an experienced Data Engineer with strong expertise in Databricks, PySpark, Spark, Python, and SQL to design, develop, and optimize scalable data processing applications.

The ideal candidate will build robust data pipelines, ensure high-quality data processing, and collaborate with cross-functional teams to deliver enterprise-grade data solutions.

Key Responsibilities :

- Design, develop, and maintain scalable data processing applications using Databricks, PySpark, Spark, and Python.

- Build and optimize data pipelines to support large-scale data processing and analytics.

- Collaborate with data engineers, data scientists, and business stakeholders to understand requirements and deliver high-quality solutions.

- Ensure data integrity, reliability, and performance across data processing workflows.

- Develop clean, maintainable, and efficient code following industry best practices.

- Perform data analysis, validation, and quality checks to ensure accurate data delivery.

- Monitor, troubleshoot, and optimize data workflows for performance and scalability.

- Implement and maintain CI/CD pipelines for automated testing, integration, and deployment.

- Participate in code reviews and contribute to continuous process improvements.

Required Skills & Experience :

- 5-8 years of experience in Data Engineering.

- Strong hands-on expertise in :

1. Databricks

2. Apache Spark

3. PySpark

4. Python

5. SQL

- Experience developing scalable ETL/ELT and data processing pipelines.

- Strong understanding of data engineering best practices and distributed data processing.

- Experience with relational databases and SQL optimization.

- Familiarity with Git and version control systems.

- Strong analytical, troubleshooting, and problem-solving skills.

- Excellent communication and collaboration skills.

Preferred Skills :

- Experience with cloud platforms such as Azure, AWS, or GCP.

- Knowledge of Delta Lake, Spark optimization, and data lake architectures.

- Experience with orchestration tools such as Apache Airflow or Azure Data Factory.

- Familiarity with Agile/Scrum methodologies and DevOps practices.

- Exposure to data warehousing concepts and modern analytics platforms.

Key Competencies :

- Databricks Development

- Apache Spark & PySpark

- Python Programming

- SQL & Data Processing

- ETL/ELT Development

- CI/CD & Git

- Data Quality & Performance Optimization

- Cloud Data Engineering

- Problem Solving & Analytical Thinking

- Cross-functional Collaboration