Company Overview:

ERGOBITE TECH SOLUTIONS PRIVATE LIMITED is a technology services firm specializing in data engineering, cloud transformation, and advanced analytics. We partner with global enterprises to modernize their data infrastructure, enabling them to derive actionable insights from complex, large-scale datasets. Our culture is built on technical excellence, collaborative problem-solving, and a commitment to delivering scalable solutions that drive digital transformation across the finance, retail, and healthcare sectors.

Role Overview:

As a Databricks Developer, you will be responsible for designing and implementing robust data pipelines that power our clients' analytical ecosystems. You will work closely with data architects, business analysts, and engineering teams to translate complex business requirements into high-performance data solutions. Your work will directly influence how our clients manage their data assets, ensuring high availability, data integrity, and efficient processing within the Azure ecosystem to support critical business decision-making.

Key Responsibilities:

- Develop and optimize scalable ETL pipelines using PySpark and Apache Spark to process large volumes of structured and unstructured data for downstream analytics.

- Architect and maintain efficient data storage solutions using Delta Lake to ensure data consistency, reliability, and performance across the enterprise data lake.

- Collaborate with cross-functional teams to design data models that meet specific reporting and analytical needs, ensuring data quality and security standards are met.

- Troubleshoot and resolve performance bottlenecks in existing Databricks notebooks and clusters to improve overall system efficiency and reduce processing costs.

- Automate data workflows and integrate them into existing CI/CD frameworks to ensure seamless deployment and maintenance of data products.

Required Skillset:

- Demonstrated expertise in building and managing complex data pipelines using Azure Databricks and PySpark within a cloud-native environment.

- Strong proficiency in SQL for advanced data manipulation, query optimization, and schema design.

- Proven ability to design and implement data lake architectures using Delta Lake, ensuring optimal partitioning and file management.

- Excellent analytical and problem-solving skills, with the ability to communicate technical concepts clearly to both technical and non-technical stakeholders.

- Ability to work effectively in a hybrid work environment in Pune, maintaining high levels of productivity and collaboration with distributed teams.

- A Bachelor's degree in Computer Science, Information Technology, or a related quantitative field, supported by 3 - 5 years of relevant professional experience in data engineering.