Celebal Technologies is a leading Solution Service company that provide Services the field of Data Science, Big Data, Enterprise Cloud & Automation.

We are at the forefront of leveraging cutting edge technologies to drive innovation and enhance our business processes.

As part of our commitment to staying ahead in the industry, we are seeking a talented and experienced Data & AI Engineer with strong Azure cloud competencies to join our dynamic team.

Key Responsibilities :

1. Architecture and Design for Data Engineering and Machine Learning Projects :

- Establishing architecture and target design for data engineering and machine learning projects.

2. Requirement Analysis, Planning, Effort and Resource Needs Estimation :

- Current inventory analysis, review and formalize requirements, project planning and execution plan.

3. Advisory Services and Best Practices :

- Troubleshooting, Performance Tuning, Cost Optimization, Operational Runbooks and Mentoring.

4. Large Migrations :

- Assist customers with large migrations to Databricks from Hadoop ecosystems, Data Warehouses (Teradata, DataStage, Netezza, Ab Initio), ETL engines (Informatica), SAS, SQL, DW, Cloud-based Data platforms like Redshift, Snowflake, EMR, etc.

5. Design, Build and Optimize Data Pipelines :

- The Databricks implementation will be best in class, with flexibility for future iterations.

6. Production Readiness :

- Assisting with production readiness for customers, including exception handling, production cutover, capture analysis, alert scheduling and monitoring.

7. Machine Learning (ML) Model Review, Tuning, ML Operations and Optimization :

- Build and review ML models, ML best practices, model lifecycle, ML frameworks and deploying models in production.

Requirements :

- Tech/BCA or any Graduation.

- Pre-Sales experience is a must.

- Hands on experience with distributed computing framework like DataBricks, Spark Ecosystem (Spark Core, PySpark, Spark Streaming, SparkSQL).

- Willing to work with product teams to best optimize product features/functions.

- Experience on Batch workloads and real time streaming with high volume data frequency.

- Performance optimization on Spark workloads.

- Environment setup, user management, Authentication and cluster management on Databricks.

- Professional curiosity and the ability to enable yourself in new technologies and tasks.

- Good understanding of SQL and a good grasp of relational and analytical database management theory and practice.