What youll do :

- Lead the design, development, and maintenance of scalable data pipelines and lakehouse architectures using Azure Databricks (PySpark, SQL).

- Architect and implement Medallion Architecture (Bronze, Silver, Gold layers) to drive optimal data transformation and organization.

- Oversee data governance, access control, and lineage using Databricks Unity Catalog; champion best practices for metadata management.

- Manage and mentor a team of data engineers; provide technical guidance and career development opportunities.

- Direct the management of data storage solutions using Azure Data Lake Storage (ADLS), ensuring robust access policies.

- Establish and maintain Delta Lake tables for reliable, scalable ACID transactions and incremental processing across projects.

- Optimize Spark jobs and Databricks clusters for performance, scalability, and cost efficiency at enterprise scale.

- Lead ingestion and transformation strategies from diverse sources (APIs, files, databases, cloud platforms).

- Partner with stakeholders and cross-functional teams to deliver well-governed, reusable data models and analytics solutions.

- Drive implementation of data quality frameworks, error handling mechanisms, and robust monitoring practices.

- Enforce data security, governance, and compliance requirements across all Azure and cloud data assets.

- Utilize Azure Synapse Analytics, Key Vault, and Azure Monitor to build resilient, secure, and observable data platforms.

- Own CI/CD and deployment best practices using Git and Azure DevOps; establish automation standards and code review processes.

- Stay ahead of industry trends in big data, ETL, and lakehouse technologies; proactively recommend improvements.

- Collaborate with data science and AI teams to support machine learning (ML), generative AI, and large language model (LLM) initiatives using enterprise data.

- Guide data consultation efforts for enterprise digital transformation, advising stakeholders on data architecture modernization and strategic priorities.

- Develop and articulate an AI-ready data roadmap, enabling the business to leverage advanced analytics, ML, and LLM solutions.

- Ensure data pipelines and platforms are optimized for ML and AI workloads, including integration and preparation of datasets for model training and inference.

What we seek in you :

- Bachelors or Masters in Computer Science, Engineering, or related field.

- 8+ years of experience in data engineering, with 2+ years in a technical lead or supervisory role.

- Advanced expertise in Databricks, PySpark, SQL, Azure cloud data engineering, and related tools.

- Proven track record architecting and leading end-to-end pipeline solutions leveraging Medallion architecture, Unity Catalog, and Delta Lake.

- Experience mentoring and scaling data engineering teams; excellent leadership and communication skills.

- Deep understanding of big data principles, data lakehouse architectures, and cloud ETL orchestration.

- Hands-on with Azure Data Lake Storage (ADLS), Data Factory (ADF), Synapse Analytics, Key Vault, Sql Server and Azure Monitor.

- Skilled in CI/CD pipelines, automation with Git and Azure DevOps, and code quality enforcement.

- Strong analytical mindset with a strategic approach to problem-solving and stakeholder management.

- Certifications in Azure/Data Engineering (Databricks, Azure) are highly desired.

- Proven ability to consult on data strategy and architecture for large-scale digital transformation initiatives.

- Strong knowledge of data preparation, feature engineering, and optimization for ML and AI workloads.

- Experience collaborating cross-functionally with data science, analytics, and business teams on ML/AI use cases.

Life at Next :

- At our core, we're driven by the mission of tailoring growth for our customers by enabling them to transform their aspirations into tangible outcomes.

- We're dedicated to empowering them to shape their futures and achieve ambitious goals.

- To fulfil this commitment, we foster a culture defined by agility, innovation, and an unwavering commitment to progress.

- Our organizational framework is both streamlined and vibrant, characterized by a hands-on leadership style that prioritizes results and fosters growth.

Perks of working with us :

- Clear objectives to ensure alignment with our mission, fostering your meaningful contribution.

- Abundant opportunities for engagement with customers, product managers, and leadership.

- You'll be guided by progressive paths while receiving insightful guidance from managers through ongoing feedforward sessions.

- Cultivate and leverage robust connections within diverse communities of interest.

- Choose your mentor to navigate your current endeavors and steer your future trajectory.

- Embrace continuous learning and upskilling opportunities through Nexversity.

- Enjoy the flexibility to explore various functions, develop new skills, and adapt to emerging technologies.

- Embrace a hybrid work model promoting work-life balance.

- Access comprehensive family health insurance coverage, prioritizing the well-being of your loved ones.

- Embark on accelerated career paths to actualize your professional aspirations.

Who we are?

- We enable high growth enterprises build hyper personalized solutions to transform their vision into reality.

- With a keen eye for detail, we apply creativity, embrace new technology and harness the power of data and AI to co-create solutions tailored made to meet unique needs for our customers.