Nextpathway | The Automated Cloud Migration Company

Next Pathway is a technology services company providing clients a pathway from existing to emerging technologies. Our automation technology helps our customers accelerate the migration of complex applications and workloads to the cloud.

About the Role :

Next Pathway is looking for a highly skilled and experienced Senior Engineer to join our team, specializing in large-scale cloud migration projects. The ideal candidate will have a minimum of 10 years of experience in data engineering and ETL, with a strong background in migrating on-premise data systems to the cloud. This role requires a hands-on professional who can design, develop, and implement complex data pipelines, working independently and as a key contributor within a project team.

Key Responsibilities :

- Development and Implementation : Design, develop, and deploy robust and scalable data pipelines for migrating data from various on-premise sources to cloud platforms.

- Technical Expertise : Serve as a subject matter expert in the project's core technologies, including Datastage, Azure Data Factory (ADF), Databricks, and PySpark.

- Solution Design : Collaborate with architects and technical leads to design efficient and effective cloud migration strategies and data transformation solutions.

- Code Quality : Ensure the delivery of high-quality, well-documented, and maintainable code. Participate in code reviews and provide constructive feedback to peers.

- Performance Optimization : Analyze and optimize the performance of data pipelines and applications to ensure efficiency and scalability.

- Problem-Solving : Troubleshoot and resolve complex technical issues related to data migration, data quality, and system performance.

- Collaboration : Work closely with cross-functional teams, including business analysts, data scientists, and other engineers, to understand requirements and deliver successful outcomes.

- Mentorship : Provide guidance and technical support to junior engineers, helping them grow their skills and contribute effectively to the team.

Required Skills and Qualifications :

- Minimum of 10 years of experience in the IT industry, with a focus on data engineering and ETL development.

- Extensive hands-on experience with Datastage for building and managing ETL jobs.

- Proven experience in Cloud Migration projects, with a strong understanding of the challenges and best practices involved in moving data to the cloud.

- In-depth knowledge and hands-on experience with Azure Data Factory (ADF) for building and orchestrating data workflows.

- Strong proficiency in Databricks and its ecosystem, including notebooks, jobs, and Delta Lake.

- Expertise in PySpark for developing data processing applications on a distributed computing framework.

- Solid understanding of cloud platforms, preferably Microsoft Azure.

- Experience with SQL and database concepts.

- Excellent analytical, problem-solving, and communication skills.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.