About QX :

The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation.

We believe without insights, businesses will continue to face challenges to better understand their customers and even lose them; Secondly, without insights businesses won't be able to deliver differentiated products/services; and finally, without insights, businesses cant achieve a new level of Operational Excellence is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.

Job Summary :

We are looking for a Lead Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.

As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.

Key Responsibilities :

- Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines, data workflows, and enterprise data architectures.

- Build and manage modern Lakehouse solutions using Microsoft Fabric, OneLake, Databricks, Delta Lake (Bronze/Silver/Gold), and cloud-native data platforms.

- Develop and orchestrate data pipelines using Azure Data Factory/Fabric Data Factory, Spark/PySpark, and other big data frameworks to support batch and real-time data processing.

- Design and implement robust data integration solutions by connecting ERP systems (SAP, NetSuite, Business Central, etc.) and other enterprise applications through APIs, batch ingestion, and cloud services.

- Architect and optimize data lakes, data warehouses, and semantic data models to enable business intelligence, advanced analytics, predictive modeling, and AI-driven solutions.

- Develop high-quality, AI-ready datasets to support Machine Learning, RAG, and Generative AI applications.

- Enhance cloud data platforms across Azure, AWS, Snowflake, Synapse, and Microsoft Fabric to ensure scalability, performance, security, and cost optimization.

- Implement data quality, governance, lineage, monitoring, observability, and testing frameworks to ensure reliable and trusted data pipelines.

- Build and maintain CI/CD pipelines using Git, Azure DevOps, GitHub Actions, and Infrastructure as Code (IaC) to automate deployments and platform management.

- Monitor, troubleshoot, optimize, and provide production support for data ingestion workflows and enterprise data platforms, ensuring high availability and operational excellence.

- Collaborate closely with business stakeholders, Subject Matter Experts (SMEs), data scientists, and cross-functional teams to gather requirements and deliver scalable data, BI, and AI solutions.

- Conduct code reviews, establish engineering best practices, drive technical innovation, and mentor junior engineers through technical guidance and knowledge sharing.

- Lead solution architecture, technical design reviews, sprint planning, effort estimation, stakeholder communication, and delivery governance to ensure successful project execution.

Required Skills & Qualifications:

- Bachelors degree in Computer Science, Mathematics, Engineering, or a related technical discipline.

- Over 10 years of experience in designing, building, and managing enterprise-scale data platforms, with a strong focus on data lake architecture and large-scale data ingestion.

- 57 years of hands-on experience in architecting and implementing data warehouse solutions, including schema design, performance tuning, and ELT processes.

- Advanced proficiency in SQL and deep understanding of data modelling and design principles, including normalization, denormalization, and dimensional modelling.

- Strong expertise in Apache Spark using Python or Scala and building real-time data pipelines with Spark Streaming in high-volume, low-latency environments.

- Proven experience working with Databricks, leveraging features such as Delta Lake, Auto Loader, Unity Catalog, and notebook-based development for scalable data engineering and analytics workflows.

- Proficiency in cloud platforms such as Azure and/or AWS, including services like AWS Glue, Amazon Redshift, Azure Data Factory, Azure Synapse, and Snowflake.

- Familiarity with big data and streaming technologies such as Apache Kafka, Apache Flink, and distributed compute engines for event-driven data processing.

- Hands-on experience with orchestration and workflow automation tools such as Apache Airflow, Prefect, or Azure Data Factory Pipelines, for managing complex dependencies in data workflows.

- Strong understanding of CI/CD practices, including version control with Git, automation tools like Jenkins, and infrastructure-as-code solutions such as Terraform or CloudFormation.

- Experience integrating data from ERP systems (e.g., SAP, NetSuite, Business Central) into data lakes, including handling business logic transformation and cross-system data alignment.

- Working knowledge of traditional ETL tools such as Talend and Pentaho, with the ability to modernize or migrate legacy ETL workloads to modern data architectures.

- Excellent problem-solving, communication, and collaboration skills, with the ability to lead cross-functional discussions and mentor junior engineers.

Competencies:

- Tech Savvy: Anticipating and adopting innovations in business-building digital and technology applications.

- Self-Development: Actively seeking new ways to grow and be challenged using both formal and informal development channels.

- Action Oriented: Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.

- Customer Focus: Building strong customer relationships and delivering customer-centric solutions.

- Optimize Work Processes: Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.

Why Join Us?

- Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.

- Work on impactful projects that make a difference across industries.

- Opportunities for professional growth and continuous learning.

- Competitive salary and benefits package.

Application Details:

Ready to make an impact? Apply today and become part of the QX Impact team!