Data Engineer

Experience : 7 - 15 Years

Employment Type : Full-Time

About the Role :

We are looking for an experienced Data Engineer to design, build and maintain scalable data pipelines and data platforms that enable analytics, reporting, AI/ML and data-driven decision-making. The ideal candidate should have strong experience working with large datasets, cloud technologies, data warehouses and enterprise data integration.

Key Responsibilities :

- Design, develop and maintain scalable and reliable data pipelines.

- Build and optimize ETL/ELT processes to integrate data from multiple sources.

- Extract and transform data from ERP, CRM, databases, APIs, IoT and other enterprise systems.

- Develop high-performance SQL queries, data transformations and stored procedures.

- Design and maintain data warehouses, data lakes and lakehouse architectures.

- Work with structured, semi-structured and unstructured data.

- Implement data pipelines using technologies such as Apache Spark, PySpark, Airflow, Kafka and cloud-native services.

- Develop data ingestion and integration frameworks for batch and real-time data.

- Ensure data quality, accuracy, consistency, security and availability.

- Monitor, troubleshoot and optimize data pipelines and workflows.

- Optimize data processing, storage and query performance.

- Collaborate with Data Scientists, AI/ML Engineers, BI Developers, Software Engineers and business stakeholders.

- Support data requirements for AI/ML models, analytics and business intelligence.

- Implement data governance, metadata management and access-control practices.

- Develop technical documentation for data architecture, pipelines and processes.

- Support migration of legacy data platforms to modern cloud-based data architectures.