Job Summary:

We are looking for an experienced Senior Data Engineer to design, develop and maintain scalable data pipelines, data platforms and cloud-based data infrastructure. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, big data technologies, cloud platforms and data warehousing.

Key Responsibilities:

- Design, develop and maintain scalable data pipelines and ETL/ELT workflows.

- Build and optimize data processing frameworks for structured and unstructured data.

- Develop data integration solutions across multiple internal and external systems.

- Work with large datasets using distributed computing technologies.

- Design and maintain data warehouses, data lakes and lakehouse architectures.

- Implement batch and real-time data processing pipelines.

- Optimize data storage, processing performance and query execution.

- Collaborate with Data Scientists, AI Engineers, Business Analysts and Software Engineers.

- Ensure data quality, governance, security and compliance.

- Implement automated data pipeline monitoring and error-handling mechanisms.

- Manage cloud-based data infrastructure and deployment workflows.

- Support AI/ML teams by providing reliable, high-quality data pipelines.

Required Technical Skills:

- Programming: Python, SQL, PySpark, Scala

- Big Data Technologies: Apache Spark, Hadoop, Kafka

- ETL/ELT Tools: Apache Airflow, dbt, Informatica

- Cloud Platforms: AWS, Microsoft Azure, GCP

- Data Warehousing: Snowflake, Amazon Redshift, Google BigQuery, Azure Synapse

- Databases: PostgreSQL, MySQL, MongoDB

- Data Lakes: Delta Lake, Apache Iceberg, AWS S3

- Data Processing: Batch Processing, Stream Processing, Real-Time Analytics

- DevOps & Deployment: Docker, Kubernetes, CI/CD, Terraform

- Data Governance: Data Quality, Data Lineage, Metadata Management

Experience & Qualifications:

- 6 - 11 years of overall experience in Data Engineering or related fields.

- Minimum 4 - 5 years of hands-on experience in data engineering.

- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering or a related field.

Preferred Certifications:

- AWS Certified Data Engineer - Associate

- Google Professional Data Engineer

- Microsoft Certified: Fabric Data Engineer Associate

- Snowflake SnowPro Core Certification

- Databricks Certified Data Engineer Associate