Description :

Senior Data Engineer (5+ years experience)

Tech Stack :

- Data Analysis, Data Modelling, Python, SQL

OR

- Python (OOPS), Airflow, great expectation, Snowflake

Must have :

- Hands on experience with Airflow

- Python experience with more focus on OOPS and building backend application or SDK

- Good understanding of Snowflake concepts

Nice to have :

- Experience with performing Data Quality on large dataset or building DQ framework

- Experience on Snowflake

Key Responsibilities :

- Pipeline Development : Build, test, and maintain robust, scalable data pipelines to ingest, transform, and load (ETL) data from diverse sources.

- Infrastructure Management : Design, construct, and manage data warehouses, data lakes, and databases (SQL and NoSQL).

- Data Quality & Security : Ensure data accuracy, reliability, and security by implementing validation methods, monitoring tools, and access controls (RBAC).

- Optimization : Monitor system performance, troubleshoot issues, and optimize data processing workflows for efficiency.

- Collaboration : Work with data scientists and analysts to understand requirements and provide reliable data for predictive modeling and analysis

Required Skills and Qualifications :

- Programming : Proficiency in Python, Java, or Scala.

- Database Expertise : Advanced SQL skills and experience with relational/NoSQL databases (PostgreSQL, MongoDB).

- Data Architecture : Knowledge of data modeling, data warehousing concepts, and cloud storage (e.g., S3).

- Big Data Tools : Familiarity with Apache Spark, Hadoop, or Kafka.

- Cloud Platforms : Hands-on experience with AWS, Azure, or Google Cloud Platform.

- Experience : Bachelors degree in Computer Science, IT, or related field, with 5+ years of experience typically required.