Senior Data Engineer - Python/SQL/PySpark

Description:

Role : Senior Data Engineer

Location : Bangalore

Exp : 8-11 Years

NP : Max 30 Days

What were looking for :

- 8 - 11 years of Professional experience designing/building/maintaining highly available data and analytics platform.

- 5+ years of experience in data engineering, with a focus on building large-scale data processing systems.

- Hands-on experience with AWS or similar cloud platform building data engineering solutions for analytics and science. (4+ years)

- Must have experience building complex data pipelines - batch and/or real time event-based processing (3+ years)

- Strong experience in designing, building, and maintaining data warehouse in Redshift or similar cloud-based solutions. (3+ years)

- Experience building complex ETL using Spark (Scala or Python) for event based big data processing (3+ years)

- Experience in Matillion or similar ETL/ELT tool for developing data ingestion and curation flow (3+ years)

- Must have strong hands-on experience in SQL. (5+ years)

- Strong hands-on experience in modern scripting languages using Python. (5+ years)

- Strong hands-on experience with NoSQL DBs - MongoDB, Cassandra or DynamoDB (5+ years)

What youll do :

- Build a deep understanding of existing systems.

- Participate in or lead design reviews with peers and stakeholders.

- Develop robust, testable code that meets design requirements.

- Review code developed by other developers, providing feedback on style, functional correctness, testability, and efficiency.

- Triage system-wide issues and identify root cause of incidents.

- Can work independently and can participate/contribute to architecture discussions.

- Identify and resolve existing critical technical debt.

- Build transparent systems with proper monitoring, observability, and alerting.

- Plan for robust build, test, and deployment automation

- Work with product stakeholders and front-end developers to understand the essence of requirements and to provide pragmatic solutions

- Work within an Agile framework