HamburgerMenu
hirist

Lead Data Engineer - Python/SQL

HyrMe
10 - 15 Years
Bangalore

Posted on: 20/07/2026

Job Description

Experience:

- 10+ years of experience in designing, developing, and operating large-scale backend or data-intensive systems.

- Proven experience owning and managing mission-critical data platforms in production environments.

- Demonstrated success operating at a Senior or Staff Engineer level, delivering high-impact technical solutions.

- Strong understanding of AI-native software development practices, including AI-assisted code generation, intelligent testing, and human-in-the-loop review processes.

- Experience driving modern engineering practices by integrating AI into development workflows to improve engineering productivity, software quality, and delivery speed.

Technical Skills:

- Strong proficiency in Python and SQL.

- Extensive hands-on experience with distributed data processing and streaming technologies, including:

1. Apache Spark

2. Apache Kafka

3. Apache Flink

4. Apache Hive

5. Trino/Presto

- Deep understanding of distributed systems, including scalability, data consistency, fault tolerance, and high availability.

- Experience designing and operating cloud-native data platforms on Amazon Web Services (AWS).

- Strong knowledge of data modeling, ETL/ELT architecture, query optimization, and performance tuning.

- Familiarity with data warehousing, data lakes, and modern analytics architectures.

Engineering Mindset:

- Comfortable working in fast-paced, ambiguous environments while making strategic technical decisions.

- Strong expertise in system architecture, data modeling, and designing scalable, maintainable solutions.

- Ability to balance short-term delivery with long-term architectural goals.

- Proven ability to influence engineering teams, drive technical direction, and mentor peers without formal authority.

- Passion for engineering excellence, operational reliability, and continuous improvement.

AI-Enabled Development:

- Hands-on experience using AI-powered engineering tools such as GitHub Copilot, ChatGPT, Cursor, or similar platforms to accelerate software design, development, debugging, and documentation.

- Strong ability to evaluate, validate, and optimize AI-generated code for correctness, security, scalability, and performance.

- Experience incorporating AI into software development workflows to enhance developer productivity and software quality.

- Understanding of prompt engineering and best practices for responsible AI-assisted software development.

Preferred Qualifications:

- Experience building large-scale data platforms supporting analytics, machine learning, or AI workloads.

- Knowledge of modern data lakehouse technologies such as Delta Lake, Apache Iceberg, or Apache Hudi.

- Experience with Infrastructure as Code (Terraform, CloudFormation), containerization (Docker), and orchestration platforms (Kubernetes).

- Familiarity with CI/CD pipelines, observability, monitoring, logging, and incident management.

- Excellent communication, collaboration, and stakeholder management skills across cross-functional teams.

info-icon

Did you find something suspicious?

Similar jobs that you might be interested in

Loading chat...