Senior Data Engineer - Python/PySpark

Job Title : Senior Data Engineer

Location : Chennai (Hybrid 4 days from office per week)

Experience : 7+ Years

Notice Period : Immediate joiners only

Job Type : Full-time

Key Responsibilities :

- Collaborate with business and technology stakeholders to understand current and future data requirements.

- Design, build, and maintain reliable, efficient, and scalable data infrastructure for data collection, storage, transformation, and analysis.

- Plan, design, and develop scalable data solutions, including data pipelines, data models, and applications for efficient and reliable workflows.

- Design, implement, and maintain data platforms such as data warehouses, data lakes, and lakehouses for structured and unstructured data.

- Develop and optimize analytical tools, algorithms, and programs to support data engineering activities, including scripting and task automation.

- Monitor and ensure optimum system performance while identifying opportunities for continuous improvement.

Required Skills :

- BigQuery and Google Cloud Platform (GCP) (BigQuery, Dataflow, Dataproc, Data Fusion, Cloud SQL).

- Airflow, PySpark, Python, and API development.

- Terraform and Tekton for infrastructure as code and CI/CD automation.

- Strong knowledge of PostgreSQL and SQL optimization.

Preferred Skills :

- Experience with large-scale data solutions in BigQuery, GCP, Airflow, Tekton, and Terraform.

- Strong background in building end-to-end pipelines and productionizing workflows.

Qualifications :

Education Required : Bachelors Degree in Computer Science, Information Technology, or related field.

Education Preferred : Advanced degree in a relevant discipline (optional).