Job Description - Data Engineer / Data Operations Engineer

Experience : 3-6 Years

Location : Remote

Employment Type : Full-Time

About the Role :

We are seeking a highly skilled and detail-oriented Data Engineer / Data Operations Engineer to join our growing data team. The ideal candidate will possess strong expertise in data pipeline operations, workflow monitoring, cloud-based data delivery systems, and production support. This role requires a hands-on professional who can efficiently manage high-volume data delivery operations while collaborating directly with clients to troubleshoot and resolve critical data issues.

The candidate should be comfortable working with both small-scale and extremely large datasets, ensuring seamless and reliable data movement across multiple client environments.

Key Responsibilities :

- Monitor, manage, and support production workflows using Apache Airflow and automated job scheduling systems.

- Ensure successful and timely delivery of data feeds from AWS environments to client destinations including Amazon S3, SFTP, and other cloud-based platforms.

- Troubleshoot and resolve data delivery failures, workflow interruptions, and production incidents with minimal downtime.

- Collaborate directly with clients and internal stakeholders to investigate and resolve operational issues through calls, meetings, and technical discussions.

- Work extensively with structured and unstructured datasets, including handling very large data files ranging up to multiple terabytes.

- Maintain operational excellence across high-volume delivery systems managing hundreds of data deliveries monthly.

- Support and optimize data pipeline operations to improve reliability, scalability, and performance.

- Develop and execute SQL and Python-based scripts for validation, troubleshooting, and operational automation.

- Ensure adherence to data governance, security, and operational best practices.

- Participate in root cause analysis and implement preventive measures for recurring production issues.

Required Skills & Qualifications :

- 3-6 years of experience in Data Engineering, Data Operations, or Production Support environments.

- Strong hands-on experience with Apache Airflow workflow orchestration and job monitoring.

- Expertise in AWS S3, cloud-based data storage, and file transfer mechanisms such as SFTP.

- Proficiency in Python and SQL for scripting, automation, and troubleshooting.

- Strong understanding of data pipelines, ETL/ELT processes, and production workflow management.

- Experience handling large-scale datasets and high-volume data delivery operations.

- Excellent analytical, troubleshooting, and problem-solving abilities.

- Strong communication skills with the ability to interact directly with clients and business stakeholders.

- Ability to work independently in a remote environment while managing multiple priorities effectively.

Preferred Qualifications :

- Exposure to distributed data processing frameworks and cloud-native data platforms.

- Familiarity with monitoring, logging, and alerting tools in production environments.

- Understanding of data security, compliance, and operational best practices.