Apps Associates - Data Engineer - ETL/ELT Frameworks

About the Role :

We are seeking a highly skilled Data Engineer to design, build, and maintain scalable data pipelines and platforms that enable robust analytics, reporting, and application data flows.

The ideal candidate has strong expertise in distributed systems, cloud data engineering, and modern data stack technologies, with hands-on experience in building reliable and performance-optimized data solutions.

Key Responsibilities :

- Design, build, and optimize end-to-end data pipelines for batch and real-time data processing.

- Develop scalable data integration solutions using Python/Java/Scala/Golang or other high-performance languages.

- Architect and manage data workflows using ETL/ELT frameworks, workflow schedulers, and orchestration tools.

- Implement and maintain API-driven data services (REST, GraphQL, gRPC); work with microservices and backend systems.

- Build and optimize data models (star schema, snowflake, wide tables) for analytics and operational workloads.

- Work with cloud platforms (AWS / Azure / GCP) for data storage, transformation, and pipeline orchestration.

- Develop and maintain CI/CD pipelines, infrastructure-as-code, and automated deployment frameworks.

- Implement strong monitoring, logging, and alerting for data pipelines using modern DevOps & observability tools.

- Ensure data quality, data governance, security, and compliance across platforms.

- Collaborate closely with data scientists, platform engineers, analysts, and product teams to support data UI, ML, and reporting workstreams.

- Optimize performance of RDBMS, NoSQL, streaming systems, and distributed compute engines.

Required Technical Skills :

- Core Engineering Strong programming proficiency in Python, Java, Scala, or Golang.

- Experience building high-performance services and backend systems. Knowledge of API architecture: REST, GraphQL, gRPC.

- Hands-on experience with Spark, Kafka, Flink, Beam, or other distributed data frameworks.

- Expertise in ETL/ELT tools (Airflow, dbt, AWS Glue, Informatica, Talend, etc.).

- Strong understanding of data warehouses & data lakes (Snowflake, Redshift, BigQuery, Databricks, Lakehouse architectures).

- Experience with RDBMS and NoSQL databases: PostgreSQL, MySQL, MongoDB, Cassandra, DynamoDB, ElasticSearch. Knowledge of data modeling techniques, OLAP/OLTP systems.

- Cloud & DevOps Solid experience in AWS / Azure / GCP cloud data ecosystems.

- Familiarity with CI/CD pipelines using Jenkins, GitLab, GitHub Actions.

- Experience with IaC tools like Terraform / CloudFormation.

- Exposure to containerized deployments using Docker / Kubernetes.

Other Skills :

- Strong analytical and problem-solving abilities. Experience defining data standards, governance, and best practices.

- Ability to optimize SQL queries and distributed jobs for performance and cost efficiency.

Preferred Qualifications :

- 10-12 years of professional experience as a Data Engineer or similar role.

- Experience with real-time & event-driven architectures.

- Background in building API-driven data services and microservices.

- Experience working on large-scale, high-traffic enterprise systems.