NewVision Software - Senior Big Data Developer

Description :

About the role : - Senior BigData Engineer

IF YOU ARE HAVING :

- 4+ years of industrial experience in Big Data Engineering

- Strong proficiency in SQL and at least one programming language : Python, Scala, or Java

- Hands-on experience with AWS and Snowflake

- Strong Experience in Data Build Tool (DBT)

- Experience with cloud storages : AWS S3, Azure Storage, Google Cloud Storage

- Practical skills with Hadoop-like environments (EMR, HDInsight, Dataproc)

- Experience with Big Data frameworks : Apache Spark, Databricks, Apache Flink, Apache Beam

- Knowledge of orchestration tools : Airflow, NiFi, AWS Glue, Azure Data Factory, Cloud Composer

- Experience with streaming platforms : Kafka, Kinesis, SQS, Event Hub, Pub/Sub, IoT Hub

- Familiarity with data warehousing and analytics : Snowflake, Redshift, Athena, Synapse

- Experience working with containerization & serverless technologies (Docker, Lambda, Fargate,

Functions, Cloud Run)

- Understanding of data governance, data quality, and data lifecycle processes

- Strong communication and collaboration skills

- Bachelor's degree in Computer Science or similar (preferred)

- AWS or Snowflake certifications (preferred)

AND YOU WANT TO :

- Design and optimize distributed computing architectures for scalable Big Data processing

- Build and enhance batch and streaming data pipelines

- Implement advanced data processing : realtime analytics, ML-driven models, predictive insights

- Develop reusable, parameterized ETL/ELT pipelines for DWH and lakehouse solutions

- Conduct end-to-end data flow validation and performance optimization

- Implement best practices for data quality, lineage, and governance

- Configure and maintain CI/CD flows and release pipelines

- Work with containerized and multi-container development environments

- Troubleshoot complex data infrastructure and processing issues

- Contribute to architectural decisions and technology selection

- Ensure secure customer data management throughout the entire project lifecycle

- Follow data retention, deletion, and off-boarding procedures

- Mentor junior engineers and support technical decision-making