Big Data Engineer - SQL/Python

Job Description :

Responsibilities :

- Strong awareness and hands-on experience with various data tools and platforms, including Hive, Hadoop, Starburst, and other Big Data technologies.

- Proficiency in programming languages such as SQL, Python, Spark, and Java.

- Good working knowledge and practical experience with ETL processes utilizing Java Spark, Python Spark, or Spring Batch.

- Advanced SQL skills and extensive experience with relational databases and database design principles.

- Strong proficiency in object-oriented programming languages, specifically Python, PySpark and Java, Java Spark (a must-have).

- Demonstrable experience working with data Lakehouse architectures, such as Iceberg/Starburst.

- Strong proficiency in scripting languages like Bash for automation and system administration tasks.

- Strong proficiency in designing, implementing, and managing data pipeline and workflow management tools (e.g., Apache Airflow, Luigi, etc.).

- Knowledge of cloud platforms such as Databricks, AWS (Amazon Web Services), or GCP (Google Cloud Platform) is a significant plus.