InsuranceDekho - Data Engineer - ETL/Python

Description :

Role and Responsibilities :

- Strong proficiency in Python for data processing and ETL

- Advanced SQL skills, including query optimization, indexing, joins, and analytical functions

- Hands-on experience with ClickHouse, MongoDB, Redis, and ElasticSearch

- Proficiency in Apache Spark/PySpark and working with data lakes

- Experience with ETL and data ingestion tools such as Apache NiFi

- Familiarity with messaging and streaming platforms like Kafka, RabbitMQ, and ActiveMQ

- Experience with workflow orchestration frameworks such as Apache Airflow

- Exposure to cloud platforms (AWS, GCP, or Azure) and their data services (S3, Redshift, BigQuery, Dataproc, etc.)

- Understanding of data warehousing, data modeling, and performance optimization techniques