Data Engineer

Experience :

- 8+ years of hands-on experience in data engineering, with at least 4 years in a lead or architect-level role.

Technical Skills :

- Deep expertise in Apache Spark, with proven experience developing large-scale distributed data processing pipelines.

- Strong experience with Databricks platform and its internal ecosystem (e.g., Delta Lake,Unity Catalog, MLflow, Job orchestration, Workspaces, Clusters, Lakehouse architecture).

- Extensive experience with workflow orchestration using Apache Airflow.

- Proficiency in both SQL and NoSQL databases (e.g., Postgres, DynamoDB, MongoDB, Cassandra) with a deep understanding of schema design, query tuning, and data partitioning.

- Proven background in building data warehouse/data mart architectures using AWS services like Redshift, Athena, Glue, Lambda, DMS, and S3.

- Strong programming and scripting ability in Python (preferred) or other AWS-compatible languages.