Description :

- Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.

- Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect.

- Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.

- Should be able to suggest optimizations in existing designs.

- Project documentation adheres to quality guidelines & schedules.

- Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.

- Articulate design decisions very clearly with stakeholders.

- Perform Proof-of-Concepts and document all observations before proposing a new solution.

- Conduct design review sessions with other teams and identify scope of improvement.

- Suggests automation to improve existing processes.

Work and Technical Experience :

Must-Have :

- In-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda.

- Experience with at least one MPP database : AWS Redshift, Snowflake, SingleStore.

- Proficiency in Big Data technologies : Apache Spark, Databricks.

- Must have strong programming skills in Python.

- Responsible for building data pipelines in AWS And Databricks.

- Experience with Big Data table formats, such as Delta Lake (open source).

- Must have very strong SQL skills.

- Experience with orchestration tools like Apache Airflow.

- Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.

- Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required.

- Strong understanding of data warehousing concepts.

Good to have :

- Cloud Databases - Snowflake, AWS Aurora.

- Big Data - Hadoop, Hive.

- Cloud Databases- AWS Aurora.

- Associate Level or Professional Level AWS Certification or Databricks Certification.

Qualifications :

- Overall, 7+ years of IT Exp.

- 5+ years of relevant experience in AWS related project.

- Bachelors degree in computer science, engineering, or related field (master's degree is a plus).

- Demonstrated continued learning through one or more technical certifications or related methods.

Qualities :

- Hold strong technical knowledge and experience.

- Should have the capability to deep dive and research in various technical related fields.

- Self-motivated and focused on delivering outcomes for a fast-growing team and firm.

- Able to communicate persuasively through speaking, writing, and client presentations.

- Able to consult, write, and present persuasively.

- Able to work in a self-organized and cross-functional team.

- Able to iterate based on new information, peer reviews, and feedback.

- Prior experience of working in a large media company would be added advantage.