Senior Big Data Architect - Scala/Python

Job Summary :

Roles & Responsibilities :

Core Responsibilities :

- Data Architecture Design : Design and implement scalable and fault-tolerant data architecture for batch and streaming systems.

- Data Modeling : Develop efficient data models and database structures to support analytics

and business processes.

- Data Integration : Integrate data from various structured and unstructured sources, including legacy systems.

- Performance Optimization : Monitor and fine-tune data infrastructure for optimal performance.

- Data Security & Governance : Ensure data security, compliance, and governance across platforms.

- Collaboration : Work closely with data scientists, engineers, and business teams to gather requirements and deliver solutions.

- Technology Evaluation : Research, evaluate, and recommend emerging data technologies and tools.

- Mentorship : Mentor junior team members and promote best practices in data engineering and architecture.

- Problem Solving : Diagnose complex data challenges and provide robust solutions.

- Pre/Post Sales Support : Engage in client interactions to support technical sales, solutioning, and delivery.

Experience Required :

- Total Experience : 10 - 12 years in IT, with at least 2+ years as a Big Data Architect.

- Big Data Technologies : Hands-on experience with Hadoop, Spark, Hive, HBase, Kafka, etc.

- Programming : Proficient in Python, Java, Scala, and Spark.

- Cloud Platforms : Experience with AWS, Azure, or GCP-based data services (e.g., EMR,

Databricks, BigQuery).

- Database Expertise : Strong knowledge of SQL and NoSQL databases (MongoDB, Cassandra,

etc.).

- ETL Pipelines : Proven experience designing and implementing scalable ETL workflows.

- Security & Authentication : Understanding of LDAP, Kerberos, Active Directory, SAML

configuration in distributed systems.

Additional Skills :

- Data Visualization : Familiarity with visualization tools (Tableau, Power BI, or similar).

- DevOps & Automation : Experience with CI/CD, containerization (Docker/Kubernetes), and automation tools.

- Soft Skills : Strong communication, collaboration, and stakeholder management abilities.

- Compliance & Governance : Knowledge of data privacy laws and regulatory requirements

(GDPR, HIPAA, etc.).