AWS Data Engineer, Pune

Company Overview :

Volto Consulting is a dynamic and rapidly growing technology consulting firm specializing in cutting-edge data engineering and cloud solutions. We empower our diverse clientele, spanning across various industries, to unlock the strategic value of their data through robust, scalable, and innovative platforms. Our focus is on delivering transformative data architectures and analytics capabilities that drive significant business outcomes for our partners.

Role Overview :

As an AWS Data Engineer at Volto Consulting in Pune, you will be instrumental in designing, developing, and maintaining sophisticated data pipelines and platforms on the Amazon Web Services (AWS) ecosystem. This role involves working closely with cross-functional teams, including data scientists, business analysts, and client stakeholders, to translate complex business requirements into high-performance, scalable data solutions. You will play a critical role in shaping our clients' data strategy, ensuring data quality, accessibility, and reliability, ultimately enabling data-driven decision-making and fostering innovation.

Key Responsibilities :

- Architect, design, and implement robust, scalable, and cost-efficient data pipelines and ETL/ELT processes on AWS, leveraging services such as S3, Glue, Lambda, Redshift, Kinesis, and EMR.

- Develop and optimize large-scale data processing jobs using PySpark and Python within Databricks environments to handle diverse data volumes and velocity.

- Build and manage data lakes and data warehouses, ensuring data integrity, security, and optimal performance for analytical and reporting needs.

- Collaborate with data scientists and business intelligence teams to understand data requirements and deliver datasets that support advanced analytics, machine learning models, and business insights.

- Implement comprehensive data governance, quality, and security frameworks to ensure compliance and maintain high standards of data reliability.

- Monitor, troubleshoot, and optimize existing data infrastructure and processes, identifying areas for improvement in efficiency, cost, and performance.

- Contribute to architectural discussions, evaluate new technologies, and mentor junior engineers, fostering a culture of technical excellence and continuous learning.

Required Skillset :

- Demonstrated expertise, typically acquired over 6-12 years, in designing and implementing scalable data solutions on AWS, encompassing services like S3, Glue, Lambda, Redshift, Kinesis, and EMR.

- Strong proficiency in Databricks for data processing, analytics, and collaborative data science workflows.

- Advanced capabilities in PySpark for developing and optimizing large-scale data transformation and aggregation jobs.

- Exceptional Python programming skills, with a focus on building robust and efficient data engineering applications and automation scripts.

- Deep understanding of data warehousing concepts, dimensional modeling, and various ETL/ELT methodologies.

- Experience with CI/CD pipelines for data solutions, ensuring automated deployment and testing practices.

- Solid command of SQL for data querying, manipulation, and database management.

- Excellent problem-solving and analytical abilities, with a proactive approach to identifying and resolving complex data challenges.

- Strong communication and interpersonal skills, capable of articulating technical concepts clearly to both technical and non-technical stakeholders and collaborating effectively within a team and with clients.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, providing a strong foundation in computational principles.

- Ability to thrive in a dynamic, fast-paced consulting environment, working on-site in Pune and adapting to evolving project requirements.