Role : Senior Data Engineer

Location : Pune Onsite

Full time permanent position with Vivid Edge Corp.

We have 2 positions Open (Jr 4 yrs exp & Sr. 6 yrs+ exp).

Note : For Senior role At least 6+ years of development experience as Data Engineer.

Responsibilities :

- Development, maintenance, and enhancement of Data Pipelines (ETL/ELT) and processes with thorough knowledge of star/snowflake schemas.

- Developing complex SQL queries and SQL optimization.

- Development experience must be full Life Cycle experience including business requirements gathering, data sourcing, testing/data reconciliation, and deployment within Business Intelligence/Data Warehousing Architecture.

- Designing and implementing data security.

- Monitoring and optimizing data storage and data processing.

- Use of Object function/object-oriented scripting languages including Scala, C++, Java and Python.

- Cloud data warehouse and data lake platforms (e.g. BigQuery, MS Synapse, MS ADLS, Apache Haddop, etc.

- Developing data services that are fit for purpose, resilient, scalable, and future proof, to meet user needs.

- Advanced SQL knowledge and experience working with relational databases and working familiarity with various databases.

- Demonstrated understanding and experience using software and tools including ETL, relational SQL and NoSQL databases, big data tools like Kafka, Spark, and Hadoop.

- Experience building and optimizing big data data pipelines, architecture, and data sets.

- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

- Strong analytic skills related to working with unstructured datasets.

- Experience and knowledge of project management best practices and agile software development methodologies.

- A successful history of manipulating, processing, and extracting value from large, disconnected datasets.

- Working knowledge of message queuing, stream processing, and highly scalable big data data stores.

- Strong project management and organizational skills.

- Experience supporting and working with cross-functional teams in a dynamic environment.

- Understanding of data science concepts.

- Familiarity with AI/ML frameworks and libraries.