NewVision Software - Senior Data Engineer - Python/PySpark

Description :

We are hiring on an immediate basis for the role of Senior Data Engineer | NewVision Software | Pune, Hyderabad & Bhopal Locations | Fulltime

Looking for professionals who can join us Immediately or within 15 days is preferred.

Please find the job details and description below.

Role Overview : -

We are looking for a skilled Senior Data Engineer with strong experience in Databricks, PySpark, SQL, and Python. The ideal candidate will be responsible for building scalable data pipelines, optimizing data processing workflows, and enabling data-driven decision-making across the organization.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using PySpark and Databricks

- Develop and optimize complex SQL queries for data transformation and analysis

- Build and manage ETL/ELT workflows for structured and unstructured data

- Work with large-scale distributed data processing systems

- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements

- Ensure data quality, integrity, and consistency across pipelines

- Implement performance tuning and optimization techniques in Databricks

- Develop reusable and efficient Python-based data processing modules

- Work on data ingestion from multiple sources (APIs, databases, files, streaming, etc.)

- Maintain proper documentation and follow data engineering best practices

Required Skills :

- Strong experience in Python and PySpark

- Hands-on experience with Databricks platform

- Advanced knowledge of SQL

- Experience in building and optimizing data pipelines and workflows

- Good understanding of data modeling concepts (Star/Snowflake schemas)

- Familiarity with big data technologies and distributed computing

- Experience with data lakes / lake house architecture

- Understanding of performance tuning and job optimization

Preferred Skills :

- Experience with cloud platforms (Azure / AWS / GCP)

- Knowledge of Delta Lake

- Exposure to CI/CD pipelines for data engineering

- Experience with orchestration tools (Airflow, ADF, etc.)

- Basic understanding of data governance and security

Soft Skills :

- Strong problem-solving and analytical thinking

- Good communication and stakeholder management skills

- Ability to work in a fast-paced, collaborative environment

Nice to Have :

- Experience working in Agile/Scrum environments

- Exposure to real-time/streaming data processing