Position : Lead Python Data Engineer

Experience : 8+ Years

Job type : Remote, India

Job base location : Noida (Delhi, NCR)

Role Overview :

We are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL, and Power BI. This role requires a proactive individual who can work independently and collaboratively in a fast-paced environment.

Required Technical Skills :

- 7+ years of hands-on experience in software development, with a strong emphasis on Python programming.

- Deep expertise in PySpark (Spark SQL, DataFrames, and Datasets) for building and optimizing scalable data pipelines. (PySpark)

- Solid experience with Azure Databricks (Unity Catalog), including working with Delta tables, workflows, and scripting in Python.

- Understanding of data governance, data management, security, and compliance in cloud environments.

- Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory,

- Experience with Power BI, including DAX and Power Query.

- Modeling data for Data Warehousing and Visualization projects through Semantic Layers.

- Skilled in writing complex SQL queries and building views involving joins across multiple tables.

- Proficient in tools like DevOps (project management), and version control and CI/CD pipelines.

Nice to have Skills :

- Experience Azure AI, Azure Copilot Studio and Azure Fabric.

- Experience building workflows with Power Automate.

- EventHub or Kafka for streaming data ingestion.

- Proficient in designing and implementing real-time data streaming solutions.

- Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies.

Soft Skills Collaboration :

- Strong collaboration mindset with enthusiasm for mobbing sessions and cross - team coordination.

- Excellent problem-solving and code review skills.

- Strong communication skills with the ability to understand business requirements and translate them into technical solutions.

- Proactive and responsive during business hours for any ad-hoc calls or meetings.

- Demonstrated ability to prioritize effectively, think creatively, and adapt quickly to new technologies and feedback.

Candidates with strong experience in :

- Pyspark

- Dataframes

- Datasets

- Azure Databricks may also be considered.