BigStep Technologies - Data Engineer - Python/SQL/ETL

Key Responsibilities :

- Build, scale, and maintain robust data solutions to support business objectives.

- Design, implement, and optimize high-performance data pipelines (extraction, loading, transformation, and orchestration) with scalability, reliability, and speed.

- Lead end-to-end software development projects involving large language models (LLMs), retrieval-augmented generation (RAG) frameworks, and other AI technologies.

- Champion modern software engineering practices such as CI/CD, infrastructure-as-code, containerization, and cloud-native deployments.

- Collaborate with business stakeholders to transform use cases into production-ready services and solutions, owning systems from concept to production.

- Implement rigorous testing and monitoring practices to ensure superior data quality and integrity.

- Mentor and guide junior team members, fostering a culture of excellence and continuous learning.

Education & Certifications :

- Bachelors degree or higher in a STEM field (Computer Science, Math, Physics, Engineering, or related), required.

- Advanced degree in Computer Science or related field, preferred.

Professional Experience :

- 5 to 7 years of total professional experience.

- 3 to 5 years of experience in Data Engineering or a related discipline with a proven track record of success.

- Experience in financial services or product development environments is a strong plus.

Technical Skills & Competencies :

- Proficiency with Databricks/PySpark and dbt for data warehousing and transformation.

- Hands-on experience with workflow orchestration tools (e.g., Airflow, Temporal).

- Experience working with LLMs, prompt engineering, RAG frameworks, and vector databases.

- Knowledge of machine learning fundamentals, feature engineering, and knowledge graphs is an advantage.

- Strong experience in designing and implementing complex data systems from scratch.

- Ability to handle large-scale data projects including ETL, data cleaning, and information retrieval.

- Strong communication skills (verbal and written).