- Develop and maintain data pipelines for ingestion, transformation, and integration across multiple platforms.
- Work with structured and semi-structured data such as CSV, JSON, Parquet, APIs, and databases.
- Design and implement data models, ETL/ELT workflows, and validation processes.
- Build and maintain data warehouses and data lakes for analytics and reporting.
- Prepare, optimize, and deliver datasets for Power BI dashboards and reports.
- Write efficient SQL queries and Python scripts for data extraction, transformation, and loading.
- Debug and optimize stored procedures (SPs) for performance and reliability.
- Collaborate with BI developers to ensure data pipelines meet reporting and analytics requirements.
- Support data governance, quality, and security initiatives.
- Participate in testing, debugging, and documenting data workflows.
- Apply performance tuning techniques such as indexing, execution plan analysis, and query optimization.
- Utilize SQL features such as CTEs, joins, dynamic queries, window functions (LEAD, LAG), and understand differences between temporary and physical tables.
- Continuously explore and adopt emerging tools, frameworks, and best practices in data engineering and cloud technologies.
Shape
Desired Profile - Bachelors degree in computer science, Engineering, or related discipline.
- 2-3 years of experience in data engineering or related role (internship/project experience acceptable).
- Strong knowledge of SQL and relational databases.
- Hands-on experience with Python for data handling (Pandas preferred).
- Exposure to Apache Spark or similar distributed processing frameworks.
- Familiarity with at least one cloud platform (Azure or AWS) and eagerness to learn both.
- Exposure and hands-on experience in Snowflake for data warehousing is a strong plus.
- Experience in preparing data models and datasets for Power BI dashboards.
- Solid understanding of data pipelines, ETL/ELT processes and data integration frameworks.
- Experience with relational databases (e.g., SQL Server, PostgreSQL, MySQL)
- Good problem-solving skills, attention to detail, and eagerness to learn.
- Strong communication skills and ability to work in collaborative teams.