Lumina Datamatics - Senior Data Engineer

Description :

Job Title : Senior Data Engineer

Experience : 5-7 Years

We are looking for a hands-on Data Engineer to design, build, and maintain reliable data pipelines and support our data warehouse environment. The ideal candidate should have strong SQL and Python skills, solid ETL experience, and practical exposure to data warehousing concepts.

Key Responsibilities :

SQL & Database Management :

- Write optimized SQL queries.

- Work with relational databases (PostgreSQL/MySQL/SQL Server or similar).

- Intermediate understanding of NoSQL databases.

ETL & Pipeline Development :

- Develop and maintain ETL/ELT pipelines using Python and SQL.

- Integrate data from structured and semi-structured sources.

- Perform data transformations and cleansing logic.

- Ensure pipelines are reliable and scalable.

Data Warehousing :

- Strong understanding of data warehousing concepts and best practices.

- Work with dimensional modeling techniques including Star and Snowflake schemas.

- Design and maintain fact and dimension tables based on business requirements.

- Support performance tuning activities including indexing, partitioning, and query optimization.

- Implement data validation and error-handling mechanisms within data pipelines.

Data Quality & Error Handling :

- Implement validation checks and reconciliation logic.

- Build basic error-handling and logging mechanisms.

- Monitor and troubleshoot pipeline failures.

- Exposure to monitoring/alerting tools (Prometheus, Stackdriver, etc.)

Orchestration & Deployment :

- Use workflow orchestration tools (e.g., Airflow or equivalent).

- Work with Git for version control.

- Exposure to Docker.

- Exposure to CI/CD concepts.

- Support production deployments (no heavy DevOps ownership expected).

Cloud Exposure (Preferred) :

- Exposure to any cloud platform - GCP preferred.

- Familiarity with BigQuery.