About the job

Job Title : Data Quality Engineer

Location : Chennai, India (Offshore)

Experience : 5 - 8+ Years (Preferred)

Joining : Immediate Joiners Only

Job Summary :

We are looking for an experienced Data Quality Engineer with strong expertise in SQL, Databricks, data quality frameworks, ETL/ELT pipelines, and data observability. The ideal candidate will be responsible for ensuring the accuracy, consistency, completeness, reliability, and integrity of enterprise data across cloud-based data platforms. The candidate will work closely with Data Engineers, Data Architects, Data Governance teams, and business stakeholders to implement data quality validation frameworks, monitor data pipelines, identify data anomalies, and resolve data-related issues. Hands-on experience with Monte Carlo Data Observability, Azure data platforms, and Collibra is highly preferred.

Key Responsibilities :

1. Data Quality Engineering :

- Design, develop, and implement comprehensive data quality validation frameworks for enterprise data platforms.

- Define and implement data quality rules to validate data accuracy, completeness, consistency, uniqueness, and timeliness.

- Perform data profiling, cleansing, reconciliation, and validation across large datasets.

- Identify data anomalies, inconsistencies, duplication, missing values, and integrity issues.

- Develop automated data quality checks and validation processes.

- Monitor data quality metrics and generate reports to support business and technical stakeholders.

- Investigate data quality incidents and perform root cause analysis.

- Collaborate with Data Engineering teams to resolve data quality issues across data pipelines.

2. SQL & Database Management :

- Develop and optimize complex SQL queries for data validation, profiling, reconciliation, and analysis.

- Work with large datasets across relational and NoSQL databases.

- Perform source-to-target data validation and reconciliation.

- Validate database transformations and business logic.

- Analyze query performance and optimize SQL scripts for efficient execution.

- Work with structured and semi-structured data formats.

3. Databricks & Data Engineering :

- Perform data quality validation using Databricks environments.

- Work with Databricks notebooks, Spark SQL, and distributed data processing.

- Validate batch and near-real-time data processing pipelines.

- Monitor ETL/ELT workflows and ensure data accuracy throughout the processing lifecycle.

- Collaborate with Data Engineers to implement quality checks within data pipelines.

- Work with Azure cloud-based data platforms and lakehouse architectures.

- Validate data transformations across data lakes, data warehouses, and lakehouse environments.

4. Monte Carlo Data Observability :

- Implement and maintain data observability monitoring using Monte Carlo.

- Monitor data pipelines, datasets, and critical data assets for anomalies.

- Configure data quality alerts, monitoring rules, and observability metrics.

- Identify unexpected changes in data volume, freshness, schema, and distribution.

- Investigate data incidents and support root cause analysis.

- Ensure timely detection and resolution of data reliability issues.

- Collaborate with data platform teams to improve observability coverage.

5. Data Governance & Metadata Management :

- Work with Collibra for data cataloging, metadata management, and data governance activities.

- Support data lineage and data asset documentation.

- Ensure data quality standards align with enterprise governance policies.

- Maintain data quality rules, metadata definitions, and business glossary information.

- Collaborate with Data Governance teams to establish data quality standards and controls.

6. Data Modeling & Architecture :

- Understand data warehouse, data lake, and lakehouse architectures.

- Work with dimensional data models, relational data models, and enterprise data structures.

- Validate data across fact tables, dimension tables, and data marts.

- Support data integration and transformation processes.

- Ensure data consistency across multiple data sources and target systems.

7. Collaboration & Reporting :

- Collaborate with Data Engineers, Data Architects, Data Analysts, and Business stakeholders.

- Document data quality issues, validation results, and remediation activities.

- Participate in Agile development, sprint planning, and technical discussions.

- Prepare data quality dashboards, reports, and metrics.

- Support regulatory and compliance-related data quality requirements.

Mandatory Technical Skills :

- Strong SQL and database querying skills.

- Hands-on experience with Databricks.

- Experience in Data Quality Engineering and data validation.

- Hands-on experience with Monte Carlo Data Observability.

- Experience with ETL/ELT pipelines and data integration.

- Experience with relational and NoSQL databases.

- Familiarity with Microsoft Azure data platforms.

- Knowledge of data warehouses, data lakes, and lakehouse architectures.

- Understanding of batch and near-real-time data processing.

- Familiarity with Collibra and data governance concepts.

Good to Have Skills :

- Programming experience in Python, Scala, or Java.

- Experience with Apache Spark and PySpark.

- Knowledge of orchestration tools such as Airflow or Control-M.

- Experience with data lineage, metadata management, and data cataloging.

- Exposure to banking, financial services, or regulatory environments.

- Knowledge of CDMP, Azure, or AWS certifications.

- Experience with automated data quality testing frameworks.

Candidate Profile :

- Strong analytical and problem-solving skills.

- Excellent understanding of data quality concepts and data engineering practices.

- Ability to work with large-scale enterprise data environments.

- Strong communication and collaboration skills.

- Ability to troubleshoot complex data issues independently.

- Immediate availability to join.

- Willingness to attend the Chennai office a couple of days per week during the initial months.

- Preferred Candidate : Data Quality Engineer / Data Quality Analyst / Data Observability Engineer with strong SQL, Databricks, Monte Carlo, ETL/ELT, and Azure experience.

Skills : sql, enterprise, data quality, data governance