Description :

Job Title : IT engineer Data Engineer

Main Tasks :

- Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone)

- Implement CDC, delta, and event-based patterns.

- Push-based HTTP, Kerberos-authenticated DLT delivery.

- Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal

- Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints.

- Establish and maintain a business-friendly, web-accessible data catalog application, with dataset profiles, metadata, and usability features.

- Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system.

- Design and implement structured access request workflows including request submission, approval chains, audit trail, and enablement triggers.

- Perform design reviews with Cybersecurity.

- Ensure documentation and compliance for all interfaces and data ingress points.

- Manage audit and traceability requirements.

- Collaborate closely with IT and business users to translate requirements into scalable technical patterns.

- Serve as technical escalation point for complex source integration.

- Define and implement a multi-layered data quality framework, including unit-level, integration-level, and cross-pipeline validation rules.

- Establish centralized and version-controlled storage of DQ rules, with integration into orchestration and CI/CD pipelines.

- Implement automatic DQ monitoring with severity levels (Critical, High, Medium, Low) and enable flagging, filtering, and quarantining logic at relevant stages of the pipeline.

- Collaborate with source system owners and business stakeholders to define meaningful and actionable DQ thresholds.

Qualifications :

- Degree in Computer Science, Data Engineering, or a related field. Azure or Databricks certification is a plus.

- 5-8 years in data engineering, with hands-on experience in ingesting structured and unstructured enterprise data into modern cloud platforms.

- Proven implementation of source system ingestion frameworks, metadata automation, and compliance-controlled interfaces.

- Not required; however, experience mentoring junior developers or leading implementation workstreams is a plus; contributes to engineering standards and code quality improvement initiatives.

- Comfortable working across geographies and time zones; collaborates effectively with global teams and enterprise stakeholders.