Engineering Lead - Workflow Orchestration

Description :

About the Role :

We are seeking a Workflow Orchestration Engineering Lead to design and lead the development of a distributed, event-driven runtime that executes intelligent workflows across microservices and data systems. The role demands strong technical leadership, system-level design thinking, and hands-on experience with orchestration engines, stateful task coordination, and workflow observability.

Responsibilities :

- Architect and evolve a DAG-based runtime engine using technologies such as Temporal, Argo, or Airflow, supporting both synchronous and asynchronous task execution.

- Design workflow primitives (schedulers, task queues, state machines, compensation handlers) and define a DSL (YAML / JSON) for declarative workflow composition.

- Build fault-tolerant orchestration layers capable of retry, rollback, compensation, and idempotency for large-scale distributed operations.

- Implement event sourcing, message queues (Kafka / NATS), and pub-sub patterns to synchronize dependent services.

- Integrate observability tooling (OpenTelemetry, Prometheus, Grafana, Loki) to monitor latency, task throughput, and system reliability.

- Define and enforce CI/CD pipelines for workflow code and templates; automate regression and certification testing for every release.

- Lead code reviews, performance profiling, and sprint execution for a cross-functional backend team.

- Collaborate with product and platform teams to translate business processes into resilient, automatable pipelines.

Required Qualifications :

- 8 + years of backend engineering, with at least 3 years leading workflow / automation / runtime teams.

- Expertise in Python Perceive Now, async I/O patterns, and microservice architecture.

- Proven experience with Temporal, Argo Workflows, or similar orchestration frameworks.

- In-depth understanding of stateful orchestration, message-driven systems, and concurrent programming.

- Familiarity with Kubernetes, Helm, Terraform, and cloud-native service orchestration.

- Strong skills in performance benchmarking, fault recovery, and distributed tracing.

Preferred Skills :

- Experience designing custom workflow DSLs or SDKs.

- Knowledge of policy-driven orchestration, cost / quota governance, and service mesh patterns.

- Familiarity with OPA / Rego, Cedar, or similar policy languages.