Senior DevOps Engineer – Multi-Region Cloud Migration

LumenWorks

Contract · Remote · Flexible hours

About the role

LumenWorks is building a real‑time analytics backbone called PulseStream to empower its operations teams with actionable insights. As a Senior DevOps Engineer you will design, implement, and automate the data pipeline that ingests, processes, and visualizes telemetry from thousands of devices. The focus is on delivering a scalable, secure, and cost‑effective solution within a tight three‑month window.

Key responsibilities

• Architect and provision a hybrid AWS‑EKS cluster for high‑throughput streaming workloads using Terraform.
• Develop CI/CD pipelines in GitLab CI to deploy Spark jobs, Airflow DAGs, and monitoring agents automatically.
• Implement data‑validation and schema‑evolution checks with Great Expectations integrated into the pipeline.
• Configure Prometheus and Grafana dashboards to expose pipeline latency, error rates, and resource utilization.
• Optimize storage costs by automating lifecycle policies for S3 data lakes and Glacier archives.
• Conduct security hardening reviews, including IAM role scoping and network policy enforcement.

Must-have skills

• Deep experience with AWS services (EKS, S3, IAM, CloudWatch).
• Proficiency in containerization (Docker) and orchestration (Kubernetes).
• Strong Terraform or CloudFormation IaC expertise.
• Design and maintenance of CI/CD pipelines for data‑intensive applications.
• Solid scripting skills in Python or Bash for automation.

Nice to have

• Familiarity with Apache Airflow or Prefect for workflow orchestration.
• Experience with data‑validation frameworks like Great Expectations.