Lead Big Data Architect for Database Optimization

DataPro Services

Contract · Flexible hours

About the role

DataPro Services is seeking an experienced Big Data Architect to design and implement a scalable data platform for a high‑growth analytics initiative. You will lead the end‑to‑end architecture, ensuring data reliability, performance, and security across the enterprise.

Key responsibilities

• Define and document the overall big‑data architecture, including ingestion, storage, processing, and serving layers.
• Design data pipelines using Apache Spark, Kafka, and AWS services to handle petabyte‑scale workloads.
• Collaborate with data engineers and scientists to translate business requirements into technical solutions.
• Implement data governance, security, and compliance best practices.
• Optimize performance and cost of data processing workloads.
• Mentor junior team members and enforce coding standards.

Must-have skills

• Extensive experience with Hadoop ecosystem and Spark.
• Strong proficiency in Python for data engineering.
• Deep knowledge of AWS services (S3, EMR, Redshift, Glue).
• Expertise in designing data models and ETL pipelines.
• Hands‑on experience with containerization (Docker) and orchestration.

Nice to have

• Familiarity with Terraform or CloudFormation.
• Experience with real‑time analytics using Kafka Streams.