Lead Big Data Architect Needed for Stream Processing Setup Sprint

Silverline Labs

Contract · Remote · Flexible hours

About the role

Silverline Labs is building a unified data platform to streamline billing, CRM, and internal API flows. As a Big Data Architect you will lead the design of the "FusionBridge" module, ensuring seamless data exchange across systems while maintaining high performance and scalability.

Key responsibilities

• Architect the end‑to‑end data pipeline for FusionBridge using Apache Spark and Kafka.
• Design schema and data models in PostgreSQL to support real‑time billing reconciliation.
• Implement secure data ingestion from CRM APIs with AWS Glue and Lambda functions.
• Establish CI/CD for data workflows in Docker containers orchestrated by Kubernetes.
• Define data governance policies, data lineage tracking, and monitoring with Grafana.
• Deliver performance benchmarks and scalability tests for quarterly rollout milestones.

Must-have skills

• Deep expertise in big data architecture and distributed processing.
• Proficiency with Spark, Kafka, and AWS data services.
• Strong SQL modeling in PostgreSQL and data warehousing concepts.
• Experience containerizing data pipelines with Docker and Kubernetes.
• Ability to design secure, fault‑tolerant integrations across multiple systems.

Nice to have

• Familiarity with Terraform for infrastructure as code.
• Background in billing systems or CRM data models.