- Company Name
- Versa Networks
- Job Title
- Data DevOps Engineer
- Job Description
-
Job Title: Data DevOps Engineer
Role Summary:
Seasoned Data DevOps Engineer (8+ years) responsible for designing, building, and maintaining end‑to‑end data pipelines for ML/AI products. Operates multi‑tenant Apache Spark and Kafka clusters, automates infrastructure with Jenkins/Ansible/Terraform, and collaborates with DevOps, data science, and software engineering teams to deliver scalable, monitored, and debuggable data solutions.
Expectations:
- Own the full lifecycle of data pipelines from ingestion to consumption.
- Ensure high availability, scalability, and performance of Spark and Kafka deployments.
- Deliver robust CI/CD pipelines and automation for rapid delivery.
- Provide actionable insights and monitoring for operational excellence.
- Manage multiple concurrent projects while maintaining quality and timelines.
Key Responsibilities:
- Design, implement, and maintain data pipelines feeding ML/AI services.
- Provision and configure multi‑tenant Apache Spark and Kafka clusters.
- Build and maintain CI/CD pipelines using Jenkins, Ansible, Terraform, or equivalent tools.
- Partner with Data Scientists and Software Engineers to translate feature requirements into pipeline enhancements.
- Enhance monitoring, logging, and debugging capabilities (e.g., Prometheus, Grafana, ELK).
- Clean, process, analyze, and store large collections of raw data.
- Evaluate database options (NoSQL, SQL, cloud SQL) and advise on appropriate storage solutions.
- Optimize pipeline performance and troubleshoot production incidents.
- Participate in on‑call rotation and incident response.
Required Skills:
- 8+ years of experience in data engineering or DevOps within ML/AI or large-scale data environments.
- Deep expertise in Apache Spark and Apache Kafka, including multi‑tenant cluster configuration.
- Strong programming skills in at least one of Python, Scala, or Java.
- Hands‑on experience with infrastructure automation (Ansible, Terraform) and CI/CD (Jenkins, GitLab CI, etc.).
- Proficiency in data cleaning, transformation, and processing at scale.
- Solid understanding of relational and NoSQL databases, and cloud database services.
- Familiarity with monitoring and observability tools (Prometheus, Grafana, ELK, or similar).
- Excellent scripting (Bash, Shell) and troubleshooting capabilities.
- Demonstrated ability to manage multiple simultaneous projects and work collaboratively across teams.
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Engineering, Data Science, or related field (or equivalent professional experience).
- Professional certifications in relevant technologies (e.g., AWS Certified Solutions Architect, Azure Data Engineer, GCP Data Engineer, Certified Kubernetes Administrator) are desirable but not mandatory.
Santa clara, United states
On site
Senior
09-03-2026