- Company Name
- DGTL PERFORMANCE
- Job Title
- DATA ENGINEER
- Job Description
-
**Job Title:** Data Engineer
**Role Summary:**
Design, build, and maintain high‑performance data pipelines and transformations using Kafka, DBT, Trino, and modern data lake technologies. Drive migration from legacy Cloudera/Spark environments to new architecture, ensure data quality, security, and compliance, and enable continuous integration/continuous delivery workflows for the data platform.
**Expectations:**
- Deliver robust, well‑documented ingestion and transformation solutions.
- Mentor teammates on DBT, Kafka, and platform best practices.
- Maintain adherence to data modeling standards, security policies, and GDPR compliance.
- Actively participate in Agile ceremonies and contribute to continual platform improvement.
**Key Responsibilities:**
- Build and document pipelines with Kafka Connect, DBT, and Trino.
- Design optimized, standardized data models that align with platform guidelines.
- Lead migration from Cloudera/Spark to Kafka, Trino, and Iceberg; optimize existing processes.
- Integrate code into CI/CD pipelines (Git, Jenkins/Argo, Airflow) and enforce deployment best practices.
- Manage data access controls, permissions, and GDPR‑compliant audit trails.
- Implement observability: logging, monitoring, job performance metrics, and access audits.
- Review and mentor on coding standards, testing, and documentation.
- Participate in daily stand‑ups, sprint planning, reviews, and retrospectives.
**Required Skills:**
- Hands‑on experience with Kafka, DBT, Trino, and Python.
- Familiarity with Cloudera/Spark, Iceberg, and other data lake formats.
- Strong ETL/ELT, data modeling, and data quality fundamentals.
- Proficiency with CI/CD and version control (Git).
- Knowledge of data security, access controls, and GDPR compliance.
- Experience with observability tools (Prometheus, Grafana, ELK).
- Comfortable working in Agile environments and cross‑functional teams.
**Required Education & Certifications:**
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related field.
- Certifications in Kafka (e.g., Confluent Certified Developer), DBT, or Trino are highly desirable.