- Company Name
- Atlantis IT Group
- Job Title
- Senior Data Engineer
- Job Description
-
**Job Title**: Senior Data Engineer
**Role Summary**
Design, build, and maintain scalable, secure data pipelines and streaming solutions on AWS. Lead end‑to‑end data integration, ETL, and workflow orchestration across multiple data formats and sources.
**Expectations**
- Deliver high‑performance, production‑ready data pipelines on AWS.
- Ensure data quality, security, and regulatory compliance at scale.
- Mentor junior engineers and drive best‑practice adoption.
**Key Responsibilities**
1. Architect and develop data pipelines using AWS services (S3, Glue, Lambda, Redshift, Kinesis).
2. Implement streaming ingest for Kafka topics (JSON, CSV) with Node.js.
3. Build ETL jobs in Python or Node.js leveraging Glue, Lambda, and Redshift.
4. Deploy infrastructure via AWS CDK (TypeScript) and manage IaC.
5. Orchestrate workflows with Step Functions and Airflow.
6. Convert data among CSV, Excel, Parquet, Avro, and other formats.
7. Apply encryption, masking, and encoding to secure data at rest and in transit.
8. Perform data cleaning, validation, and quality checks on large volumes from diverse sources.
**Required Skills**
- 10+ years data engineering; strong AWS expertise.
- Proficient in Python/Node.js, TypeScript, and AWS CDK.
- Hands‑on with S3, Glue, Lambda, Redshift, Step Functions, Airflow, Kinesis, Kafka.
- Familiar with multi‑format data processing (CSV, Excel, Parquet, Avro).
- Solid knowledge of data security (encryption, masking, RBAC).
- Experience in CI/CD and automated testing for data pipelines.
**Required Education & Certifications**
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect (preferred).