- Company Name
- Brooksource
- Job Title
- Associate Data Engineer
- Job Description
-
**Job title**: Associate Data Engineer
**Role Summary**
Build, deploy, and maintain end‑to‑end data pipelines and analytics solutions in a hybrid cloud environment. Actively participate in all stages of the development lifecycle—from design and ingestion to deployment and production support—while championing Agile and test‑driven practices.
**Expactations**
- Deliver high‑quality, scalable data engineering solutions that address business problems.
- Collaborate with cross‑functional teams (product owners, data scientists, application developers) to translate requirements into technical designs.
- Maintain CI/CD pipelines and enforce coding, security, and privacy standards.
- Troubleshoot and resolve production issues in a timely manner.
- Continuously improve performance, reliability, and maintainability of data assets.
**Key Responsibilities**
- Design, develop, and deploy batch and streaming data pipelines using Azure services (Data Lake Storage, Data Factory, Event Hub, Stream Analytics, Functions, Databricks).
- Migrate and integrate source data into Azure data lake and enterprise data warehouses.
- Model data, create schemas, and publish consumption layers for BI tools.
- Build and maintain CI/CD pipelines (GitHub, Azure DevOps, Jenkins, Terraform).
- Perform performance tuning, indexing, partitioning, and query optimization.
- Enforce data governance and security practices (privacy, access control, audit).
- Provide technical support, documentation, and training to internal stakeholders.
- Participate in DevOps activities, including infrastructure provisioning, monitoring, alerts, and incident response.
**Required Skills**
- Programming: Python, Scala/Spark, SQL, Unix shell scripting.
- Azure Cloud: Data Lake Storage, Data Factory, Databricks, Event Hub, Functions, Blob Storage, Power Apps, Power BI, Azure DevOps, Azure Interface.
- Data Platforms: Teradata, Cassandra, MongoDB, Oracle, SQL Server, Snowflake, ADLS, Azure Data Explorer.
- BI/Analytics: Power BI, Tableau, Cognos, Alteryx, Grafana, Denodo.
- CI/CD & Configuration: GitHub, Jenkins, Azure DevOps, Terraform, Git.
- Data Warehousing: DataStage, Informatica.
- Data Governance: Informatica Axon/EDC, BigID.
**Required Education & Certifications**
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field.
- Minimum 2 years of software development experience in Agile/DevOps environments.
- Minimum 2 years of SQL‑based data analytics experience.
- Minimum 1 year of cloud development and data lake experience (Azure).
- Certifications: Microsoft Azure – Developer Track; Apache Spark (preferred).