Data Engineer

Job Description & Requirements

Role Overview
This role is responsible for designing, building, and deploying a scalable and reliable data lakehouse platform on AWS. The Data Engineer will architect end-to-end data pipelines, establish data quality frameworks, and deliver production-ready solutions that form the foundation of the organisation’s data infrastructure.

Responsibilities and Requirements

• Design and implement scalable ETL/ELT pipelines using AWS services including Glue, Step Functions, Lambda, and S3

• Architect and build data lakehouse solutions leveraging Apache Iceberg or S3 Tables

• Implement lakehouse capabilities including schema evolution, partition evolution, and ACID transactions

• Optimise data pipelines for performance, cost efficiency, scalability, and reliability

• Design and implement data processing workflows supporting production-grade data platforms

• Define, implement, and maintain automated data quality validation frameworks

• Establish data quality metrics and monitoring to ensure accuracy, completeness, and consistency of data

• Enforce data governance standards and ensure compliance across the data platform

• Write production-quality code for data engineering solutions

• Deploy solutions on AWS cloud infrastructure using modern DevOps practices

• Develop and maintain CI/CD pipelines and infrastructure-as-code practices using Terraform, CloudFormation, or equivalent tools

• Ensure deployments are repeatable, auditable, and aligned with operational requirements

• Collaborate with cross-functional teams and communicate technical designs clearly to non-technical stakeholders

• Produce technical documentation to support handover of developed solutions to Day 2 operations teams

• Design and build scalable and reliable data pipelines using modern data lakehouse architectures

• Apply strong understanding of Apache Iceberg features and S3 table optimisation techniques

• Implement automated validation frameworks and governance controls for enterprise data platforms

• Optimise application performance and cloud deployments

• Support production-ready data engineering solutions in AWS environments

• Degree in Computer Science, Data Engineering, Information Systems, or related field

• At least 3-5 years of experience in data engineering, ETL/ELT development, or data platform roles

• Strong hands-on experience with AWS services

• Strong proficiency in Apache Iceberg or similar open table formats

• Experience working in Government Commercial Cloud (GCC) environments is strongly preferred

• AWS certifications such as AWS Certified Data Analytics or AWS Certified Solutions Architect are advantageous

• Strong collaboration and communication skills with ability to explain technical concepts clearly to non-technical stakeholders