Data Engineer

Must-have technical skills

AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.

Key Responsibilities

• Build and maintain data pipelines using cloud technologies and distributed systems.

• Work with data transformation frameworks and pipeline orchestration tools.

• Manage large volumes of data and optimise data processing.

• Design and implement effective tests, including unit, component, integration, end-to-end and performance tests.

• Coach team members on coding practices, design principles and implementation patterns that produce high-quality, maintainable solutions.

• Manage stakeholders and prioritise work effectively across multiple workstreams.

Requirements

• Formal training or certification in data engineering concepts, together with applied experience.

• Recent hands-on professional experience as a data engineer.

• Experience coding in a recent version of Python.

• Hands-on experience with AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.

• Experience with cloud technologies and distributed systems.

• Experience with data transformation frameworks and data pipeline orchestration tools.

• Experience managing large volumes of data and optimising data processing.

• Understanding of event-based architecture, data streaming and messaging frameworks.

• Experience designing and implementing effective software tests.