Data Engineer - #1599

Data Modelling and Transformation

• Design and maintain data models that unify cyber asset information from diverse sources (WOG central systems, agency-specific data sources, vulnerability scanners, CMDBs)

• Build and maintain transformation pipelines that clean, normalise, enrich, and relate ingested data into a coherent asset inventory

• Establish and enforce data quality standards deduplication, completeness checks, schema validation, and lineage tracking

• Evolve the data model as new data sources are onboarded, ensuring backward compatibility and minimal disruption to existing dashboards

Dashboard and Visualisation Development

• Build dashboards that address agency-specific use cases including asset visibility, vulnerability prioritisation, patch tracking, and incident response readiness

• Collaborate with the Business Analyst to develop compelling data narratives selecting the right metrics, views, and drill-downs that connect data to agency decision-making

• Iterate on dashboard designs based on agency feedback, balancing clarity with analytical depth

• Maintain and update existing dashboards as underlying data models or agency requirements evolve

Platform Data Operations

• Validate successful data ingestion in coordination with the Platform Infrastructure Engineer confirming completeness, freshness, and schema conformance

• Monitor data pipeline health, investigate anomalies, and resolve data quality issues

• Optimise query performance and data refresh schedules to ensure dashboards remain responsive and current

• Document data models, transformation logic, and dashboard specifications for operational continuity

Insights and Collaboration

• Partner with the Business Analyst to identify patterns and insights within ingested data that support agency engagement
  Provide technical input on feasibility and effort when new agency use cases are proposed

• Contribute to defining what "good" looks like for asset visibility coverage metrics, quality scores, and completeness indicators

Requirements

Essential

• Minimum 3 years of experience in data engineering, analytics engineering, or business intelligence development

• Strong proficiency in SQL and experience with data transformation frameworks (e.g., dbt, Apache Spark, or equivalent)

• Hands-on experience building dashboards and visualizations with BI tools (e.g., Power BI, Tableau, Grafana, Superset, or platform-native tooling)

• Experience designing data models for operational or analytical use cases star schemas, entity resolution, or graph-based asset relationships

• Ability to work with messy, heterogeneous data from multiple sources and produce clean, reliable outputs

• Understanding of data quality practices including validation, deduplication, lineage, and monitoring

Desirable

• Experience with cyber asset data CMDBs, vulnerability scanners (Qualys, Tenable, Rapid7), endpoint management platforms, or network discovery tools

• Familiarity with attack surface management concepts including asset ownership, exposure scoring, and vulnerability lifecycle

• Experience with the Singapore government IT landscape, GCC (Government Commercial Cloud), and WOG shared services

• Familiarity with data pipeline orchestration tools (e.g., Airflow, Dagster, Prefect)

• Experience with Python for data manipulation and automation

• Prior experience in cross-functional teams working alongside infrastructure engineers and business analysts

Competencies

• Analytical Rigor: Ability to make sense of complex, heterogeneous data and produce models that are both correct and useful

• Outcome Orientation: Focuses on delivering insights that drive agency action, not just technically correct outputs

• Collaboration: Effective at working across disciplines partnering with Business Analysts on storytelling and Platform Infrastructure Engineers on data ingestion

• Adaptability: Comfortable working with imperfect data from diverse agency environments and iterating toward progressively better coverage and quality

• Communication: Able to explain data models, quality trade-offs, and dashboard logic to non-technical stakeholders