Senior Member of Technical Staff (AI Data)

About Company

Founded in 2022, XG Tech is driving the future of smart vehicles. Its mission is to empower the digital transformation of automobiles, moving from distributed computing to a centralized, cross-domain platform.

XG Tech focuses on the intelligent cockpit—the next frontier of differentiation—while seamlessly integrating advanced driving systems. By reimagining cars as mobile living spaces, XG Tech aligns with the evolving trend of vehicles becoming the “third living space.”

Role Summary

As an AI Data and Evaluation Engineer, you will design and develop data pipelines and evaluation frameworks for personal AI computers, covering the collection, processing, curation, and quality assessment of training data, as well as model benchmarking and performance evaluation. You will establish reliable evaluation methodologies to measure model capabilities, identify data quality gaps, and drive continuous improvements in training datasets and model performance, ensuring AI models are evaluated accurately and optimized based on measurable results.

Key Responsibilities

Data supply and quality

• Own collection pipelines end to end — sourcing, ingestion, storage, and versioning — for SFT, preference, and agent-trajectory data.
• Build cleaning and quality control that is measurable: deduplication, filtering, labelling standards, spot checks, and batch acceptance criteria that a vendor can be held to.
• Manage vendor relationships: scope the work, define quality bars, run trials, audit deliveries, and hold suppliers to the standard rather than hoping for it. Renegotiate or replace when the data does not clear the bar.
• Build the tooling that makes quality visible — dashboards, per-batch reports, and failure taxonomies the model teams can act on rather than merely read.
• Work with researchers so that data supply tracks what training actually needs, not what is easy to collect or what a vendor already has lying around.

Evaluation

• Build evaluation baselines for the domains that matter to us now: software engineering (coding agents) and video generation.
• On the coding side: curate and lock datasets, build reproducible end-to-end runs, and report model-level metrics that survive review by people who do not trust benchmarks by default.
• On the video side: design a consistency-evaluation scheme, generate evaluation sets at meaningful scale, and produce model comparisons with conclusions rather than scores.
• Build the infrastructure — test-case definition, metric plumbing, latency/token/cost accounting — so that evaluating a new model is a routine operation rather than a project.
• Keep the loop honest: detect contamination, judge drift, and the gap between offline metrics and how the thing actually feels in use, and say so when the numbers and the experience disagree.

How you will stand out

• Strong background in Data Engineering, Computer Science, Machine Learning, or a related field, with solid experience building and maintaining scalable data pipelines and infrastructure.
• Proficiency in Python, with hands-on experience in data processing, storage systems, schema design, and data quality management.
• Demonstrated experience managing large-scale datasets, data validation, quality assurance, and developing reliable data processing workflows.
• Experience designing and implementing model evaluation frameworks, benchmarks, and performance metrics to assess AI model capabilities and improvements.
• Strong analytical and problem-solving skills, with the ability to identify data quality issues, evaluate noisy or inconsistent metrics, and establish measurable quality standards.
• A strong academic or technical track record demonstrated through research publications, particularly in data quality, model evaluation, benchmarking, or reproducibility, as well as production data platforms, open-source contributions, or technical achievements.
• Ability to work with internal and external stakeholders to establish data quality requirements, annotation standards, and evaluation processes

Nice to have:

• Experience working with AI agent trajectories, UI interaction data, or multimodal datasets.
• Familiarity with data annotation platforms and human-in-the-loop quality assurance processes.
• Experience with LLM-as-a-Judge evaluation, calibration, and its limitations.
• Knowledge of privacy-aware data processing and secure handling of personal or sensitive data.
• Experience optimizing storage, computing, and processing costs for large-scale data infrastructure.

XG Tech is committed to providing equal employment opportunities by country, state, and local laws. XG Tech does not discriminate against employees or applicants based on conditions such as race, color, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.