top of page

Data Governance under the EU AI Act

4 days ago
8 min read

If you provide a high-risk AI system, Article 10 of the EU AI Act sets specific requirements for the data you use to train, validate and test it. Some of these are about keeping records. Others are about how you actually handle the data: how you collect it, prepare it, check its quality and look for bias. In this article we break down what you need to do and give some tips on how to do it. Many of them are simply good data management practices that you should be following anyway. Others are a little more involved.


Data governance under the EU AI Act, veritas_fox

Documentation and Lineage:


For the datasets you use to train, validate and test your models, Article 10(2) requires data governance practices covering:

  • The design choices behind each dataset, and why it is designed the way it is

  • Where the data comes from and how it was collected, including, for personal data, the purpose it was originally collected for

  • How the data has been prepared, including annotation, labelling, cleaning, enrichment and aggregation

  • What the data is supposed to measure and represent

  • Whether you have enough suitable data for the job


Table of Article 10 requirements, the Articles they come from and how to meet each one in practice

Luckily the first three should be straightforward to do if you have a strong data platform and follow good data engineering practices. 


All of your datasets should be documented with their intended purpose, what they are meant to measure and the design decisions behind them. This makes it clear why a given dataset is in your data warehouse in the first place and stops you from collecting pointless ones. It also means you are less likely to end up with 10 different ways of measuring the same thing. These descriptions can live in places like a DBT model file or your data warehouse’s table metadata, so people can find them when they are exploring your data.


If you use an orchestration system like Airflow, Dagster or Orchestra and set it up correctly, its lineage graph gives you a clear view of what data flows into your tables and where it came from. Some of these systems also surface the documentation for your datasets, giving you a clearer picture of how they fit together.


The code for your data transformation pipelines should be well structured and version controlled. With version control you can see which version of the code was deployed when, so you can trace what transformations were applied to your data at any point in time. Combine that with your orchestration system’s lineage graph and you can explain how you modified the source data to create the datasets you used to train your models.


The same goes for your data ingestion systems. They should be part of your orchestration’s lineage graph, and their source code should also be in version control. It is also good practice to store an unmodified copy of your raw data in your data warehouse and apply any transformations later in your pipelines. That way you can always trace what happened to the data and where the final results came from, even if something goes wrong. If the raw data includes personal data, set a retention period for it. GDPR’s storage limitation principle and the right to erasure still apply, so keeping everything forever isn’t an option.


Lineage and version control tell you how your data was transformed, but not exactly which data a model was trained on. Source data changes over time, so re-running the same code later may not produce the same dataset. Snapshot or version every training, validation and test set and record which dataset version trained each model version. Table formats like Iceberg and Delta Lake do this through time travel, and tools like DVC do it for files.

One step that often falls outside your lineage is labelling. If people annotate your data by hand, in house or through an outside vendor, that work won’t appear in your orchestration graph. Record who labelled the data, the instructions they worked from and how you checked their work.


All of these are good practices across your entire data platform, whether or not the data ends up training a high-risk AI system. Following them has the added benefit of making compliance with the EU AI Act a lot easier.


Data quality


Article 10(3) requires your datasets to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the system’s intended purpose. They also need appropriate statistical properties for the people or groups the system will be used on.

The practical way to meet this is to make data quality part of your pipelines rather than a one-off check. Tools like dbt tests or Great Expectations can check for missing values, duplicates, out-of-range values and unexpected shifts in distribution every time your data refreshes. The test results also give you a record showing that you monitor data quality.


Bias


Five sources of bias in training data: historical, sampling, measurement, proxy variables and labelling

Article 10 also requires you to examine your data for bias and act on what you find. Specifically, you need to:

  • Examine possible biases that could affect people’s health and safety, harm their fundamental rights or lead to discrimination

  • Take appropriate measures to detect, prevent and mitigate those biases

  • Identify gaps or shortcomings in your data that stop you from complying, and how you will address them

The Act doesn’t expect a perfectly unbiased dataset. It does expect you to look hard for problems and do something about them.

The Act singles out one kind of bias in particular: cases where a system’s outputs influence the data it receives in future. A study of predictive policing shows why this matters. In 2016, researchers Kristian Lum and William Isaac applied the algorithm behind PredPol, a widely used predictive policing tool, to drug crime data from Oakland, California. Public health survey data suggested that drug use was spread fairly evenly across the city. Drug arrests, however, were concentrated in Black and low-income neighbourhoods, which were policed more heavily. Trained on arrest data, the algorithm directed police back to those same neighbourhoods. More policing there would produce more arrests, making the algorithm even more confident that it was right. The result is a feedback loop that makes the data more skewed each time.


A provider in this position would need to recognise that arrest data measures where police look as much as where crime happens. They might use crimes reported by the public rather than crimes discovered by police, although that brings its own bias, because some communities report crime less than others. They could weight down discovered incidents in proportion to how heavily an area was patrolled. Or they could do what Lum and Isaac did and compare their data against an independent source, such as survey data, to measure how skewed it is. Even if they couldn’t fix the problem completely, they would have to document it and explain how they were managing it.


Where bias comes from


Bias can enter at many stages, so it helps to know what to look for:

  • Historical bias: the data accurately reflects a world that was itself unfair, such as past hiring decisions.

  • Sampling bias: some groups are under- or over-represented in the data you collected.

  • Measurement bias: what you can measure is a poor stand-in for what you care about, like arrests standing in for crime.

  • Proxy variables: an apparently neutral field, such as postcode, closely tracks a protected characteristic, such as ethnicity.

  • Labelling bias: the people annotating your data bring their own assumptions to it.

Article 10(4) adds that your data must reflect the setting where the system will be used. A dataset that is representative for one country, language or customer group may not be for another.


Measuring and reducing bias


In practice, mitigation usually involves some combination of:

  • Comparing the make-up of your dataset with a reference population, such as census data

  • Re-sampling or re-weighting the data so that under-represented groups count properly

  • Testing model outcomes separately for each relevant group, not just overall

  • Building representation checks into your pipelines as automated tests, so that a shift triggers an alert rather than going unnoticed


The special category data problem


There is a catch. To check whether your system treats ethnic groups fairly, you often need to know people’s ethnicity, and GDPR restricts processing that kind of data. Article 4a of the AI Act, added by the Digital Omnibus, lets providers of high-risk systems process special categories of personal data where this is strictly necessary to detect and correct bias, and only where other data, such as synthetic or anonymised data, won’t do. Strict safeguards apply: 

  • pseudonymisation and tight access controls

  • no sharing with other parties

  • deletion once the bias has been corrected or the retention period ends

  • a record of why the processing was necessary

These are data engineering problems as much as legal ones, so plan for them when you design your platform.

The high-risk requirements apply from 2 December 2027 for systems listed in Annex III, and from 2 August 2028 for systems covered by the EU product legislation in Annex I.


Bias after launch


Your obligations don’t end at release. If your system keeps learning after deployment, Article 15(4) requires you to address feedback loops of exactly the kind the PredPol study shows. All high-risk systems also need post-market monitoring under Article 72. For your data platform, that means monitoring the data coming into the live system for drift and checking regularly that the people it affects still resemble the people it was trained on.


Diagram of the predictive policing feedback loop: arrest data, algorithm, more patrols, more arrests

Summary

Requirement

Article

In practice

Document dataset purpose and design

10(2)(a), (d)

Descriptions in dbt models or table metadata

Show where data came from and how it was collected

10(2)(b)

Orchestration lineage, ingestion code in version control, raw data retained

Show how data was prepared

10(2)(c)

Version-controlled transformations, labelling records

Trace the exact data behind each model

Good practice, supports Annex IV

Versioned dataset snapshots linked to model versions

Ensure data quality

10(3)

Automated data quality tests

Examine and mitigate bias

10(2)(f)–(g)

Representation analysis, subgroup testing, re-weighting

Identify data gaps

10(2)(h)

Documented gaps and plans to address them

Fit the context of use

10(4)

Check data against the target population and setting

Monitor after launch

15(4), 72

Drift and representation monitoring on live data



veritas_fox is an EU AI Act and ISO/IEC 42001 compliance consultancy. We start by working out whether your AI system is high-risk at all, because that decides which obligations apply to you. If it is, we run a gap assessment against the Act and show you exactly where you fall short. That includes a review of your data governance against Article 10: lineage, data quality, bias examination and data gaps. We then write the policies and governance frameworks that turn those findings into everyday practice, and we run AI literacy training so your teams understand the systems they build and use. If you’re working towards ISO/IEC 42001, we assess your readiness and help you implement the management system, reusing your AI Act work wherever it fits. To find out where your data governance stands before the high-risk rules apply, book a free consultation with us.


FAQs


When do the EU AI Act data governance requirements apply? For high-risk AI systems listed in Annex III, from 2 December 2027. For high-risk AI in products covered by the EU product legislation in Annex I, from 2 August 2028. The Digital Omnibus on AI (Regulation (EU) 2026/1744), in force since 27 July 2026, set these dates.


Does the EU AI Act require a bias-free dataset? No. Article 10 requires providers of high-risk AI systems to examine their data for biases that could affect health and safety, harm fundamental rights or lead to discrimination, and to take appropriate measures to detect, prevent and mitigate them.


What data quality does Article 10(3) require? Training, validation and test data must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the system’s intended purpose, with appropriate statistical properties for the people or groups the system will be used on.


Can I process special category personal data to detect bias? Yes, under Article 4a of the AI Act, added by the Digital Omnibus, where it is strictly necessary to detect and correct bias and other data such as synthetic or anonymised data will not do. Safeguards apply, including pseudonymisation, access controls, no sharing with other parties, deletion once the bias is corrected or the retention period ends, and a record of why the processing was necessary. Article 4a has applied since 27 July 2026 and replaced Article 10(5).


Do I need to version my training datasets? Article 10 does not name versioning, but recording which dataset version trained each model version is good practice and supports the technical documentation required under Annex IV.

Comments


bottom of page