ROOM 03 / TASK 02

Training Data

Follow training data from its source to a model, and learn how to check quality, permissions, privacy, and hidden secrets.

Notes documented Oct 8, 2026 · 7 min read · Plain English

Make yourself comfortable
18px

Training data is the collection of examples a model learns from. The examples may be text, pictures, sound, numbers, or a mixture. During training, the system changes the model’s internal numbers, called parameters, to improve its results.

Imagine a fictional bicycle repair shop building a model that sorts messages into “booking,” “repair question,” and “other.” Its training examples affect what the model learns. A folder full of booking requests gives it little practice with repair questions. More files do not automatically mean better learning.

This note explains the data questions behind TryHackMe’s Training Data task. The bicycle shop examples are original illustrations.

Where can training data come from?

Four sources of training data pass through checks before an approved version is used to train a model. Source records remain connected to the data.

Open the diagram for a larger view. Each check asks a different question; no single check makes every dataset safe.

A dataset is an organized collection of data. Common sources include:

Source Simple meaning A question to ask
Web data Content collected from websites Who created it, and is this use permitted?
Licensed data Material supplied under stated terms Do those terms cover this training purpose?
Internal data Information from inside an organization Does the model need private details in it?
Synthetic data Examples made by software or people to represent situations Are the examples accurate and varied enough?

Web scraping means automatically collecting content from web pages. Common Crawl provides downloadable web archives. An archive is a saved collection of earlier web content, not a promise that every page is correct or suitable for training. Common Crawl’s data guide.

These categories can overlap. A licensed collection might contain web material. Synthetic examples might be based on internal records. “Internal” does not mean harmless, and “synthetic” does not guarantee privacy.

For our bicycle shop, invented messages can add practice with unusual wording. But if every generated message uses perfect English, the model may still struggle with short messages such as “brake bad need fix.” A person should review whether the examples match real needs.

Keep the history of the data

Provenance means knowing where something came from and how it changed. A useful record connects the original source to the exact dataset version used for training.

Record the source, collection date, responsible owner, relevant terms, and changes made. A dataset copied between websites may lose this information. The Data Provenance Initiative found missing and incorrect licensing information in its audit of text datasets. A label on a download page is therefore something to check, not a complete history. Data Provenance Initiative research.

Here is an original record for the bicycle shop:

Record field Example entry
Dataset version Repair messages v3
Purpose Sort incoming questions by topic
Source Approved message export and reviewed invented examples
Changes Contact details removed; repeated messages grouped
Known limit Few examples about electric bicycles
Owner Named person responsible for reviewing updates

Dates and version names matter. “We checked the data” is unclear if nobody knows which copy was checked. Keep the record with the dataset, and update it when the contents change.

Check quality before training

Data quality means how well the examples support the intended job. Review several kinds of problem:

  • Wrong labels: a booking request is marked as a repair question.
  • Missing cases: the collection has no messages written by beginners.
  • Old information: examples describe services the shop no longer offers.
  • Broken content: a file contains unreadable text or incomplete sentences.
  • Repeated examples: one message appears many times and dominates the collection.

Deduplication means finding and removing unnecessary copies. Check exact copies and very similar versions. Research on language models found that removing duplicates can reduce memorized repetition and improve evaluation. It does not remove every risk. Research on training-data deduplication.

For the shop, forty copies of “Can I book Friday?” are not forty different customer needs. Keep enough variation to teach the category without allowing accidental copies to overwhelm other examples.

Keep the final test separate

A model needs examples for learning and separate examples for checking its performance:

  • Training set: examples used to change the model’s parameters.
  • Validation set: examples used to compare choices during development.
  • Test set: reserved examples used for a final evaluation.

Data leakage, in evaluation, happens when information enters development that should only be available later. It can make a score look better than real performance. Keep test examples out of training and model selection. Learn preparation rules, such as numeric averages, from training data rather than the full collection. scikit-learn’s explanation of leakage.

Suppose the shop trains on a customer’s message and tests on a slightly edited copy. The test may reward remembering that message. Group related copies before splitting the data so they stay together. This evaluation problem is different from leaking someone’s private information, although both are called “leakage.”

Find private information and secrets

Personally identifiable information, often shortened to PII, can identify a person. Examples include contact details and combinations of details that point to someone. A secret controls access: a password, private key, or API key. An API key lets software identify itself to another service.

For the shop’s message sorter, a customer’s street address usually adds no value. Remove unnecessary private details before the data reaches training. Use automated checks and review samples; a scanner can miss unfamiliar formats. Keep restricted copies under access controls, and avoid putting sensitive examples into public reports. OWASP guidance on sensitive information disclosure.

This is a real concern. In February 2025, Truffle Security reported finding 11,908 working secrets in the December 2024 Common Crawl archive. The study examined that archive. It did not establish that every secret entered a particular model’s final training set or could be extracted from that model. Truffle Security’s original research.

If a real credential is exposed, removing its text is not enough. The responsible owner must revoke it or replace it, then investigate possible misuse. Old copies may still exist.

Does a model store a copy of every document?

Model weights are learned numbers, not a searchable folder of source documents. However, models can memorize some training text. Researchers have demonstrated extraction of memorized passages from language models, including personal information. Original training-data extraction research.

Avoid both extremes: “the model remembers everything” and “the model cannot reveal training data.” Neither is a reliable rule. Removing a source file after training does not automatically change existing model weights. Investigate the affected model and choose a tested response.

Make an inventory people can use

A datasheet explains a dataset’s purpose, contents, collection process, intended uses, and limitations. It helps the next person understand what they are receiving. Datasheets for Datasets research.

An ML-BOM, or Machine Learning Bill of Materials, records information about models, datasets, and related components. CycloneDX supports this kind of inventory. Think of it as an ingredient record that helps connect a model to its dependencies. It supports investigation; it does not prove that every ingredient is safe. CycloneDX ML-BOM guide.

For the shop, link “message sorter v2” to “repair messages v3.” If that dataset later has a problem, the team can identify which model needs review. Record decisions and results without copying customers’ private messages into the inventory.

Check your understanding

A dataset is downloadable for free. Is it ready for training?

No. Check its origin, relevant terms, quality, private information, and suitability for the job. Free access answers only the question of how you can obtain it.

The model scores well on copies of its training messages. Is that a strong test?

No. It may recognize examples it already saw. Use separate, suitable test examples to check how it handles new messages.

A password was removed from the dataset. Is the account protected now?

Not yet. Someone may already have copied the password. The owner must replace or revoke it and investigate access. Dataset cleanup and credential replacement solve different parts of the problem.