If you’re asking what is training data, here is the short answer: it is the sample of examples a machine learning model learns from. Images, text, audio and tables all count. You’ll see how it’s stored, split and kept honest.
This explainer sits within the wider topic of emerging technologies.
| Item | Value |
|---|---|
| Standard number of data partitions | 2 or 3 data sets (training, test, and sometimes validation) |
| Common train/test ratio | 80/20 (% of samples) |
| Alternative train/test ratios | 60/40 or 90/10 (% of samples) |
| Example three-way splits | 75/15/10, 70/15/15 or 70/20/10 (% train/validation/test) |
| Typical validation set size | Similar to the test set, or slightly smaller |
These figures come from a TensorFlow Datasets tutorial and an AI wiki entry on datasets. They are rules of thumb and can change, so check the sources before you rely on them.
What training data looks like
Training data is the collection of examples a model studies to find patterns. The training dataset is the sample of data used to train the model, and the model learns from this data.
Format varies widely. According to the AI wiki, training data can take many forms, including:
- images and MRI scans
- text and CSV or tabular data, such as database queries
- audio recordings and video frames
- geospatial data, such as radar or vector data
- logs and time-series data, such as stock trades
- binaries or computer applications
Whatever the format, the pattern is the same: examples go in, and the model adjusts itself to fit them.
Where training data is stored
The same wiki names AWS S3, Snowflake, Redshift, AWS EBS, BigQuery and on-premise file systems as the most common providers of data in machine learning (AI wiki, datasets and machine learning). These are neutral examples of storage, not recommendations.
Many datasets start life in a database. They can’t be used there directly: a dataset must be extracted from a database and stored in a file system or object store such as S3 for training.
Because training sets can hold images, audio and logs, it is worth asking who the data describes and whether they agreed to its use. The guide on ai privacy concerns covers those questions in more depth.
Training, validation and test sets
It is standard practice to partition data into two or three data sets: training, test, and sometimes validation, which is recommended. All three should sample a larger body of data (AI wiki).
Each set has its own job:
- Training set: the sample the model learns from.
- Validation set: used during development to see how the model performs on new data, and to tune hyperparameters.
- Test set: a holdout sample used to evaluate the final model after training and tuning are complete.
A plain random split isn’t always enough. The wiki recommends a curated approach so that the data being tested against represents the new real-world data the model will see in the future.
One detail trips people up. During training the model sees validation data but does not learn from it, because hyperparameters are not learnable parameters, and the weights are not updated from the loss on that data (deeplizard).
How big should each split be?
There is no single rule. A tutorial on TensorFlow Datasets says the ratio between training and testing data is commonly 80/20, a fairly sensible default. Depending on dataset size you might choose 60/40 or even 90/10 (Stack Abuse).
When you add a validation set, the same tutorial says it is common to make it similar in size to the test set, or slightly smaller. It lists splits such as 75/15/10, 70/15/15 and 70/20/10 as examples.
Size also depends on how much data you have. If the testing set already holds many samples, there is no need to dedicate a large percentage of the data to it.
Why the test set stays locked away
You can’t judge a model fairly on the data it trained on. Evaluating that way would reward memorization instead of generalization (Stack Abuse).
The validation set helps you spot this early. Overfitting means a model becomes very good at classifying its training data but can’t make accurate classifications on data it wasn’t trained on (deeplizard). If results on training data are strong but validation results lag behind, the model is overfitting.
The test set is the final check. In Artificial Intelligence: A Modern Approach, Russell and Norvig call using test-set performance to both choose and evaluate a hypothesis “peeking”, and they suggest locking the test set away completely until all model tuning is complete (MachineLearningMastery).
The vocabulary is messy, too. The same article notes there is much confusion about what a validation dataset is and how it differs from a test dataset. In short, the validation set is used while tuning, and the test set gives an unbiased estimate of the final tuned model.
Public datasets as a starting point
You don’t always have to collect data yourself. Public datasets are large, typically labeled datasets made available for public use, and they are often the starting point for developing a model (AI wiki).
Famous examples include MNIST, ImageNet, CIFAR-10, MS COCO, Sentiment140, IMDB and LSUN. Many come from academia and some come from industry.
Many of these arrive with official splits. In the TensorFlow Datasets library, each dataset has an official split: some have only a training split, some have training and test, and some include validation too.
To see how one kind of model built on such data is applied, read the explainer on what is emotion recognition ai.
Frequently asked questions
Is training data the same as a dataset?
Show answer
Not quite. A dataset is the larger body of data, and the training data is the sample from it used to train the model. The rest is usually held back as validation and test data.
What is the difference between validation and test data?
Show answer
Validation data is held back from training and used to estimate model skill while tuning hyperparameters. Test data is also held back, but it gives an unbiased estimate of the final tuned model.
Does every model need a test set?
Show answer
Not always. A tutorial notes that some models, such as generative adversarial networks that create images, can’t be tested simply by comparing real and predicted labels. For most supervised tasks, though, a test set is standard.
What happens if a model overfits its training data?
Show answer
It performs well on the examples it has seen but fails to generalize to new ones. Watching validation results during training is one way to catch the problem early.
