AI engineering glossary · Computer vision
What is fine-tuning?
Fine-tuning: Continuing to train a pretrained model on a smaller task-specific dataset, usually with a low learning rate, so its existing features adapt to the new problem. It needs far less data than training from scratch.
Fine-tuning: an example
A team fine-tunes an ImageNet ResNet on 2,000 labelled X-rays for 10 epochs at a learning rate of 0.0001.
Fine-tuning in Python
A short example you can run (Python 3.11, scikit-learn). The output below is what it really prints.
from sklearn.datasets import load_digits
from sklearn.linear_model import SGDClassifier
X, y = load_digits(return_X_y=True)
general, task = (X[y < 5], y[y < 5]), (X[y >= 5], y[y >= 5])
model = SGDClassifier(random_state=0)
model.partial_fit(general[0], general[1], classes=list(range(10))) # "pre-training"
print("before fine-tuning:", round(model.score(*task), 3))
for _ in range(5):
model.partial_fit(task[0][:300], task[1][:300]) # fine-tune on new task data
print("after fine-tuning: ", round(model.score(task[0][300:], task[1][300:]), 3))Output
before fine-tuning: 0.0 after fine-tuning: 0.903
Related terms
More Computer vision terms
- Fully convolutional network
- Gaussian blur
- Global average pooling
- Grad-CAM
- Grayscale image
- Histogram equalisation
- Histogram of oriented gradients
- Homography
Where you learn it
Fine-tuning is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.