AI engineering glossary · Deep learning core
What is autograd?
Autograd: PyTorch's automatic differentiation engine. It records tensor operations as they run and, when asked, walks that record backward to fill each tensor's gradient. It makes writing new network layers straightforward because only the forward computation must be coded.
Autograd: an example
x = torch.tensor(3.0, requires_grad=True); y = x**2; y.backward() leaves x.grad equal to 6.0.
Related terms
More Deep learning core terms
- Backpropagation
- Backpropagation through time
- Backward pass
- Batch normalisation
- Batch renormalisation
- Batch size
- bfloat16
- Bias term
Where you learn it
Autograd is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.