AI engineering glossary · Deep learning core
What is BF16 (bfloat16)?
bfloat16 (also called BF16, Brain floating point): A 16-bit number format with the same exponent range as 32-bit floating point but fewer precision bits. Its wide range makes training more stable than standard FP16 without loss scaling, and it is supported on modern GPUs and TPUs.
bfloat16: an example
Values up to about 3e38 fit in it, whereas FP16 overflows above 65,504.
Related terms
More Deep learning core terms
- Bias term
- Bidirectional RNN
- Binary cross-entropy
- Bottleneck
- Catastrophic forgetting
- Cell state
- Checkpoint
- Cross-entropy loss
Where you learn it
bfloat16 is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.