AI engineering glossary · Inference serving optimisation
What is FP8?
FP8: An 8-bit floating-point format, usually in two variants: E4M3 with more precision and E5M2 with more range. Recent GPUs multiply FP8 numbers natively, giving about twice the arithmetic speed of 16-bit while keeping the dynamic range that integers lack.
FP8: an example
An H100-class GPU can run a transformer's matrix multiplications in FP8 with per-tensor scale factors, saving memory and time.
Related terms
More Inference serving optimisation terms
Where you learn it
FP8 is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.