AI engineering glossary · Computing foundations
What is FP16 (half-precision floating point)?
Half-precision floating point (also called FP16): A 16-bit floating-point format that uses half the memory of the usual 32-bit format, with less range and precision. It is widely used in deep learning to speed up computation and shrink models on GPUs.
Half-precision floating point: an example
A 7-billion-parameter model needs about 14 GB in 16-bit format versus 28 GB in 32-bit.
Related terms
More Computing foundations terms
Where you learn it
Half-precision floating point is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.