AI engineering glossary · Computing foundations
What is memory bandwidth?
Memory bandwidth: The amount of data a memory system can move per second, usually quoted in gigabytes per second. Many AI workloads, especially generating text from large models, are limited by how quickly weights can be read rather than by arithmetic speed.
Memory bandwidth: an example
A GPU with 2,000 GB/s bandwidth can read a 14 GB model's weights about 140 times per second at best.
Related terms
More Computing foundations terms
Where you learn it
Memory bandwidth is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.