AI engineering glossary · Inference serving optimisation
What is nvidia-smi?
nvidia-smi: A command-line tool from NVIDIA that shows the state of GPUs on a machine, including memory used, utilisation, temperature, power and running processes. It is the first thing engineers check when debugging out-of-memory errors or idle GPUs.
nvidia-smi: an example
Running nvidia-smi shows 38 GB of 40 GB in use, explaining why a second model failed to load.
Related terms
More Inference serving optimisation terms
Where you learn it
nvidia-smi is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.