AI engineering glossary · Inference serving optimisation
What is GPU hours?
GPU hours: The unit in which cloud GPUs are billed: one GPU rented for one hour. Estimating serving cost means counting how many GPU hours a workload needs and multiplying by the hourly rate.
GPU hours: an example
Eight GPUs running for 3 hours consume 24 GPU hours.
Related terms
More Inference serving optimisation terms
- GPU kernel
- GPU utilisation
- Group-wise quantisation
- High bandwidth memory
- Inference engine
- INT4
- INT8
- Inter-token latency
Where you learn it
GPU hours is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.