AI engineering glossary · Inference serving optimisation
What is NVLink?
NVLink: A high-speed direct link between NVIDIA GPUs, much faster than the standard PCIe slot. It matters when a model is split across several GPUs, because they must exchange partial results at every layer.
NVLink: an example
Eight GPUs in one server connected by NVLink can run tensor-parallel inference with little communication delay.
Related terms
More Inference serving optimisation terms
- Ollama
- Online inference
- ONNX
- ONNX Runtime
- OpenAI-compatible API
- Outlier features
- PagedAttention
- Per-channel quantisation
Where you learn it
NVLink is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.