AI engineering glossary · Inference serving optimisation
What is NCCL?
NCCL: NVIDIA's library of fast communication routines, such as all-reduce, for exchanging data between GPUs inside one machine and across machines. Multi-GPU serving and training frameworks rely on it to combine partial results quickly.
NCCL: an example
A multi-GPU vLLM deployment uses it under the hood for collective operations.
Related terms
More Inference serving optimisation terms
Where you learn it
NCCL is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.