AI engineering glossary · Inference serving optimisation
What is ONNX?
ONNX: Open Neural Network Exchange, an open file format for representing a trained model as a graph of standard operations, so a model built in one framework can run in different runtimes and on different hardware.
ONNX: an example
A PyTorch model is exported to a .onnx file and then served with ONNX Runtime on a CPU-only server.
Related terms
- ONNX Runtime
- TensorRT
- Model export
- Inference engine
More Inference serving optimisation terms
- ONNX Runtime
- OpenAI-compatible API
- Outlier features
- PagedAttention
- Per-channel quantisation
- Per-tensor quantisation
- Percentile latency
- Post-training quantisation
Where you learn it
ONNX is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.