AI engineering glossary · Inference serving optimisation
What is OpenAI-compatible API?
OpenAI-compatible API: An HTTP interface that copies the request and response format of OpenAI's chat and completion endpoints. Many open-source servers offer it, so applications written for one provider can switch by changing only the base URL and model name.
OpenAI-compatible API: an example
Pointing an existing client's base URL at a self-hosted vLLM server lets the same code call an open model.
Related terms
More Inference serving optimisation terms
- Outlier features
- PagedAttention
- Per-channel quantisation
- Per-tensor quantisation
- Percentile latency
- Post-training quantisation
- Preemption
- Prefill-decode disaggregation
Where you learn it
OpenAI-compatible API is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.