AI engineering glossary · Reinforcement learning
What is UCB (upper confidence bound)?
Upper confidence bound (also called UCB): A bandit method that picks the option with the highest optimistic estimate: its average reward plus a bonus that shrinks as it is tried more. Options tried rarely get a big bonus, so the method explores automatically. Formula for UCB1: mean + sqrt(2 * ln(t) / n).
Upper confidence bound: an example
After 100 rounds, an arm tried 5 times with mean 0.4 has score 0.4 + sqrt(2*ln(100)/5) = 1.7, beating an arm tried 60 times with mean 0.5.
Related terms
More Reinforcement learning terms
Where you learn it
Upper confidence bound is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.