AI engineering glossary · Deep learning core
What is dropConnect?
DropConnect: A relative of dropout that randomly zeroes individual weights, instead of whole neuron outputs, during each training step. It gives a finer-grained form of noise-based regularisation for fully connected layers, since every neuron sees a different random subset of its connections.
DropConnect: an example
With probability 0.2, each of a layer's 10,000 weights is masked on a given forward pass.
Related terms
More Deep learning core terms
- Dropout
- Dying ReLU
- ELU
- Embedding layer
- Epoch
- Evaluation mode
- Exploding gradient
- Exponential moving average of weights
Where you learn it
DropConnect is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.