AI engineering glossary · Deep learning core
What is fully connected layer?
Fully connected layer (also called Dense layer, Linear layer): A layer in which each output neuron receives a weighted contribution from every input value. It is implemented as one matrix multiplication plus a bias, which makes it fast on GPUs. It is also called a dense or linear layer.
Fully connected layer: an example
A fully connected layer mapping 512 inputs to 10 outputs has 512*10 weights plus 10 biases = 5,130 parameters.
Related terms
More Deep learning core terms
- Gated linear unit
- Gating mechanism
- GELU
- Gradient checkpointing
- Gradient norm
- Group normalisation
- GRU
- He initialisation
Where you learn it
Fully connected layer is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.