Paper2015
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · CVPR · 2015
Abstract
Solves the problem that made very deep networks worse than shallow ones: past a certain depth, accuracy degraded even on training data, so the issue was optimisation rather than overfitting. The fix is to have each block learn a residual — a correction added to its input — so an unhelpful layer can approximate the identity instead of destroying the signal.
Why it matters
Residual connections are in every transformer block you will ever use, which makes this required background even though the paper is about images. The framing is the lesson: the obstacle to depth was trainability, not capacity.
architecturetrainingpretraining
Read the source
https://arxiv.org/abs/1512.03385