Paper Readings
Paper Readings
Notes from papers I actually sat with — architecture, training tricks, and the occasional thing that changed how I read the next one.
Transformer architecture
Attention Residuals: A Comprehensive Understanding
This paper addresses a fundamental problem in training deep transformer models: uncontrolled hidden-state magnitude growth as model depth increases. The authors propose Attention Residuals (AttnRes), which replaces standard...
More to come.