News:Tech News
Depth is the hallmark of deep neural networks. But more depth means more sequential computation and higher latency. This begs the question – is it possible to build high-performing “non-deep” neural networks? We show that it is. To do so, we use parallel subnetworks instead of stacking one layer after another. This helps effectively reduce depth while maintaining high performance.
2021: Ankit Goyal, Alexey Bochkovskiy, Jia Deng, V. Koltun
https://arxiv.org/pdf/2110.07641v1.pdf
view more
More Episodes
Fine-Tuning Language Models from Human Preferences
2023-03-01
117
2023-03-01
117
Mastering Diverse Domains through World Models
2023-02-23
146
2023-02-23
146
Adding Conditional Control to Text-to-Image Diffusion Models
2023-02-21
101
2023-02-21
101
Token Merging: Your ViT But Faster
2023-02-21
58
2023-02-21
58
Dual PatchNorm
2023-02-16
2023-02-16
Reversible Vision Transformers
2023-02-15
87
2023-02-15
87
Offsite-Tuning: Transfer Learning without Full Model
2023-02-14
69
2023-02-14
69
A Length-Extrapolatable Transformer
2023-02-09
91
2023-02-09
91
Multimodal Chain-of-Thought Reasoning in Language Models
2023-02-07
101
2023-02-07
101
Why do Nearest Neighbor Language Models Work?
2023-01-19
100
2023-01-19
100
Get this podcast on your
phone, FREE
