enEnglish

Yannic Kilcher

Name: A critical analysis of self-supervision, or what we can learn from a single image (Paper Explained)
Uploaded: 2020-05-26T14:18:55.000Z

I make videos about machine learning research papers, programming, and issues of the AI community, and the broader impact of AI in society.

Official YouTube channel ↗

Videos: 487
Subscribers: 333K
Total views: 18.3M
Latest uploads: Mar 6, 2026

Independent page
Machine Learning
Content language: English
Region: Switzerland

This is an independent directory page operated by Youtop. It is not run by, sponsored by, or endorsed by Yannic Kilcher or YouTube. Official YouTube channel ↗Suggest a correction Are you this creator? Claim this page

A critical analysis of self-supervision, or what we can learn from a single image (Paper Explained) — Yannic Kilcher · Youtop

A critical analysis of self-supervision, or what we can learn from a single image (Paper Explained)

Yannic Kilcher · Published May 26, 2020

Watch on YouTube ↗

From YouTube metadata

Does self-supervision really need a lot of data? How low can you go? This paper shows that a single image is enough to learn the lower layers of a deep neural network. Interestingly, more data does not appear to help as long as enough data augmentation is applied. OUTLINE: 0:00 - Overview 1:40 - What is self-supervision 4:20 - What does this paper do 7:00 - Linear probes 11:15 - Linear probe results 17:10 - Results 22:25 - Learned Features https://arxiv.org/abs/1904.13132 Abstract: We look critically at popular self-supervision techniques for learning deep convolutional neural networks without manual labels. We show that three different and representative methods, BiGAN, RotNet and DeepCluster, can learn the first few layers of a convolutional network from a single image as well as using millions of images and manual labels, provided that strong data augmentation is used. However, for deeper layers the gap with manual supervision cannot be closed even if millions of unlabelled images are used for training. We conclude that: (1) the weights of the early layers of deep networks contain limited information about the statistics of natural images, that (2) such low-level statistics can be learned through self-supervision just as well as through strong supervision, and that (3) the low-level statistics can be captured via synthetic transformations instead of using a large image dataset. Authors: Yuki M. Asano, Christian Rupprecht, Andrea Vedaldi Thumbnail Image: https://commons.wikimedia.org/wiki/File:Golden_Gate_Bridge_during_blue_hour_(16_x_10).jpg Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://www.bitchute.com/channel/yannic-kilcher Minds: https://www.minds.com/ykilcher

More from Yannic Kilcher

See all

13:46I BUILT A FULLY AUTOMATIC MANSPLAINERMar 6, 2026

Live2:33:37Traditional X-Mas StreamDec 29, 2025

Live0:00Traditional Holiday Live StreamDec 28, 2025

← Back to Yannic Kilcher