Distill Fine-Grained Features Without Supervision — Previously, I have written several articles briefly discussing self-supervised learning and, in particular, contrastive learning. What was not yet covered, however, was a concurrent branch of self-supervised approach using interactions of multiple networks that seems to emerge and excel recently. As of today, one of the state-of-the-art training methods is…

When a neural network uses so much data it becomes “universal” — Pre-defined classes and categories: this is the limitation where new classes can only be classified by machine learning and neural networks after retraining. …

Can they one day replace photos? — The deep learning era began through the advancements it brought in traditional 2D image-recognition tasks such as classifications, detections, and instance segmentations. As the techniques matured, the research in deep-learning-based computer vision has been shifted towards fundamental 3D computer vision problems — one of the most notable being synthesising new…

Demystifying each learning task — The exponential number of research and publications have introduced many terms and concepts in the domain of machine learning, yet a number of them have degenerated to merely buzzwords without people fully understanding their differences. This article demystifies the four core regimes in the field of machine learning — supervised…

Image Denoising with No Data and a Random Network — Deep learning and neural networks have been tightly associated with big data. Whether it is image classification of language translation, you almost always require a vast quantity of data to boost the task accuracy for the model to be applicable to real-world datasets. Even under few-shot or one-shot scenarios, the…

Look deeper into your deep models — This is a collaborative article with Neptune.ai. Original article can be found here. The blossom of the deep learning era began in 2012 when Alex Krizhevsky created a convolutional neural network that boosted the accuracies in image classification by more than 10%. The drastic success was soon followed by other…

The frontier of deep learning techniques in computer vision — What is currently going on in the computer vision community? If you are an avid computer vision enthusiast like me, here are several of my favourite papers in recent years that I believe will have massive impacts going forward. Disclaimer: These papers are what I think of as “foundational” papers…

Learning features from wildlife to generate new creatures — Generative networks have opened up a plethora of fun projects one can do with deep learning — one of which is generating images based on a set of realistic photos. We can see in our previous article that a very simple GAN network can have good results on generating numbers…

The exponential progress of contrastive learning in self-supervised tasks — Deep learning research has been steered towards the supervised domain of image recognition tasks, many have now turned to a much more unexplored territory: performing the same tasks through a self-supervised learning manner. One of the cornerstones that lead to the dramatic advancements in this seemingly impossible task is the…