This is the last post in the machine learning summer school series. Meanwhile if you haven’t yet seen the previous posts, do have a look at the part I (poster presentation) and part II (social events). Here, I share the ideas and insights I gained from the lectures and interactions with tutors and peers. I’ve focused on the topics related to my research, mainly in the area of online learning, deep models for unsupervised learning and applied machine learning.
Online learning
- Francis Bach’s lecture on Optimisation
Despite being very dense and mathematical in content, my attention was drawn to the parts of the lecture focusing on stochastic approximation and its relationship to online learning. Francis discussed the catastrophic forgetting that occurs over time in online learning and possible approaches to tackle it. Lecture slides can be found at the link.
- Lizhe Sun – Online Regression
During the poster session, I met Lizhe Sun, a Statistics PhD student at the University of Florida who is developing an online learning framework based on running averages to solve regression problems. In his opinion, this algorithm would not be suitable for deep learning.
Deep learning model for unsupervised learning
- Sebastin Nowozin’s lecture
Generative Adversial Networks (GANs) can perform unsupervised learning of images as they are able to differentiate between features. Intuitively, it is some kind of clustering based on similar features. It further strengthened my thoughts that my problem was achievable using deep models. Find the code and lecture slides on his github page.
- David Cabello – Face Anti-spoofing
This was one of the most useful contacts I made and luckily he was interested in my problem domain. David explained to me how hand-crafted features are extracted by domain experts. His work was still at the phase of supervised learning but he was hoping to start looking into unsupervised learning in the future. We brainstormed about how Convolutional Neural Network (CNN) could achieve fully unsupervised learning, and he reasoned that the embedding in the feature space could be clustered using some similarity measurement. He pointed me to Google’s FaceNet paper.
- David Stutz – the 3D car model
This one was helpful because it showed how a weak supervision apporach could be used to train a CNN model. The CNN model is trained on labeled shapes of 3D cars and when fed with partial point clouds reconstructs the 3D image of a car. His work is already published and available at the link.
Other interesting insights
- Causal inference and applied Machine Learning (ML)
I found the causal inference class by Joris Mooij quite dense. It wasn’t until Suchi Saria lectured on Causal inference for personalised decision making that I began to appreciate the relevance of the topic to my research domain, if not now possibly in the future. The intuition she gave is that before ever starting to look into your data, one must first step away from the data. Think: what is the question you are trying to solve, and would it be possible to estimate given the data? This involves first mapping out the causal variables using domain knowledge.
The industry folks who presented showed some interesting application of ML in the industry. Financial and Logistics were the most prevalent. The famous research institution, Instituto de Ingeniería del Conocimiento (IIC), presented how they are using LYNX to solve fraud detection. BBVA and Prowler.io presented other interesting industrial solutions using ML.
I’m glad you followed through to this last post of the summer school series. It’s going to be really intensive this month of October as I will be working on a collaborative mini-project and preparing for the coursework of my teaching module. Look out for my forthcoming posts.