Tom Schaul, M.Sc.

Tom Schaul, M.Sc. is a Ph.D. student in machine learning at IDSIA, supervised by Jürgen Schmidhuber. He works on reinforcement learning, optimization, neural networks, artificial curiosity, and related topics. Tom is winner of the 2009 GECCO Best Paper Award and winner of the Bronze medal at the 2000 International Mathematical Olympiad.
 
Tom’s research areas include:
 
Reinforcement learning
Within the broad framework of reinforcement learning he’s most interested in policy-gradient methods (that don’t use value functions, e.g. reward-weighted regression). He’s currently working on new, general benchmarks that should both make the plethora of RL algorithms more comparable, and clearly show the limitations of each, thereby simplifying the choice of algorithm for particular real-world problems.
 
Black box optimization
In black box optimization he optimizes a fitness function in the absence of any knowledge about it. It is only possible to evaluate the function at a limited number of points in the (high-dimensional) parameter space. The goal is to find parameters that correspond to high fitness with as few evaluations as possible. He is working on improving the state of the art for this problem with novel algorithms (e.g. , FEM).
 
Artificial curiosity
Humans explore the world quite efficiently, going for areas/topics that they do not know well, but where they also expect to be able to learn more about the world. In AI this idea has been formally introduced as artificial curiosity. He’s currently trying to improve the exploration strategies of existing algorithms in reinforcement learning and optimization based on this.
 
Multi-dimensional neural networks
Recurrent neural networks are among the most powerful tools for sequence learning. At IDSIA, he developed multi-dimensional RNNs that take this idea one step further and can elegantly and compactly handle the spacial structure of multi-dimensional sequences, like images, videos, or fMRI scans. He showed that they scale nicely on board games.
 
Games
Tom has always been interested in games, and they turn out to be very appropriate application domains for a variety of machine learning techniques. He’s particularly interested in board games with spacial structure and patterns (like Go) or complex graph structure (like Sokoban).
 
He coauthored PyBrain, Metalearning, Towards Practical Universal Search, Frontier Search, Efficient Natural Evolution Strategies, Stochastic Search using the Natural Gradient, Scalable Neural Networks for Board Games, Ontogenetic and Phylogenetic Reinforcement Learning, Fitness Expectation Maximization, Countering Poisonous Inputs with Memetic Neuroevolution, Natural Evolution Strategies, and Episodic Reinforcement Learning by Logistic Reward-Weighted Regression. Read the full list of his publications!
 
Tom earned his Master’s in Computer Engineering with the thesis Evolving a Compact Concept-based Sokoban Solver at École Polytechnique Fédérale de Lausanne in 2005. He was an exchange student at University of Waterloo from 2002 to 2003 and at Columbia University from 2004 to 2005. He attended Machine Learning Summer School at Cambridge University in 2009.