Machine learning is an incredibly dynamic field, and we participate in a variety of events to stay up to speed. The coolest is probably our weekly FeatureX ML seminar (papers listed below), where a member of our team or the local machine learning community presents an important paper and we discuss over drinks and snacks. If you would like to be added to our email invitation list, please email Emily Rogers.

In addition, we attend conferences like NIPS and ICLR, research talks next door at MIT, and meetups in the Boston AI ecosystem. We even sponsor the weekly "ML Tea" at MIT, where new research is presented in an informal setting.

Internally, the following are current and possible future research projects at FeatureX:

**The Structure of CNNs**

Deep learning, and CNNs in particular, have shown incredible promise in how they extract high-level features from local, hierarchically-structured data (e.g. how pixels relate to each other in images). Once one moves away from playing with ImageNet, however, the best structure for a CNN becomes a very interesting research problem.

**Adversarial Learning**

Generative/discriminative adversarial learning (GAN) can be used to extract high-level features from image data. We are also exploring the use of adversarial learning to extract high-level features from non-image data.

**Coevolutionary Learning**

In a manner similar to GAN, coevolutionary rule induction can be used to infer high-level features in data. We are exploring the use of adversarial evolutionary computation to extract features from a wide variety of datasets.

**Super Resolution**

We are exploring the use of deep learning and GAN as methods of enhancing the human-understandable aspects of many kinds of data, including remote sensing image data.

**Autoencoders**

Deep learning autoencoders can be used to reduce dimensionality, but can also be used to learn a trajectory through time that describes the changing states of loosely coupled time series data. The results can be very useful for developing an intuitive understanding of complicated data.

**Autoencoders, part 2**

Deep learning isn't the only technology that can learn high-level abstractions from detailed data. Evolutionary computation can also be used, especially when doing rule induction. We are exploring the use of rule induction to extract high-level features from data in a manner similar to neural network autoencoders.

**Predicting Financial Asset Price Movements**

Mathematicians and financial engineers have delved into the use of mathematical models and computer simulations to predict price movements for many years, but machine learning is changing the game. We are meeting these challenges head on with small data sets, low signal, high noise, and high profit.

**Deviation From a Known Distribution**

A collection of data points will have a sampled probability distribution, and perhaps the distribution is even known a priori. However, when spread over time, this collection of datapoints can vary locally over time from the overall distribution. We are working on creating reliable algorithms to detect and characterize these deviations.

**Taking Multiple Comparisons Into Account When Comparing Two Datasets**

Comparing two datasets, when each is independently drawn from an assumed distribution, is relatively straightforward. However, when the selection criteria for drawing data involves multiple comparisons, things get more complicated and ordinary statistical techniques no longer apply. Terms like Akaike information criterion, Bayes factors, White's reality check, or family-wise error rates are relevant here. Accounting for multiple comparisons is critical when determining the reliability of backtested results.

**Cross Asset Prediction and Machine Learning**

Working with multiple financial assets and making cross asset predictions creates a number of challenges for machine learning approaches, including auto-correlation, cross-asset correlation, dependent variable overlap and double counting, and survivorship bias. These issues lie at the intersection of machine learning and statistics, and create a very interesting set of problems to solve.

**RNNs and LSTMs**

Financial data is a collection of time series data. Help us explore the use of RNNs, including LSTMs and Neural Turing Machines, on financial data. One key challenge is non-local hierarchical structure, since the structure in financial data is not as predictably local as it is in images, audio or linguistic data.

**Reinforcement Learning and Market Dynamics**

Most approaches to asset management, including both quantitative and AI approaches, make the assumption that the past can be used to predict the near term future. A different view one can take is that current market behavior impacts future market behavior. Reinforcement learning can play a role in modeling market dynamics over time.

**Application of GPUs**

At FeatureX, we use a mix of open source and proprietary algorithms. Some of our algorithms could possibly be ported to run on GPUs. Come help us explore using GPUs to perform search in addition to gradient descent.

**Sparse Distributed Representations**

When building models of dynamic behavior over time, complicated factors can combine together in arbitrarily complex ways. An interesting way to model these dynamics is to use sparse distributed representations (SDRs). We feel it will be possible to simultaneously learn both SDRs and how they combine over time to produce behavior. This kind of research overlaps the work being done on hierarchical temporal memory (HTM) systems.

**Deep Learning and Stochastic Gradient Descent**

Deep learning systems can be very effective in a number of domains, but little is understood about the search landscape traversed by stochastic gradient descent techniques like back propagation, which is slow and requires a lot of data. Recent work by OpenAI suggests that there may be other approaches to optimizing the parameters of a deep learning system that may be more effective, and that allow some of the constraints of deep learning systems to be relaxed.

**Hierarchical Composition of Features**

An appealing intuition for why deep learning systems are effective is that they compose higher-level features from low-level features, layer by layer. A downside is that this approach doesn't lend itself to human understanding, nor does it provide for easy inclusion of domain knowledge in the form of inductive bias. We are exploring alternative methods of feature composition that allow inductive bias, as well as separable induction of high-level features using ML.

## FeatureX Machine Learning Seminars

Every week FeatureX hosts an invitation-only Machine Learning seminar. To date, we have presented on the following topics:

### Generative Adversarial Networks

### Visualizing and Understanding Convolutional Networks

### Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

### Conditional Image Generation with PixelCNN Decoders

### Deep Residual Learning for Image Recognition

### Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

### Bayesian Optimization

### LSTM Networks

### Neural Image Caption Generation with Visual Attention

### Neural Turing Machines

### Spatial Transformer Networks

### Synthesizing the preferred inputs for neurons in neural networks using deep generator networks

### Hierarchical Temporal Memory Including HTM Cortical Learning Algorithms

### Value Iteration Networks

### Latent Dirichlet Allocation

### Generative Shape Models: Joint Text Recognition and Segmentation with Very Little Training Data

### End-to-end People Detection in Crowded Scenes

### Wasserstein GAN

### Mastering the Game of Go with Deep Neural Networks and Tree Search

### Using Fast Weights to Attend to the Recent Past

### Generalization and Equilibrium in Generative Adversarial Nets (GANs)

### Understanding Deep Learning Requires Rethinking Generalization

### Evolution Strategies as a Scalable Alternative to Reinforcement Learning

### Convolutional Sequence to Sequence Learning

### Inferring and Executing Programs for Visual Reasoning

**Deep Nets Don’t Learn via Memorization****Opening the Black Box of Deep Neural Networks via Information****Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex****Semi-supervised Learning with Deep Generative Models****Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks****A simple neural network module for relational reasoning****Unsupervised Learning of Disentangled Representations from Video****InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets****Human-level control through Deep Reinforcement Learning****Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs****Robust Imitation of Diverse Behaviors****Automated Curriculum Learning for Neural Networks****Self-Normalizing Neural Networks****Wavenet: A Generative Model for Raw Audio****Neural Style Transfer: A Review****The Shattered Gradients Problem: If resnets are the answer, then what is the question?****Towards Biologically Plausible Deep Learning****AlphaGo and AlphaGo Zero****Don’t Decay the Learning Rate, Increase the Batch Size****FiLM: Visual Reasoning with a General Conditioning Layer****Multi-level Residual Networks from Dynamical Systems View****The Numerics of GANs****FeUdal Networks for Hierarchical Reinforcement Learning**### Visualizing the Loss Landscape of Neural Nets