Machine learning is an incredibly dynamic field, and we participate in a variety of events to stay up to speed. The coolest is probably our weekly FeatureX ML seminar (papers listed below), where a member of our team or the local machine learning community presents an important paper and we discuss over drinks and snacks.
In addition, we attend conferences like NIPS and ICLR, research talks next door at MIT, and meetups in the Boston AI ecosystem. We even sponsor the weekly "ML Tea" at MIT, where new research is presented in an informal setting.
Internally, the following are current and possible future research projects at FeatureX:
The Structure of CNNs
Deep learning, and CNNs in particular, have shown incredible promise in how they extract high-level features from local, hierarchically-structured data (e.g. how pixels relate to each other in images). Once one moves away from playing with ImageNet, however, the best structure for a CNN becomes a very interesting research problem.
Generative/discriminative adversarial learning (GAN) can be used to extract high-level features from image data. We are also exploring the use of adversarial learning to extract high-level features from non-image data.
In a manner similar to GAN, coevolutionary rule induction can be used to infer high-level features in data. We are exploring the use of adversarial evolutionary computation to extract features from a wide variety of datasets.
We are exploring the use of deep learning and GAN as methods of enhancing the human-understandable aspects of many kinds of data, including remote sensing image data.
Deep learning autoencoders can be used to reduce dimensionality, but can also be used to learn a trajectory through time that describes the changing states of loosely coupled time series data. The results can be very useful for developing an intuitive understanding of complicated data.
Autoencoders, part 2
Deep learning isn't the only technology that can learn high-level abstractions from detailed data. Evolutionary computation can also be used, especially when doing rule induction. We are exploring the use of rule induction to extract high-level features from data in a manner similar to neural network autoencoders.
Predicting Financial Asset Price Movements
Mathematicians and financial engineers have delved into the use of mathematical models and computer simulations to predict price movements for many years, but machine learning is changing the game. We are meeting these challenges head on with small data sets, low signal, high noise, and high profit.
Deviation From a Known Distribution
A collection of data points will have a sampled probability distribution, and perhaps the distribution is even known a priori. However, when spread over time, this collection of datapoints can vary locally over time from the overall distribution. We are working on creating reliable algorithms to detect and characterize these deviations.
Taking Multiple Comparisons Into Account When Comparing Two Datasets
Comparing two datasets, when each is independently drawn from an assumed distribution, is relatively straightforward. However, when the selection criteria for drawing data involves multiple comparisons, things get more complicated and ordinary statistical techniques no longer apply. Terms like Akaike information criterion, Bayes factors, White's reality check, or family-wise error rates are relevant here. Accounting for multiple comparisons is critical when determining the reliability of backtested results.
Cross Asset Prediction and Machine Learning
Working with multiple financial assets and making cross asset predictions creates a number of challenges for machine learning approaches, including auto-correlation, cross-asset correlation, dependent variable overlap and double counting, and survivorship bias. These issues lie at the intersection of machine learning and statistics, and create a very interesting set of problems to solve.
RNNs and LSTMs
Financial data is a collection of time series data. Help us explore the use of RNNs, including LSTMs and Neural Turing Machines, on financial data. One key challenge is non-local hierarchical structure, since the structure in financial data is not as predictably local as it is in images, audio or linguistic data.
Reinforcement Learning and Market Dynamics
Most approaches to asset management, including both quantitative and AI approaches, make the assumption that the past can be used to predict the near term future. A different view one can take is that current market behavior impacts future market behavior. Reinforcement learning can play a role in modeling market dynamics over time.
Application of GPUs
At FeatureX, we use a mix of open source and proprietary algorithms. Some of our algorithms could possibly be ported to run on GPUs. Come help us explore using GPUs to perform search in addition to gradient descent.
Sparse Distributed Representations
When building models of dynamic behavior over time, complicated factors can combine together in arbitrarily complex ways. An interesting way to model these dynamics is to use sparse distributed representations (SDRs). We feel it will be possible to simultaneously learn both SDRs and how they combine over time to produce behavior. This kind of research overlaps the work being done on hierarchical temporal memory (HTM) systems.
Deep Learning and Stochastic Gradient Descent
Deep learning systems can be very effective in a number of domains, but little is understood about the search landscape traversed by stochastic gradient descent techniques like back propagation, which is slow and requires a lot of data. Recent work by OpenAI suggests that there may be other approaches to optimizing the parameters of a deep learning system that may be more effective, and that allow some of the constraints of deep learning systems to be relaxed.
Hierarchical Composition of Features
An appealing intuition for why deep learning systems are effective is that they compose higher-level features from low-level features, layer by layer. A downside is that this approach doesn't lend itself to human understanding, nor does it provide for easy inclusion of domain knowledge in the form of inductive bias. We are exploring alternative methods of feature composition that allow inductive bias, as well as separable induction of high-level features using ML.
FeatureX Machine Learning Seminars
Every week FeatureX hosts an invitation-only Machine Learning seminar. To date, we have presented on the following topics: