[Pycon] [new paper] "Umit Mert Cakmak" - Recent advancements in NLP and Deep Learning: A Quant's Perspective

[info a pycon.it]

Title: Recent advancements in NLP and Deep Learning: A Quant's Perspective
Duration: 60 (includes Q&A)
Q&A Session: 15
Language: en
Type: Talk

Abstract: Summary
=======
There is a gold-rush among hedge-funds for text mining algorithms to quantify textual data and generate trading signals. Recently, harnessing the power of alternative data sources became crucial to find novel ways of enhancing trading strategies.

With the proliferation of new data sources, natural language data became one of the most important data sources which could represent the public sentiment and opinion about market events, which then can be used to predict financial markets.

In this talk, we will review the recent advancements in NLP and explore the usage of deep learning architectures employed for predicting the financial markets. In each part, I will give examples of related python usage.

Talk is split into 5 parts;

1. Who is a quant and what kind of NLP work they perform?
----------------------------------------
Quants use mathematics and statistics to create algorithmic trading strategies.

Due to recent advances in available deep learning frameworks and datasets (time series, text, video etc) together with decreasing cost of parallelisable hardware, quants are  experimenting with various NLP methods which are applicable to quantitative trading.

In this section, we will get familiar with the brief history of text mining work that quants have done so far and recent advancements.

2. How deep learning is really changing NLP?
---------------------------
In recent years, data processing and representation methods are vastly improved. Rather than working with very high dimensional sparse matrices and suffering from curse of dimensionality, distributional vectors a.k.a. word embeddings are much easier to work with.

Models such as Word2vec or GloVe helps us create word embeddings from large unlabeled corpus which represent the relation between words, their contextual relationships in numerical vector spaces and these representations not only work for words but also could be used for phrases and sentences.

In this section, I will talk about what has been changed in recent years in terms of workflow when building NLP models.

3. Let's get dirty with embeddings
-----------------------
There are 2 popular methods being used in community to produce word embeddings, Word2vec and GloVe, with Word2vec being the pre-dominant method in the community.

Word2vec is simple 2-layer neural network to be trained with word pairs. When you feed this network with huge amount of data, number of neurons in your hidden layer will be equal to number of dimensions for each vector representation.

In this section, I will talk about inner workings of these models and important points when creating domain-specific embeddings (e.g. for sentiment analysis in financial domain).

4. Performant deep learning layer for NLP: The Recurrent Layer
-----------------------------------
RNNs successfully model textual data of varying length and this property makes them suitable for sequence modeling tasks which researchers have to deal with unbounded context in NLP tasks.

RNNs can capture and hold the information which was seen before (context) and it is very important in NLP applications. However, Simple RNN does not work well for more than 10 time lags usually and Long Short Term Memory (LSTM) networks deal with this problem which is a special type of RNN. LSTM networks can understand the context even if words have long term dependencies, words which are far back in their sequence.

In this talk, I will compare LSTMs with other deep learning architectures and will look at LSTM unit from a technical point of view.

5. Using all that to make money
---------------------------------------
Financial news, especially if it's major, can change the sentiment among investors and affect the related asset price with immediate price corrections.

For example, what's been communicated in quarterly earnings calls might indicate whether the price of share will drop or increase based on the language used. If the message of the company is not direct and featuring complex sounding language, it usually indicates that there's some shady stuff going on and if this information extracted right, it's a valuable trading signal. For similar reasons, scanning announcements and financial disclosures for trading signals became a common NLP practice in investment industry.

In this section, I will talk about the various data sources that researchers can use and also explain common NLP workflows and deep learning practices for quantifying textual data for generating trading signals.

I will end with summary with application architecture in case anyone would like to implement similar systems for their own use.


Tags: [u'nltk', u'nlp', u'Deep-Learning', u'Keras', u'Python', u'machine-learning', u'data-science', u'spaCy']