Research Blog Events
About Careers Newsroom Get Started

Big Data & Machine Learning Revolution

View an extract of this session held at the London Big Data and Machine Learning Revolution event in April 2018.You can also access the full video and slides.


We're extremely excited to have put together an amazing group of individuals, mostly financial practitioners as well as academics that have truly worked on practical applications of big data and machine learning. We have speakers and representatives from J.P.Morgan, Jefferies and Morgan Stanley to name a few.

Full Video access slides

Big Data Machine Learning Finance

The attending delegates from the London Big Data and Machine Learning Symposium is quite interesting if you look at the distribution of the members that attended today. The majority here are from the buy side:

intro big data and machine learning revolution

From the buy side, the the majority were Quants as well as Fund Managers:

intro big data and machine learning revolution

Big Data in Finance

Big data is not just about the data itself, it's also about the analysis that one can do on that data and most of the raw information that we have is unstructured which makes it very difficult to make sense of it. In the case of finance text news and filings when it's video, we're talking about CNBC or Bloomberg broadcast of CEO interviews, and when it comes to audio, we're talking about earnings transcripts and press calls. But images are also quite interesting. And more importantly, trying to combine both the tabular and the more sort of structured data like fundamental data and market data that we've been accustomed to. Then to combine that with the more alternative unstructured information and to try to make sense of it all.

Machine Learning in Finance

Now when we think about machine learning in finance one of the key things to bear in mind is that it's all about the learning part. You know how do you build an algorithm or how to build an environment that can actually learn from something. And most of the emphasis that we've made has really been about learning. But I think we should also really focus on the teaching part of it and at the moment, one of the reasons why many of the machine learning techniques don't really work is because we're not very good at training them or the training sets are just not enough.


What are we trying to say? Well a revolution is defined as a fundamental change in an organizational structure that takes place in a relatively short amount of time and that's when people essentially revolt against the current norm or the current order. And I think there's a revolution taking place in finance specifically because of the following five signs:

1. Unstructured Data

The first sign I think is that there is explicit discontent with the traditional investment management process right whether it's a lack of accountability or lack of understanding, a lack of benchmarks as to how money is being managed and how money is being invested. I think overall the old ways of managing money are essentially changing and people are looking for more sophisticated, more quantitative, more accountable ways of doing it.

2. Digital Oligarchy

The second sign I'm seeing is that a small group of companies actually control most of the data right now whether it's Google or Facebook. And the more retail space, whether it's the exchanges themselves, or the large data providers, they essentially control over 90 percent of the data that is used in finance. And I think most participants want a fair market and are looking for access and looking for ways to actually get that data into their models. But most of these vendors or most of these exchanges control it in ways that make it harder. And your average retail investor is certainly not getting the benefit of the big data boom.

3. Data Growth

The third sign that we see is essentially data has been growing at a frantic pace and specifically unstructured content and as data grows it becomes unmanageable and more difficult to use. And it frustrates people and it makes it essentially difficult to create a strategy that is predictable and that we can actually understand. And as data is growing we're also seeing a spike in the availability of new and alternative datasets, where before the large amounts of information came from structured market data and perhaps fundamental data.

4. Alternative Data

Now alternative data is significantly larger than those two sources and it's extremely difficult to process. And more importantly a lot of these data vendors don't really understand how to make it work or how to map it to some of the actual securities that we trade. So the fact that there is availability and the fact that people are realizing that there are other ways of analyzing markets, not just fundamentals, and not just through market data, means a new opportunity for people to do things differently.

5. Decision Making

Lastly, we still despite all of the technological progress and the talk about AI, machine learning, and big data, we still make decisions based on institution. The large majority of investments are still very much discretionary, and the management of money is still done by a few individuals that make the decisions. It's time for a change. It's time for more a quantitative, systematic way of looking at it things, a way that actually provides accountability, and more importantly, a way to understand that the way we are doing things is right.

Full Video access slides

Request Event Materials

Request Event Materials

We will process your personal data with the purpose of managing your personal account on RavenPack and offering our services. You can exercise your rights of access, rectification, erasure, restriction of processing, data portability and objection by emailing us at For more information, you can check out our Privacy Policy.

Discover the RavenPack Platform

Choose from over 180 curated datasets covering a variety of different topics.

Explore Now