Using AI to exploit the power of news to boost trading activities

This article offers some insight into the mechanisms of news production and how news analytics technology is being advanced to help traders exploit the rich and powerful insights that news offers on the global financial markets.

The old and the new
If we step back in time to before the internet, finding information often required a trip to your local library. Remember stepping in to those grand, often imposing buildings to find corridor after corridor, shelf after shelf of books looming towards you. You know what you want to read about but you’re just not sure where to start. Cue a bespectacled librarian to whom you explain what you are looking for and they miraculously take you to the exact spot that you need.

Ok, so admittedly, the librarian is very much a stereotype, but ‘infobesity’ has amplified in today’s information age and professional news readers have had to find their own, modern day version of the librarian. But knowing where to look is only half of the battle today. With so much competition, warranted or not, being able to analyse the news and extract uncompromising value from it are the key drivers in today’s advancing news analytics capabilities.

So how has news analytics technology evolved and what capabilities can be exploited? How do they bolster traders’ activities and why should a professional, augmented news feed still be at the top of a trader’s tool box?

The Value of Signage
Just as a traditional library uses physical signage in their building, on their shelves and on the spines of their books, a professional news feed also uses a navigational system of codes, known as ‘metadata’.

During the editorial process these codes are assigned to every story, either by an editor or automatically by a machine or some combination. The codes identify the types of news content being published and the entities that are being written about. For example, an article on a company could have codes for the industry, sector, asset type, entity name and location. Equally important, each story also includes codes which identify the type of content the article is, such as breaking headlines or commentary and analysis, for reasons which we will come on to shortly.

Dow Jones deploys a rich, structured set of codes across their newswires. Not only does this help their readers to find the content they want, but it also helps Dow Jones to segment and prioritise content, so that their readers cut out the noise and focus their attention where it matters at that moment.

For example, important news stories are coded as ‘significant’ to draw attention to newsworthy articles within trader workflow. Or in the event of a breaking news story where all the details are yet to be established and multiple headlines are pushed out in quick succession, a sophisticated method of linking each headline, known as ‘chaining’, is used. This chaining of codes help publishers and readers to understand the sequencing of events as the story unfolds and to help distinguish stories from potentially multiple reporting streams.

With the help of natural language processing (NLP) technology, the application of codes can offer even greater benefits. For example, when a new theme such as ESG (Environment Social Governance) emerges, historical articles can be ‘back coded’ to give readers insights that add value today.

Or, in another example, NLP can be used to apply codes to stories that are not immediately obvious to the editor at the time of writing. For example, a fire in a mobile phone factory is identified as being involved in an ESG scandal.

The creation of alternative data sets

Reliance on coding alone however, won’t necessarily provide traders with sufficient advantages needed to profit from the trades. It’s clear that coding helps to segment and filter news content in much more palatable ways but to augment value from the news in an actionable way requires the help of NLP technology.

NLP technology has given rise to news-based alternative datasets, such as sentiment data. This is a useful, alternative data set that can help traders understand and act on price behaviour, thereby acting as a directional tool. A typical assumption is that if the news is positive for an asset, its price will rise and vice versa. Sentiment data can improve price forecasts as well as performance.

However, when implementing sentiment analysis algorithms, one must carefully analyse the data sets being used. Poor quality content will adversely affect the enrichment process and may not bring any value to users. Demand therefore of high quality news sources like Dow Jones Newswires has continued to grow over the last few years despite the rise in open access news.

Since the first news analytics technology was developed more than 15 years ago, the process of sentiment classification has continued to evolve. From rules-based approaches using dictionaries of polarised words to sophisticated deep learning-based approaches using “bag-of-words” or word embeddings text vectorization methods. This has allowed sentiment to be captured with higher accuracy and granularity but excitingly, it has also given rise to new analytical opportunities.

Technology
Navigating the news landscape of the financial markets can be likened to navigating the English Channel. – Image by Imray.
Understanding the complex web of multi-hop relationships
Having evolved from simply analysing news for sentiment, NLP is now widely used to better identify and understand the complex and often obscured relationships between entities, people, geographies and wider themes, concepts or topics.

This has come to be known as a ‘knowledge graph’, made famous by Facebook, where disparate data points are connected to create a map of relationships and other metrics that affect that relationship, including sentiment data.

As a trader, if we understand the relationship between assets we can then predict the effect of one asset on another. This could be summarised as ‘semantic news arbitrage’. FX traders following EURUSD, will already be familiar in the knowledge that news about the EUR will impact the USD and vice versa; therefore following news on both currencies is a worthy investment of their time. However, understanding the wider influences that are affecting the pair and then what other news is important to follow becomes a far greater challenge for traders.

Equity traders, whose assets are knowingly affected by the events of their supply chain network, complex corporate structures and global market exposure, are likely to be more familiar with this dilemma than currency traders. A news story breaking in China about a fire in a battery factory for mobile phones may not seem important at first glance, but as a supplier to Apple it would be a hugely significant story.

This automatic identification of associated assets and the tracing of multi-hop relationships is an exciting development in news analytics, an approach that requires multiple semantic web technologies including Named Entity Recognition (NER) and NLP to be adopted across a wide range of financial disciplines. For traders, the ability to identify these connections can expose opportunities as well as help manage risks.

Solutions like graph databases (a database designed to treat the relationships between data as equally important as the data itself) and query languages, offering an almost constant-time operation complexity that efficiently analyses millions of connections per second, have allowed for the democratization and more widespread adoption of these technologies.

Conclusion
News analytics technology has undergone significant improvements in recent years as it evolves to keep up with investor demands for new alternative data sets. Graph analytics in particular continue to be a trending discipline in data science, and Gartner predicts that graph processing and graph database management systems application will continue growing at 100% annually through 2022, allowing for more efficient modelling, exploration and querying of interlinked data.

As with sentiment data analysis, we are likely to see further evolution in both the technology and its applications in trading. And just like library signage was an early form of meta tagging, each evolution will be the foundation of something new, offering traders more and more value from one of the oldest tools in the industry: good, high-quality journalism.