Skip to main content
Insights

Early Warning Signals from Unstructured Data: Spotting Future Disruption in the Global Pork Market

By October 6, 2020September 6th, 2023No Comments

Case Study Overview

Market disruption often stems from innovative experimentation with new technologies in combination with new business models or an event in a remote and seemingly unconnected part of the world triggering a chain reaction.

Enterprises rely upon traditional and limited sources of business intelligence, such as market prices, as their first signal to change within their markets. Driven off structured data, these signals invariably arrive too late, leaving entities operating in reactive mode, often with disastrous consequences.

The following case study indicates how emerging capabilities that harness artificial intelligence (AI) to create insight from unstructured data are enabling a new generation of early warning signals. These early warning signals place enterprises on the front foot and enable them to proactively respond to survive or capitalize on rapid and hitherto unforeseen market change. The case study focuses on how European organisations with the right insight and business intelligence could have benefited and reacted differently to the 2018 outbreak of African Swine Fever (ASF) in China.

The Tech: DeepInsight

AMPLYFI’s proprietary business-intelligence and market-research platform, DeepInsight, uses AI to go beyond the reach of standard search engines to unlock the Deep Web. It automatically finds and mines vast volumes (potentially millions) of relevant documents (structured and unstructured) before simultaneously reading and analysing them all. The following case study centres on a single internet search term: “global pork market”.

Using unsupervised learning, DeepInsight discovered and quantified over 1,000 key terms and topics, as well as every location, organisation, and person mentioned alongside them (quantifying the changing strength of their connections over time). In doing so, it tracked the unfolding events that eventually caused at least one company, German specialist pork product producer Ital-Meat Fleisch, to enter voluntary administration in January 2020.

Spotting and Tracking the Spread of African Swine Fever

In mid-2018, China reported its first ever official case of ASF. The disease subsequently spread to northern India and Papua New Guinea for the first time too. Originally identified in East Africa in the early 1900s, ASF is highly contagious and kills almost 100% of the animals it infects. No vaccine is currently available. In analysing tens of thousands of documents directly relevant to the global pork market, DeepInsight identified topics specifically related to ASF and the global locations with which they were associated. It was then able to plot its spread over time (see Figure 1).

Figure 1: African Swine Fever Emerges in China to Spread Throughout the Region

Through regular (monthly, quarterly etc.) updates, analysts using DeepInsight would have been alerted early to the emergence of ASF in China in 2018 and been able to continually monitor its geographic spread. In addition, DeepInsight’s proprietary ranking algorithms would have directed analysts to the most relevant documents to enable deep-dive desktop research to be conducted efficiently.

China’s Response

Chinese authorities officially reported the outbreak of ASF in its domestic pigs in Liaoning, north east China, on 3rd August 2018. This triggered a nationwide programme to contain the impact of the disease. In March 2020, less than 2 years later, Reuters reported meat industry estimates that ASF had “shrunk China’s 440-million-hog herd by more than half” through managed culls, early slaughtering, or lost to the disease.[1] The vast unplanned loss of livestock eventually impacted domestic pork prices. However, due to the gradual spread of the disease, existing stocks being progressively exhausted, and the policy of early slaughtering, there was a 13-month lag between China’s first reported case and the first significant rise in its prices (see Figure 2).

Figure 2: Impact of African Swine Fever on China Domestic Pork Prices

Global Pork Market Disruption

At the start of 2020, Vietnam, the Philippines, and a wide swath of Eastern Europe had become focal points of the ASF outbreak. By March 2020, the epidemic had taken a quarter of the world’s pigs off the market, hurt livelihoods, caused meat prices to spike globally, and pushed food inflation to an eight-year high. With shortages of domestic supply, rising prices and overseas markets increasingly becoming off limits, Chinese buyers turned to European and US markets to source pork, placing pressure on those markets.

In analysing the tens of thousands of structured and unstructured documents harvested from the internet, DeepInsight extracted and quantified hundreds of relevant topics including “Pork Prices”, “Outbreak of African Swine Fever”, “Pork Exports”, and “Pork Exports to China”. It then used a range of proprietary AI techniques to calculate a weighting for each topic. Called Significance, this weighting can be viewed as a measure of the intensity with which each topic was being discussed globally (see Figure 3). The historic trends that it also automatically generated would allow analysts to monitor events continually and highlights how the global dialogue for each topic increased following that first recorded case in China. Collectively, these topics signal and corroborate potential market disruption.

Figure 3: Impact of ASF in China on Global Dialogue

Benefit of Early Warning

Most organisations rely on traditional sources of business intelligence generated from standard structured datasets—such as market indices, commodity prices, exchange rates, interest rates etc.—as their first signal to impending disruption to their industry. Unfortunately, such signals arrive too late for meaningful and proactive responses to be implemented. However, they are invariably preceded by deep and widespread debate or commentary speculating how markets might be impacted by unfolding events or changing circumstances. Often the strength of the dialogue itself creates sufficient sentiment momentum to cause markets to change. The challenge for traditional business intelligence is that these early signals are locked in unstructured data and, until now, have been impossible to quantify statistically and objectively.

In this instance, DeepInsight was able to analyse unstructured content to extract and quantify the emerging global dialogue concerning the impact that the spread of ASF might have on global pork prices (as revealed through the topic “Pork Prices”). The strength of the dialogue rose ahead of and virtually mirrored that of the eventual rise in actual pork prices in China (see Figure 4). Deeper analysis into the monthly changes reveals that these early warning signals began to emerge around 21 months before actual market prices responded.

Figure 4: Rise in Actual Pork Prices Lags Behind the “Pork Prices” Global Dialogue

For an organisation such as Ital-Meat Fleisch, the influx of Chinese buyers into the European market proved a disaster. Specialising in purchasing, processing, and curing raw pork meat to produce high quality pork products, it is completely exposed to market commodity prices. As European pork prices rose on the back of increased Chinese buying activity, inflexible supply contracts left it unable to pass the rising costs onto its customers and, in January 2020, it entered voluntary administration. Armed with the early warning signals that DeepInsight would have given, things may have panned out differently for the company. Although the lead time may not have been sufficient to vertically integrate back down the supply chain to isolate itself from the price shocks, it would have been enough to bring other defensive options into play before the disruption hit, such as:

  • renegotiation of its inflexible contracts before rising market prices alerted its customers to their true value under high-market-price conditions
  • stockpiling of raw-pork commodities ahead of increased competition from Chinese buyers
  • locking in future prices by contracting ahead for future supplies
  • placing call-options with wholesale suppliers to create a price cap and guarantee supply during supply-shortage periods.

Similarly, organisations such as lenders and insurers with exposure to companies or portfolios in this sector would also have benefited from such early warning insights and been able to take a more informed view of the likely increase of risk in and across their portfolios.

Figure 5: Continual Monitoring by DeepInsight Indicates Increase in ASF Dialogue

Future Insight

COVID-19 shares many similarities with ASF and, since January 2020, has displaced it from the global news headlines. However, DeepInsight’s unbiased, unsupervised machine-learning algorithms mean that it does not place any weighting on content type, coverage, or prevalence. It constantly and consistently sees beyond the headlines and focusses on robust analysis of the underlying fundamentals. Even today, DeepInsight continues to reveal topics relating to ASF to be on an upward trajectory, enabling analysts to monitor unfolding events to determine any likely future heightening or easing of its impact ahead of traditional business-intelligence signals (see Figure 5).