Crystal Cube: Forecasting Disruptive Events

Figures & data

Figure 1. Architecture of the Crystal Cube event forecasting system.

Figure 2. Method developed for computing event similarity.

Figure 3. Crystal Cube’s inputs and outputs.

Figure 4. LSTM predictions for Ukraine for the test period: 1 Jan 2012–31 Dec 2020.

Figure 5. LSTM predictions for UK for the test period: 1 Jan 2012–31 Dec 2020.

Figure 6. LSTM predictions for Sudan for the test period: 1 Jan 2012–31 Dec 2020.

Figure 7. LSTM forecasts for July 2015.

Figure 8. LSTM forecasts for March 2019.

Table 1. Detailed description of events forecast for July 2015

Country and Event	News Example
Greece: After almost one-third of PM Alexis Tsipras’ Syriza members of Parliament defected from his bailout agreement with the European Union, Tsipras re-signed and a snap election was called in August.	Renee Maltezou and Michele Kambas, “Tsipras Resigns, Paving Way for snap Greek Elections,” Reuters, August 20, 2015.
Macedonia: In January 1996, PM Nikola Gruevski resigned after the Przino Agreement was reached with mediation of European Union. The agreement ended a wiretapping scandal in which the opposition claimed the government had illegally wiretapped 20,000 people.	“Macedonian PM to Resign in Bid to End Crisis,” Radio Free Europe Radio Liberty, January 14, 2016.
Australia: In September 2015, despite claiming that his leadership was not under challenge, PM Tony Abbott was removed as the leader of his party in a party vote, and subsequently resigned as prime minister.	“Australian PM Tony Abbott Ousted by Malcolm Turnbull,” BBC News, Sep 14, 2015.
Burundi: In July 2015, General Leonard Ngendakumana warned that he could lead a rebellion due to President Pierre Nkurunziza’s insistence on running for a third time despite condemnation. Two days of violent clashes began between rebels and the army.	Fumbuka Ng’wanakilala, “Burundi Coup General Says Force Only Way to Oust President,” Reuters, July 6, 2015.
Nepal: In July 2015, protestors stormed a stadium where the deputy PM was giving a speech and discussing a draft for the new constitution. Protestors also clashed with police.	Renee Maltezou and Michele Kambas, “Tsipras Resigns, Paving Way for snap Greek Elections,” Reuters, Aug 20, 2015.

Table 2. Detailed description of events forecast for March 2019

Country and Event	News Example
Sudan: Omar al-Bashir was removed from power by a military coup d’état on 11 April 2019. After leading the coup, Lieutenant General Ahmed Awad Ibn Auf stepped down one day later on 12 April 2019 due to mass demonstrations.	“Sudan Defense Minister Steps Down as Head of Transitional Military Council,” Al Arabiya, Apr 12, 2019.
United Kingdom: Amid Conservative Party disagreements on Brexit and the failure to pass a Brexit deal, PM Theresa May announced her resignation on 24 May 2019 and left office on 24 July 2019.	Eliza Mackintosh, “Brexit Failure Forces British Prime Minister Theresa May to Announce Resignation,” CNN, May 24, 2019.
France: Yellow vests movement protests on 16 March 2019 led to rioting and looting. President Emmanuel Macron’s government called in soldiers for security and protestors were banned in areas of Paris.	Kim Willsher, “France Drafts in Troops to Prevent Further Gilets Jaunes Violence,” The Guardian, Mar 22, 2019.
Venezuela: Venezuela has been suffering from a severe political, socioeconomic, and humanitarian crisis under President Nicolas Maduro. In March 2019, the U.S. announced the withdrawal of diplomatic staff from its embassy due to the crisis.	Fabiola Sanchez and Scott Smith, “US Announces Withdrawal of Last of Its Embassy Personnel from Venezuela,” USA Today, Mar 12, 2019.
Israel: Israel’s Knesset was dissolved in December 2018 with elections scheduled for April 2019. In March 2019 the Supreme Court banned a far-right, anti-Arab leader from the elections. Tel Aviv was also hit by rocket fire from Gaza, sparking fears of conflict before the election.	“Israel’s Supreme Court Bans Jewish Extremist from Election,” Associated Press, Mar 17, 2019.

Figure 9. Random forest and LSTM ROCs for forecasting 1 month and up to12 months ahead.

Figure 10. Random forest and LSTM Precision-Recall curves for forecasting 1 month and up to 12 months ahead.

Figure 11. Random baseline and LSTM ROCs for forecasting 1 month and up to 12 months ahead.

Figure 12. Random baseline and LSTM Precision-Recall curves for forecasting 1 month and up to 12 months ahead.

Table 3. Comparison of CC forecasts with those from Beger, Dorff, and Ward (2016) and Ward and Beger (2017). Avg PR was not computed in Beger, Dorff, and Ward (2016), – accuracy was computed instead

Model	Test AUC	Test Avg PR
Beger, Dorff, and Ward (2016), – 1 month forecast	0.839	Accuracy 0.992
CC Model 1 - – 1 month forecast	0.876	0.049 (Accuracy 0.996)
Ward and Beger (2017) – 6 months forecast	0.823	0.059
CC Model 2–6 months forecast	0.836	0.092

Figure 13. ROC for 1 month forecast – test data from .Beger, Dorff, and Ward (2016)

Figure 14. ROC for 6 month forecast by Crystal Cube – test data from Ward and Beger (2017).

Beger, A., C. L. Dorff, and D. Ward. 2016. Irregular leadership changes in 2014: Forecasts using ensemble, split-population duration models. International Journal of Forecasting 32 (1):258–281. doi:10.1016/j.ijforecast.2015.01.009.

 Web of Science ®Google Scholar