Our latest deep learning-based classifier, given historical price data, recommends one of three FOREX trade actions (buy, short, or do nothing). We trained it on price data from roughly 2010 through most of 2016, validated it from late 2016 to mid 2018, and tested it from late 2018 to the present. (Using sequential training, validation, and testing sets in this manner is a timeseries forecasting best practice).
The ROC curves below show the classifier performance on the test set for three currency pairs. The first, USD/JPY, proved an extremely strong classifier given how noisy FOREX signals are. The second ROC curve, for NZD/USD, demonstrates our average performance for most currency pairs.
However, consider the third ROC curve below, for GBP/USD: The classifier performs abysmally on the test set. We are not surprised by this, considering the chaos Brexit has inflicted on the British Pound over the last two years. It is clear that Brexit drives GBP/USD, rather than machine learnable (in a technical analysis sense) patterns that underpin the price movements of the other currency pairs we’ve considered.
In this way, Brexit serves as a “negative control”, from which we can derive confidence in the models trained for other currency pairs (since we expect GBP/USD to be messier in the test set than the other currency pairs).