Execution Quality: Machine Learning in Algorithmic Trading

This study examines the true extent to which machine learning (ML) algorithms play a part in improving execution quality in institutional trading. While these days, financial markets are oftentimes portrayed as being completely dominated or one of the most dominated fields for the use of artificial intelligence (AI), existing research suggests that the execution quality of trades are determined by existing market microstructure factors rather than accurate predictive signals generated by algorithms. To further investigate this, this paper combines a synthesis of academic literature and interviews with excerpts in trading systems and execution processes.

Findings from both secondary and primary research do indeed indicate that while ML models are widespread in use to generate predictive signals, they do not directly influence execution quality. Instead, execution quality is determined by existing structural market conditions that require human judgement to properly overcome. These results suggest that the role of ML in trading is more limited than previously assumed.

In modern culture, algorithmic trading and financial markets in general are often portrayed as environments that are increasingly being dominated by artificial intelligence (AI) and machine learning (ML) algorithms. Popular narratives are spun in a way that suggests that AI based algorithms are capable of independently identifying their own opportunities and executing trades by themselves, consistently being able to outperform their human counterparts in the quality of their trades. In reality, however, the true role of machine learning in institutional trading is much more limited. While predictive algorithms based on ML models are widely used in most forms of trading to analyze markets and generate predictive signals, they do not operate independently, and often need to be thoroughly vetted by humans for accuracy. Variable factors in the market such as order flow and market structure constraints create limits on how much a given model can influence the outcomes of an order, raising questions about how much AI and ML in trading truly improves execution quality.

The purpose of this paper is to evaluate to what extent machine-learning based models or algorithms influence the trading decisions and average execution quality of institutional firms.