This book is part of the AI for Everything Series
Written by Edward P.K. Tsang. Published by CRC Press in June 2023
The synergy between AI and finance will bring lots of opportunities
In algorithmic trading market speed is crucial for taking advantage of obvious opportunities. However, investing in faster computers and networks is costly and leads to intense competition. A more effective approach is to focus on researching and discovering opportunities before competitors. The competition lies in the speed of seeking opportunities rather than the speed of computing equipment. It is essential to have expertise in both finance and computing. Financial experts know where to look for opportunities, while computing experts can search quickly. By combining their expertise, there is a higher likelihood of finding exploitable patterns before others who lack the same level of collaboration.
The success of AlphaGo (Deepmind’s AI board game Go player) has generated high expectations for general AI’s ability to learn by itself with minimal human input. However, there are significant challenges to applying this approach to real-world problems. Combinatorial explosion makes it difficult for computers to solve many complex problems optimally, highlighting the importance of clever algorithms that find better solutions more efficiently. Machine learning requires useful variables, which can be expensive and challenging to create, especially in fields like healthcare and finance. Feedback in machine learning can also be noisy, leading to suboptimal results. Therefore, while AI has made impressive strides, caution is necessary when considering its capabilities and limits.
The collaboration between computer scientists and financial experts improves the applications of machine learning in finance greatly. It also highlights the limitations of classical economics due to its unrealistic assumptions of perfect rationality, homogeneity, and perfect information. These classical assumptions ignore the role of computation, rendering them impractical in a world experiencing combinatorial explosion. The author suggests that AI and classical economics have a complex, interdependent relationship, where neither can truly flourish without the other.
There are two forms of machine learning: “supervised learning” and “unsupervised learning”. In supervised learning, the program is trained on labeled data, and it learns to predict the output for new input data. This type of learning is used for tasks such as classification, regression, and object detection. On the other hand, unsupervised learning involves training the machine on unlabeled data to find patterns and relationships in the data. It is often used for clustering, dimensionality reduction, and anomaly detection.
Human experts are important in determining what to learn, choosing the right variables, and designing candidate solutions for machine learning to succeed. It also clarifies that there is no magic in machine learning and that expertise is essential before the General AI approach can succeed.
Models can be valuable in navigating complex situations by:
- Identifying Key Components: The process of constructing an LLM forces us to break down a scenario into its essential elements. For example in a war simulation focus can be on troop strength, terrain features, and weather conditions.
- Understanding Relationships: Models not only identify these components but also map their interactions and influences. We see how rivers, mountains, and weather dynamics can impact troop movements and battle outcomes.
- Focusing Attention: By pinpointing the key players and their interplay, models guide our focus in complex situations, helping us prioritize information and make informed decisions.
Model builders often start with the most basic components and relationships, leaving out less important components and their relations for a later stage. The initial models are naturally imperfect, but simpler models are easier to study. After studying the simple model, more components and relationships can be added incrementally to assess the impact of each additional component and relationship until a model close to reality is achieved. However, most situations worth studying are complex, hence a model is never a perfect description of reality. As George Box once said, “All models are wrong, but some are useful”
For instance, the Basic Alternating-Offer Bargaining model does not describe human bargaining realistically. Communication in human bargaining is a lot more complicated. For example, in a market, a buyer may walk away, hoping that the seller will call him/her back with a better offer. Eye contact and body language are also important in these situations.
Simple models in finance and economics allow for mathematical analysis of their properties, providing insights into underlying dynamics. However, many interesting models are too complex for mathematical analysis, particularly when certain parameters are unknown or uncertain. For instance, determining the subgame equilibrium in a bargaining model becomes challenging when one player lacks knowledge of the other player’s utility decreasing rate. Thus, analytical methods have limited applicability in complex models, requiring alternative approaches such as simulations or approximations to understand behavior and outcomes.
AI modelling, simulation, and machine learning are powerful tools that help one to focus on what to pay attention to and reason about a subject, such as a financial market or a payment system. Mathematical reasoning is elegant and powerful, but it is only useful for simple situations. To study a complex system, simulation is cost-effective and sometimes the only way. Modelling and simulation enable one to assess the risk of a portfolio. Adding machine learning to modelling and simulation allows one to find subgame equilibrium in complex game models. Modelling, simulation, and machine learning could combine to form a powerful tool. Modelling enables simulation, and machine learning helps to improve models.
Financial portfolio optimization is a core problem that aims to maximize return and minimize risk. Diversification is the basic principle to reduce risk. Practitioners commonly begin with the Markowitz model, though relaxing simplifying assumptions complicates the solution process. As more factors are considered, algorithm expertise demands increase significantly. Portfolio optimization poses a two-objective challenge, overlooked by many financial specialists despite extensive multi-objective optimization research among computer scientists. Unaddressed concerns include costly computing expertise and finite computation times. Understanding how models correspond to reality remains vital amidst inherently intricate markets.
DC offers a valuable alternative to Time Series for understanding financial markets. Its fresh perspective unlocks new research avenues and opportunities for those willing to explore its potential. The field is ripe for exploration with significant discoveries waiting to be made.
Financial market dynamics are traditionally recorded in Time Series. However, the best approach is to look at the market from a different angle: let events dictate when to record a transaction. This motivates the definition of Directional Change (DC), an event-based representation of time. DC provides an alternative way to Time Series in transaction data collection. It provides one with more information about the market, such as new measures in volatility. With the same raw transaction data collected differently, one sees the market from a different angle. This allows one to see things that one could not have seen before. Stylized facts observed under DC fuel new research too. As a new representation, DC research demands new reasoning methods. The new representation provides opportunities to those who know how to interpret and analyze DC series. DC research is in its infancy, and many low-hanging fruits are waiting to be picked
Computers can run every aspect of a market, including order clearing in the stock exchange, market making in foreign exchanges, algorithmic trading, and electronic contracts. In a hypothetical world where all programs start from formal programming specifications and are automatically generated from specifications, we can study the behavior of these programs rigorously, as in mathematics and logic. Markets can then be studied rigorously, just like how we use physics to study the natural world. Arguably, markets should be easier to study than the natural world because all computer systems are human-made, so we should know exactly how they work. Therefore, when all processes are rigorously specified and implemented correctly in an automated market, properties of the market could be studied as hard science. In this hypothetical world, experiments can be repeated, and control experiments can be conducted. However, there is a big gap between the current situation and the hypothetical world sketched above. Most computer program specifications are written in natural language, not formal specification languages. Natural language can sometimes be ambiguous, and program implementations are rarely bug-free. Programming bugs are generously tolerated; people rarely will go beyond moaning when they encounter operating system failures (“blue screens”), for example. Traders generally