High frequency finance andmarket microstructure

Fabrizio LilloFabrizio LilloScuola Normale Superiore di Pisa and University of Palermo (Italy)

Santa Fe Institute (USA)

MiFIT, Perm, November 2012

Observatory ofObservatory ofComplex Complex SystemsSystems

Outline of the course (tentative)

• 1. Introduction to empirical finance and EMH• 2. Trading mechanisms, Roll model• 3. Sequential models: Glosten-Milgrom• 4. Strategic models: Kyle• 5. Market impact • 6. Order flow• 7. Optimal execution

Introduction to Empirical Finance and Efficient Market Hypothesis

Fabrizio LilloLecture 1

Financial markets• Financial markets allow two classes of agents to

meet:– Entrepreneurs: who have industrial projects but need

funding– Investors: who have money to invest and are ready to

share profits and risks of the projects• Therefore financial markets are systems where a

large number of investors interact through trading to determine the best price for a given asset.

• From this point of view financial markets can be seen as a collective evaluation system.

5

Financial markets as complex systems

A financial market can be described as a `model’ complex system.

In a financial market there are many agents interacting to perform the collective task of finding the best price for a financial asset.

6

In a financial market there are many agents interacting to perform the collective task of finding the best price for a financial asset.

There are many different types of financial markets

- Stock exchanges (New York, London, Tokyo) - Foreign exchange markets (Global market) - Derivative markets (Chicago, New York, Paris) - Bond markets (London) - Commodities - …..

Financial market as a model complex systemFinancial market as a model complex system

• The study of financial markets has an obvious importance on his own. • However I believe that financial markets are an ideal model system to study the interaction of many individuals taking decisions under risk. The system is ideal because

• It is an extremely competitive environment where the fitness of an investor can often be identified with her ability of generating profit• The interaction mechanism is clearly defined• The availability of very detailed and large datasets (down to individual behavior) allows to perform careful empirical analyses • In some cases the flow of external information can be identified and monitored (news stream, financial analyst's forecasts, etc)

A big simplification• The structure of a financial market is quite

complicated and the process of trading is very structured (and have important consequences on the statistical properties of prices)

• For the moment we will abstract from the detailed process of trading (this is a subject for market microstructure) and we will consider the dynamics of asset price and its modeling – On a relatively long time scale– Without reference to the trading process

Price• Price of an asset is one of the most important financial

variables• The (instantaneous) definition of price is not

straightforward

Coca Cola Co.

Order book data• A higher resolution of financial data contains data

on all the orders placed or canceled in the market• Many stock exchanges (NYSE, LSE, Paris) works

through a double auction mechanism • Order book data arefundamental to investigate the price formation mechanisms

Representation of limit order book dynamicsRepresentation of limit order book dynamics

What is the price of AstraZeneca at time t=2000?

Market microstructure• Market microstructure “is devoted to theoretical,

empirical, and experimental research on the economics of securities markets, including the role of information in the price discovery process, the definition, measurement, control, and determinants of liquidity and transactions costs, and their implications for the efficiency, welfare, and regulation of alternative trading mechanisms and market structures” (NBER Working Group)

Garman (1976)

• “We depart from the usual approaches of the theory of exchange by (1) making the assumption of asynchronous, temporally discrete market activities on the part of market agents and (2) adopting a viewpoint which treats the temporal microstructure, i.e. moment-to-moment aggregate exchange behavior, as an important descriptive aspect of such markets.”

• In this first lecture we neglect the complications due to the trading mechanism and we state the basic principle and approach to quantitative finance

• We then consider whether empirically data follows the prediction of this approach

15

Quantitative approach to financial markets

Roughly speaking two types of approaches are possible in the study of financial markets, and, more generally, of social systems.• Assume that agents in the system have a given amount (homogeneous or heterogeneous) of rationality. The process of price formation is based on the decision making of agents.• Make use of a more pragmatic approach consisting in analyzing the dynamical properties of financial variables looking for statistical regularities

16

Perfect rationality• The standard approach in financial economics consists in

assuming that agents in the market have perfect rationality and perfect knowledge of other agents’ preferences

• In this idealized market it is possible to investigate the conditions allowing an equilibrium between supply and demand

• Moreover the model is (sometimes) able to make falsifiable prediction on the behavior of aggregate quantities, such as price (e.g. Capital Asset Pricing Model)

17

Bounded rationality

• In recent years there has been an increasing interest of scientific community on models of bounded rationality (H. Simon), i.e. models where agents have only limited cognitive and computational abilities.

• An extreme approach consists in assuming that agents have zero intelligence, i.e. they act randomly. Surprisingly some empirical facts can be explained by this kind of model, proving the importance of interaction rules

Zero-intelligence model

BID

SELL LIMIT ORDERS

AS

K

BUY LIMIT ORDERS

SELL MARKETORDERS

BUY MARKET ORDERS

),( tpΩ

p0

),( tpn

18

€

5 / 2δ1/ 2σ −1/ 2

α 2Volatility =

Farmer et al., PNAS 2005

Strategy or structure?

19

Arbitrage opportunity

The main paradigm used for the modeling of a financial market is the absence of arbitrage opportunity.

An arbitrage opportunity is present in a market when an economic actor can devise a trading strategy which is able to provide her a financial gain continuously and without risk.

20

Example of arbitrage opportunity:

In an efficient market, the exploiting of an arbitrage opportunity implies its disappearance in a (usually) short time period.

St.Louis Miami

At a given time 1 kg of wheat costs 1.30 USD in St. Louis and 1.45 USD in Miami. The cost of transporting and storing 1 kg of wheat from St. Louis to Miami is 0.05 USD

By buying 10,000 kg in St. Louis and selling them immediately after in Miami it is possible to make a risk-free profit

10000 (1.45-1.30-0.05)=1000 USD

If this action is repeated this implies that the price in St. Louis increases and in Miami decreases.

Fundamental analysis

• Williams (1938), Graham and Dodd (1934): “intrinsic” or “fundamental” value of any security equals the discounted cash flow which that security gives title to, and actual price fluctuate around fundamental values

• Cowles (1933) demonstrated that the recommendations of major brokerage houses (based on fundamental analysis) did not outperform the market

Random walk model

• Working (1934) argued that random walks develop patterns that look like stock prices

• Kendall (1953) and Granger and Morgenstern (1963) performed statistical analysis showing that stock prices follow a random walk

• “If stock prices were patternless, was there any point to fundamental analysis” (LeRoy 1989)

23

Efficient Market Hypothesis

The origin of the Efficient Market Hypothesis can be traced back to Bachelier (1900) and Cowles (1933)

The modern literature begins with Samuelson “Proof that Properly Anticipated Prices Fluctuate Randomly” (1965) In word he stated that in an informationally efficient market, price changes must be unforecastable if they fully incorporate the expectations and information of all market participants

Also Mandelbrot (1966) arrived to similar conclusion

• “A capital market is said to be efficient if it fully and correctly reflects all relevant information in determining security prices. Formally, the market is said to be efficient with respect to some information set if prices would be unaffected by revealing that information to all participants. Moreover, efficiency with respect to an information set implies that it is impossible to make economic profits by trading on the basis of that information set”

(Malkiel, 1992)

The importance of information set

• Roberts (1967) introduced the following taxonomy:– Weak-form efficiency: The information set

includes only the history of prices– Semistrong-form efficiency: The information set

includes all the publicly available information – Strong-form efficiency: The information set

includes all information, i.e. also private information known to any market participant

26

Efficient Market Hypothesis

• Therefore, under the efficient market hypothesis, price changes must be unforecastable

• The more efficient the market, the more random is the sequence of price changes

• A widespread model of price that incorporates the efficiency of the market is the Random Walk Hypothesis

• Note that the random walk hypothesis is more restrictive than the martingale hypothesis

27

Random Walk Hypothesis

The attempts to model the price of a financial asset as a stochastic process go back to the 1900 pioneering work of Louis BachelierThe simplest model for price dynamics in discrete time is

where , is a constant, and is a noise term consistent with the Efficient Market Hypothesis

• The Efficient Market Hypothesis suggests that a good framework to describe price dynamics is continuous or discrete time stochastic processes• The price must be described by a martingale

i.e. the best forecast of tomorrow’s price is simply today’s price

28

Random walk hypothesis

Depending on the properties of , we distinguish• Independent Identically Distributed increments: for example

Gaussian distributed (stable laws)• Independent increments• Uncorrelated increments: the weaker form implies the

vanishing of the linear autocorrelation

29

This equation is used as one of the fundamental assumptions of the so-called Black and Scholes (B&S) model. The B&S model allows to obtain the rational price for a simple financial contract (an European option) issued on an underlying fluctuating financial asset.

dWtPdttPtdP )()()( σ +=

In continuous time the geometric Brownian motion is considered the simplest random process describing the price dynamics of a financial asset.

30

Random walk hypothesis

31

Ideal vs real

What do real data say?

An idealized model of stock market where the stock price dynamics is described by a geometric Brownian motion exists and provides the theoretical foundation for quantitative finance.

Financial data

• Data are essential for the development and testing of scientific theories

• In the last years social sciences have experienced a transition from a low rate of data production to an high rate of data production

• This is due to the availability of datasets combining an high resolution and a large size

• New levels of resolution raise new issues in data handling, visualization, and analysis

Financial data

• In the last thirty years the degree of resolution of financial data has increased– Daily data– Tick by tick data– Order book data– “Agent” resolved data

Daily data• Daily financial data are available at least since nineteenth

century• Usually these data contains opening, closing, high, and low

price in the day together with the daily volume• Standard time series methods to investigate these data

(from Gopikrishnan et al 1999)

Tick by tick data

• Financial high frequency data usually refer to data sampled at a time horizon smaller than the trading day

• The usage of such data in finance dates back to the eighties of the last century – Berkeley Option Data (CBOE)– TORQ database (NYSE)– HFDF93 by Olsen and Associates (FX)– CFTC (Futures)

• Higher resolution means new problems • data size: example of a year of a LSE stock

– 12kB (daily data)– 15MB (tick by tick data)– 100MB (order book data)

• irregular temporal spacing of events• the discreteness of the financial variables under

investigation• problems related to proper definition of financial variables• intraday patterns• strong temporal correlations• specificity of the market structure and trading rules.

Data size Irregular temporal spacing

PeriodicitiesTemporal correlations

Lillo and Miccichè, Encyclopedia of Quantitative Finance, 2010

• More structured data require more sophisticated statistical tools

• data size: • more computational power• better filtering procedures

• irregular temporal spacing of events• point processes, ACD model, CTRW model…

• the discreteness of the financial variables under investigation• discrete variable processes

• problems related to proper definition of financial variables• intraday patterns• strong temporal correlations

• market microstructure• specificity of the market structure and trading rules.

• better understanding of the trading process

Order book data• The next resolution of financial data contains

data on all the orders placed or canceled in the market

• Many stock exchanges (NYSE, LSE, Paris) works through a double auction mechanism

• Order book data arefundamental to investigate the price formation mechanisms

Representation of limit order book dynamicsRepresentation of limit order book dynamics

(Ponzi, Lillo, Mantegna 2007)

Eisler, Kertesz, Lillo, 2007

More structured data require

more sophisticated visualization tools

Agent resolved data

• In the recent years there has been an increasing availability and interest toward databases allowing to distinguish, at least partly, the trading activity of “agents” or “classes of agents”.

• In principle, this type of databases allows to investigate empirically the agent’s behavior and strategies, and to study the interaction between agents.

Structure of a financial market

Stock marketStock market

Market memberMarket

memberMarket

memberMarket

member

Market memberMarket

member

Market memberMarket

member

InstitutionInstitution

InstitutionInstitutionInstitutionInstitution

InstitutionInstitution

InstitutionInstitution

IndividualIndividual

IndividualIndividual

IndividualIndividual

IndividualIndividual IndividualIndividual

IndividualIndividual

Market memberMarket

member

Market memberMarket

member

44

Example: Momentum and contrarian strategies

Momentum investors are buying stocks that were past winners.

A contrarian strategy consists of buying stocks that have been losers (or selling short stocks that have been winners).

The contrarian strategy is formulated on the assumption that the stock market overreacts and a contrarian investor can exploit the inefficiency related to market overreaction by reverting stock prices to fundamental values.

• Is it possible to detect empirically such strategies?• Are there classes of agents using preferentially these strategies?

45

Grinblatt, Titman and Wermers (1995)

¶M.Grinblatt et al, American Economic Review 85, 1088-1105 (1995)

They investigated the trading pattern of fund managers by examining the quarterly holdings of 155 mutual funds (information from CDA Investment Technologies and CRSP data) over the 1975-1984 period.

• The large majority of funds (77%) had a momentum investment profile.• Authors found relatively weak evidence that funds tended to buy and sell the same stocks at the same time (herding).

Grinblatt and Keloharju (2000)

Grinblatt and Keloharju¶ investigated the central register of shareholdings for Finnish Central Securities Depository, a comprehensive data source. This data set reports individual and institutional holdings and stock trades on a daily basis. Data consists of each owner’s stock exchange trades from Dec 27, 1994 through Dec 30, 1996.

¶M.Grinblatt and M.Keloharju, J. of Financial Economics 55, 43-67 (2000)

• Foreign investors tend to be momentum investors• Individual investors tend to be contrarian• Domestic institutional investor tend to present a mixed behavior.

46

47

Example: profitability of classes of agents

Studies performed by Barber, Lee, Liu and Odean¶ have the performance of individual and institutional investors at the Taiwan Stock Exchange. Data allow authors to identify trades made by individuals and by institutions, which fall into one of four categories (corporations, dealers, foreigners, or mutual funds).

• Individual investor trading results in systematic and, more importantly, economically large losses• In contrast, institutions enjoy an annual performance boost of 1.5 percentage points

¶ B.M.Barber, Y.-T.Lee, Y.-J.Liu and T.Odean, Do Individual Day Traders Make Money? Evidence from Taiwan (2004).¶ B.M.Barber, Y.-T.Lee, Y.-J.Liu and T.Odean, Just How Much Do Individual Investors Lose by Trading? (2005).

What do the data say?

Statistical regularities or stylized facts

Phenomenology of asset return distributions

49

Random walk hypothesis

The distribution of returns is clearly non Gaussian

From Campbell, Lo, MacKinlay

€

κ =x 4

x 2 − x2

( )2 − 3

€

ς =x 3

x 2 − x2

( )3 / 2Skewness

Excess Kurtosis

Both are zero for a Gaussiandistribution

53

Mandelbrot Levy stable modelIn 1963 Mandelbrot modeled ln P(t) for cotton prices asa stochastic process with Lévy stable non-Gaussian increments. His finding was supported by the investigations of Fama in 1965, which were performed by analyzing stock prices in the New York Stock Exchange.

The typical value of the index of the Lévy stable pdf was found close to 1.6

54

Empirical properties of return pdfReturn (log price differences) pdf are leptokurtic

Example: Chevron Co. daily data 1989-1995

55

Different time horizonsLeptokurtosis increases at short time horizons

Example: Xerox Co. 10 minutes data 1994-1995

Leptokurtosis

Return pdfs areleptokurtic.

There is today alarge evidence that the second momentof the unconditionalpdf is FINITE!!

from Mantegna & Stanley, Nature 376, 46-49 (1995)

One-minute index returns

Gaussian

Lévy =1.4

57

Tail behaviorPioneering work in this field has been done by Casper G.de Vries (1990) for the foreign exchange market. In 1996 T. Lux at that time at Bamberg Universityobserved similar power-law behavior for stock return tails of the German Stock Exchange.

T. Lux observed that the stocks composing the DAXindex were characterized by positive and negative tailsof the cumulative return distribution of exponent

3

Slow convergence to a

Gaussian distribution

According to Central Limit Theorem, one should expect a convergence to a Gaussian distribution when the time horizon used to calculate returns is increased (and under some important assumptions!)

62

Probability of return to the originThe crossover between the two regimes can be investigatedby monitoring the probability of return to the origin P(Sn=0)

100

0.1

5.1

===

l

γ

α

For Lévy processes

( ) ( )( ) γπ

/1

/10

nSP n

Γ==

simulation parameters

Truncated Levy flight

63

The degree of leptokurtosis varies at different time horizons.There is a progressive convergence towards the geometricBrownian motion at longer time horizons.

From Mantegna &Stanley, Nature 383,587-588 (1996)

64

Tails could also be described by different stochastic models characterized by different tails.One example is a Truncated Lévy Flight†

with an exponential tail

Bouchaud & Potters, Theory of financial risk, CUP

† Mantegna & Stanley,PRL 73, 2946-2949 (1994)

Testing for efficiency

• Sequence and reversals (Cowles and Jones 1937, Cowles 1960): #(00,11) vs #(01,10)

• Runs (Mood 1940): #(111). Fama (1965) concludes “there is no evidence of important dependence”

• Filter rules (Alexander 1961): buy when price increases of x% and sell when price drops by x%. Fama (1965):no profit after transaction costs

• Technical analysis (???)• Test for serial (linear) correlations

Linear autocorrelation• One of the simplest method to test for efficiency is to

compute the linear autocorrelation of the return time series rt

• Under the assumption that returns are IID with finite sixth moment, the sample autocorrelation function of a time series of length T is asymptotically distributed as

• This (and similar) result allows to perform many different statistical tests

€

ρ(τ ) = ACF(τ ) =rtrt +τ − rt

2

rt2 − rt

2

€

T ˆ ρ τ( ) ≈ N(0,1)

Linear efficiency

Example: S&P 500sampled at 1 mintime horizon1983 - 2004

Characteristic decay time:378s (1983-1988)144s (1988-1993)96s (1994-1999)36s (1999-2004)

• Returns are essentially linearly uncorrelated• This tells us that one cannot forecast returns

with linear regression• But lack of linear of correlation does not imply

independence !! (the opposite is trivially true)• Are functions of returns (e.g. powers) also

linearly uncorrelated?

Even function of returns (e.g. absolute value, square) are a measure of volatility, i.e. the

diffusion rate of prices

72

Higher-order correlationsAre higher order correlation present?

Example: General Electric Co. 1995-1998

The volatility, i.e. the standard deviationof returns, is itself a stochastic process.

73

Volatility autocorrelationThe volatility autocorrelation is a slow-decaying function

The decay is compatiblewith a power-law decay

( )( ) ηττσ −∝ACF

S&P 500 sampled at 1 min time horizon1984 - 1996

3.0≈η

Long-memory processesLong-memory processes• There is strong evidence that volatility is a long

memory process, i.e. a process with an autocorrelation function decaying asymptotically as

• Long memory processes lack a typical time scale and are related to anomalous diffusion

• A long memory process depends on its entire past history and has often strong predictability on long time horizons.

Volatility distribution

Inverse gamma

Volatility distribution: different proxies

Bouchaud and Potters 2003

Scaling of kurtosis

Bouchaud and Potters 2003

Leverage effect

Bouchaud et al 2001

€

L τ( ) =rt +τ

2 rt

rt2 2

Skewness

Cont 2001

Multifractality of returns

Muzy et al. 2000

€

M q, l( ) = Pt +l − Pt

q≈ Cq lζ q

For a monofractal (as Brownian motion)

€

ζq = qH

81

Absence of time reversal in a financial market

• Leverage effect• Volatility structure• Omori law

source: Zumbach

83

(i)The Omori law in geophysicsThe Omori lawOmori law is governing the dynamics of the number of aftershocks occurring after a major earthquake.

Cumulative number of aftershocks in the earthquake occurring in eastern Pyrenees on February 18, 1996 (from Moreno et al., J. of Geophys. Res., 106 B4, 6609-6619 (2001))

∫=∝ −t

p dssntNttn0

)()(;)(

84

Relaxation after a market crashThe stochastic dynamics of the price of an asset traded in a financial markets is altered after a major financial crash.

Examples of dynamical changes have been detected in:

- the dynamics of implied volatility(1);- the dynamics of the variety of a portfolio(2);- the leverage effect(3).

(1) Sornette, Johansen and Bouchaud, J. Phys. I 6, 167 (1996).(2) Lillo and Mantegna, Eur. Phys. J. B20, 503 (2001).(3) Bouchaud, Matacz and Potters, Phys. Rev. Lett. 87, 228701 (2001).

85

The time series of index returns after a major financial crash shows a non-stationary time pattern

S&P 500 Index after the Black Mondayfinancial crash (19 Oct 1987).

one-minute return

86

We fit empirical data with the Omori functional form

N(t)=K[(t+τ)1-p- τ1-p]/(1-p)

Specifically, we measure† the cumulative number of S&P500 index returns exceeding a given threshold

†Lillo and Mantegna, PRE 68,

016119 (2003)

List of stylized facts (incomplete, Cont 2001)

• Absence of autocorrelations• Heavy tails• Gain/loss asymmetry• Aggregational Gaussianity• Intermittency• Volatility clustering• Conditional heavy tails• Slow decay of autocorrelation in absolute returns• Leverage effect• Volume/volatility correlation• Asymmetry in time scales

• Several other stylized facts (statistical regularities) has been discovered

• Up to now there is no econometric model which is able to take into account all these empirical facts

• Also the origin of these facts is controversial– The focus is on the economic interpretation of the

facts– The focus is on the “mechanical” interpretation

(i.e. facts reproduced by random models)

Can we understand the microscopic origin of some of

these stylized facts?

90

What is the origin of fat tails and clustered volatility?

• A common belief is that large part of the story is explained by the inhomogeneous rate of trading• Number of transactions and traded volume in a given time interval are known to be highly fluctuating quantity• The basic idea of many theories is that price return conditioned to a given trading intensity is thin tailed and the corresponding volatility is short memory

91

Fluctuations in trading rate as the causing factor

• Theories making use of this point of view are for example:–Subordinated processes (Mandelbrot and Taylor 1967, Clark 1973, Ane and Geman 2000)–Volume fluctuations (Gabaix et al. 2003)

92

Subordinated stochastic processes• Clark’s idea (1973) is that price shift due to individual transactions are Gaussian (or thin tailed), but when many trades are aggregated in a time interval, the return distribution can be fat tailed. This is due to the fluctuation of number of trades or volume in the time interval

• More precisely the price process P(t) is given by

where X is a Brownian motion and τ(t) is a stochastic time clock whose increments are IID and uncorrelated with X.• Clark hypothesized that the time clock τ(t) is the cumulative trading volume in time t

93

Investigated data

• We investigated two markets, the New York Stock Exchange (NYSE) and the London Stock Exchange (LSE).

• 20 LSE stocks in the period 5/2000 - 12/2002• 20 NYSE stocks in the period Jan 1,1995 - Jun 23,

1997, when the tick size was 1/8 $• 20 NYSE stocks in the period Jan 29,2001 - Dec 31,

2003, when the tick size was 1 pence = 1/100 $

94

We define transaction timetransaction time as

where ti is the time when transaction i occurs.

Similarly we define volume timevolume time as

where Vi is the volume of transaction i.

Alternative time clocks

Volatility under alternative time clocksWe construct price return time series by using transaction time rather than real time

Transaction time volatility for the stock AZN, which is defined as the absolute value of the price change over an interval of 24 transactions. The resulting intervals are on average about fifteen minutes long.

Volatility clustering and fat tails

96

““Shuffled times”Shuffled times”• We associate to each trade the corresponding price change. • Then we randomly shuffle transactions. We do this so that we match the number of transactions (or the transaction volume) in each real time interval, while preserving the unconditional distribution of returns but destroying any temporal correlations.

• Thus the samples are taken at uniform intervals of real time and the number of transactions matches that in real time.

Which time reproduces better the real time volatility?Transaction time or shuffled transaction time ?

According to Clark, Ane and Geman, etc, the stylized facts of price should be better reproduced by shuffled transaction time, since in this case the fluctuations in trading activity are preserved.

97

AZN (LSE) 8-9/2001

real time

transaction time

shuffled transaction

time

98

Correlations between volatilitiesCorrelations between volatilities

shuffled

non shuffled

solid=transactionopen=volume

black = LSEred = NYSE1blue = NYSE2

AZN PG-8

PG-100

black = real timered = transaction timeblue = volume time

100

Shuffling experiments: return distribution

101

Return distribution for fixed number of transactions

102

Clustered volatilityClustered volatilityNYSE1 LSELSE NYSE2

• solid circles = trans. time

• solid diamond= volume time

• open circles= shuffled trans.time

• open diamond= shuffled volume time

€

Corr σ (t)σ (t + τ )[ ] ≈ τ 2H −2

0<H<1 Hurst exponent

103

• Fat tails of return distribution and clustered volatility are closer to the real one in transaction (volume) time rather than in shuffled transaction (volume) time.

• These results indicate that the main drivers of heavy tails are the fluctuations of the price reaction to individual transactions.

• Tick size is also important as emphasized by the comparison of NYSE1 and NYSE2 dataset.

• Our analysis suggests that fluctuations in trading rate are not the most important determinant of return’s fat tails and clustered volatility