Apr 21, 2011

Inferring beliefs from prediction market data

Rajiv Sethi put up a nice post on his blog, asking to what extent we can infer the nature of traders' beliefs about a particular event from the transactions that occur in a prediction market. (Related: my earlier post on prediction markets) In responding to a post by Nate Silver at fivethirtyeight, who used data from prediction markets to discuss the electoral outcomes, Rajiv says:
This interpretation of prices as probabilities is common and will be repeated frequently over the coming months. But what could the "perceived likelihood according to the market" possibly mean?

Markets don't have perceptions. Traders do, but there is considerable heterogeneity in trader beliefs at any point in time. Prediction market prices contain valuable information about this distribution of beliefs, but there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense. In fact, to make full use of market data to make inferences about the distribution of beliefs, one needs to look beyond the price at last trade and examine the entire order book...

All that the price at last trade can tell us about is the beliefs of the two parties to this transaction. If both are risk-averse or risk-neutral, they each must believe that entering their respective positions will yield a positive expected return. Hence the buyer must assign probability at least 62.5% to the event that the Democrat is elected, while the seller assigns a likelihood of at most 62.5% to this event.

This tells us nothing about the beliefs of traders who are not party to this transaction.

Rajiv goes on to post an example order book (for the US presidential election outcome for 2012), and he discusses some conclusions that could be drawn from this data. In a followup post, Rajiv notes:
In contrast, the order book, which is the collection of all unexpired bids and offers that cannot currently be matched against each other, contains a wealth of information about the distribution of trader beliefs. Under certain assumptions about the risk preferences of market participants, one can deduce a distribution of trader beliefs from this collection of standing orders. The imputed distribution may then be used to infer what the average trader (in a well-defined sense) perceives the likelihood of the underlying event to be. Furthermore, it can be used to gauge the extent of disagreement about this likelihood within the trading population.

I'd like to make a couple of comments about this line of reasoning. First, I disagree with Rajiv that price of the last trade is a poor indicator of the consensus probability. The statement that the last trade "tells us nothing about the beliefs of traders who are not party to this transaction" is simply not true: the exchange mechanism was designed was to match orders whose desired prices overlap, and up until the last trade we can assume that the mechanism did its job. So the two parties to this transaction were not arbitrary -- each party represents the "marginal" trader on their respective side of the bet. The transaction does not just represent the beliefs of these two individuals, it also tells us that there was a group of less-confident traders waiting behind whose trades were not matched. This is very important information.

(Update: After emailing a link to this post to Rajiv, he responded and wanted to correct the record:
I did not say that the price at last trade "is a poor indicator of the consensus probability," only that "there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense." It's an important difference. My follow up post shows that the price can be a pretty good approximation if the order book is reasonably symmetric.
Thanks, I stand corrected!)

Second, while the order book provides "promises" by other traders to take a side of the bet, these promises are very weak. For example, when a trader posts an order in the order book, this trader is exposed to the risk that news is released that moves the presumed likelihood of the outcome, and other traders with fast reactions can buy up these still-cheap orders. Hence, if we interpret posted orders as statements about beliefs, then these estimates will be biased according to this potential risk. Rajiv makes this point as well.

It should be noted also that traders can manipulate the order book maliciously. Since posting orders is free, a trader can lay down a large buy order and then, perhaps via a trading bot, pull the order after any change in the price. Depending on Intrade's mechanism, say if the company were to announce the clearing of every individual contract, this would allow traders to post orders in an entirely risk-free manner. (I don't know the precise details of the mechanism, but clearly some risk is required if the market can clear a large order all at once).

(During the health care debate, the market for contracts "Will Obamacare pass?" was likely manipulated, as you can see here. In this case, an individual trader made a huge bet against passage, moving the price more than 40 points. The market quickly recovered.)

This also gets at something that's bothered me for a long time: why is it that the "typical" mechanism for trading such betting contracts is to run an exchange with a posted order book? Despite the fact that this is how the stock market operates, I see no reason that this should be the gold standard. Recent research in prediction markets has led to the design of automated market makers which possess some very nice properties. In particular, they don't involve an order book, and instead require the market maker to simply post the current prices -- the bets placed by traders will move the prices according to an advertised "potential function." Under simple conditions, it can be shown that the market maker has a bounded risk to operate this market, and that this risk scales according to the amount of "liquidty" it provides. For such a market, the liquidity is essentially the speed with which the prices move according to the size of a given trade.

With an automated market maker of this type, much of Rajiv's concerns, i.e. those regarding how to translate between trades/orders and beliefs, are no longer relevant. The prices posted by the market maker contain all the relevant information about the status of the contracts.

(Interested individuals can have a look at my upcoming EC paper, with Jenn Wortman Vaughan and Yiling Chen, on designing automated market makers for combinatorial events spaces)