Apr 21, 2011

Inferring beliefs from prediction market data

Rajiv Sethi put up a nice post on his blog, asking to what extent we can infer the nature of traders' beliefs about a particular event from the transactions that occur in a prediction market. (Related: my earlier post on prediction markets) In responding to a post by Nate Silver at fivethirtyeight, who used data from prediction markets to discuss the electoral outcomes, Rajiv says:
This interpretation of prices as probabilities is common and will be repeated frequently over the coming months. But what could the "perceived likelihood according to the market" possibly mean?

Markets don't have perceptions. Traders do, but there is considerable heterogeneity in trader beliefs at any point in time. Prediction market prices contain valuable information about this distribution of beliefs, but there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense. In fact, to make full use of market data to make inferences about the distribution of beliefs, one needs to look beyond the price at last trade and examine the entire order book...

All that the price at last trade can tell us about is the beliefs of the two parties to this transaction. If both are risk-averse or risk-neutral, they each must believe that entering their respective positions will yield a positive expected return. Hence the buyer must assign probability at least 62.5% to the event that the Democrat is elected, while the seller assigns a likelihood of at most 62.5% to this event.

This tells us nothing about the beliefs of traders who are not party to this transaction.

Rajiv goes on to post an example order book (for the US presidential election outcome for 2012), and he discusses some conclusions that could be drawn from this data. In a followup post, Rajiv notes:
In contrast, the order book, which is the collection of all unexpired bids and offers that cannot currently be matched against each other, contains a wealth of information about the distribution of trader beliefs. Under certain assumptions about the risk preferences of market participants, one can deduce a distribution of trader beliefs from this collection of standing orders. The imputed distribution may then be used to infer what the average trader (in a well-defined sense) perceives the likelihood of the underlying event to be. Furthermore, it can be used to gauge the extent of disagreement about this likelihood within the trading population.

I'd like to make a couple of comments about this line of reasoning. First, I disagree with Rajiv that price of the last trade is a poor indicator of the consensus probability. The statement that the last trade "tells us nothing about the beliefs of traders who are not party to this transaction" is simply not true: the exchange mechanism was designed was to match orders whose desired prices overlap, and up until the last trade we can assume that the mechanism did its job. So the two parties to this transaction were not arbitrary -- each party represents the "marginal" trader on their respective side of the bet. The transaction does not just represent the beliefs of these two individuals, it also tells us that there was a group of less-confident traders waiting behind whose trades were not matched. This is very important information.

(Update: After emailing a link to this post to Rajiv, he responded and wanted to correct the record:
I did not say that the price at last trade "is a poor indicator of the consensus probability," only that "there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense." It's an important difference. My follow up post shows that the price can be a pretty good approximation if the order book is reasonably symmetric.
Thanks, I stand corrected!)

Second, while the order book provides "promises" by other traders to take a side of the bet, these promises are very weak. For example, when a trader posts an order in the order book, this trader is exposed to the risk that news is released that moves the presumed likelihood of the outcome, and other traders with fast reactions can buy up these still-cheap orders. Hence, if we interpret posted orders as statements about beliefs, then these estimates will be biased according to this potential risk. Rajiv makes this point as well.

It should be noted also that traders can manipulate the order book maliciously. Since posting orders is free, a trader can lay down a large buy order and then, perhaps via a trading bot, pull the order after any change in the price. Depending on Intrade's mechanism, say if the company were to announce the clearing of every individual contract, this would allow traders to post orders in an entirely risk-free manner. (I don't know the precise details of the mechanism, but clearly some risk is required if the market can clear a large order all at once).

(During the health care debate, the market for contracts "Will Obamacare pass?" was likely manipulated, as you can see here. In this case, an individual trader made a huge bet against passage, moving the price more than 40 points. The market quickly recovered.)

This also gets at something that's bothered me for a long time: why is it that the "typical" mechanism for trading such betting contracts is to run an exchange with a posted order book? Despite the fact that this is how the stock market operates, I see no reason that this should be the gold standard. Recent research in prediction markets has led to the design of automated market makers which possess some very nice properties. In particular, they don't involve an order book, and instead require the market maker to simply post the current prices -- the bets placed by traders will move the prices according to an advertised "potential function." Under simple conditions, it can be shown that the market maker has a bounded risk to operate this market, and that this risk scales according to the amount of "liquidty" it provides. For such a market, the liquidity is essentially the speed with which the prices move according to the size of a given trade.

With an automated market maker of this type, much of Rajiv's concerns, i.e. those regarding how to translate between trades/orders and beliefs, are no longer relevant. The prices posted by the market maker contain all the relevant information about the status of the contracts.

(Interested individuals can have a look at my upcoming EC paper, with Jenn Wortman Vaughan and Yiling Chen, on designing automated market makers for combinatorial events spaces)

Apr 1, 2011

COLT call for open problems

I've been informed that the COLT PC this year is looking for more submissions for Open Problems which will appear at this year's conference. The deadline has been extended to May 11, more information is here (the extension is correct, despite the date the website currently states). Personally, I'm a fan of the open problems session -- I've coauthored 1,2,3 of them -- as it provides a venue for suggesting future research directions that does not require having a fully-baked result.

Please consider submitting something!
Call for open problems:

COLT 2011 will include an Open Problems session. A description of the problems posed will also appear on the conference web site (and in the electronic proceedings).

We encourage everyone to propose and present a clearly defined open research problem (or several closely related open problems), relevant to COLT. Authors should carefully prepare a brief writeup including a clear, self-contained description of the problem, their motivation for studying it, and the current state of this problem (including any known partial or conjectured solutions and relevant references). As last year, we encourage submissions of problems not conventionally in the scope of COLT, as long as there is a convincing reason to include it in the conference. You should be able to clearly express the problem in a 5-10 minute talk. A reward for solving an open problem is encouraged but not required.

If you would like to submit a problem, please email it to colt2011open@gmail.com by March 27, 2011 May 11, 2011. Submissions should be at most 2 pages long and be in the COLT format.