August 6, 2003
The recently blasted "terrorism futures" market isn't as bad an idea as it's been made out. DARPA's highly ideological embodiment was modeled on the commodities market, a questionable implementation that turned a means of measuring shared knowledge into an instrument of psyschological warfare. But the underlying idea consensus knowledge mining has been around for quite a while. And it's not a bad idea. As a matter of fact, I'd like to propose an entire society based on the principle.
Getting there requires a somewhat circuitous route. The first stop is an alternate future where these techniques are used and abused on a regular basis, pretty much along the DARPA line of thought. Then a short random walk down Wall Street to probe the concepts of information sharing and market efficiency. Briefly returning to the fantasy future for a glimpse of a democratic systhesis of public health and domestic intelligence as a humane alternative to DARPA's earlier attempt to make being spied on a way of life in the land of the free and the home of the brave. A quick lap around the racetrack examines market efficiency and consensus knowledge in parimutuel gambling (with a minor tip on sure-thing bets on the ponies). Agent Easy gave me a bit of a hard time for offering criticism on this site without making a counter proposal, so I'm going to pitch a market model that will not only beat the socks off of the DARPA flop but could help resolve some of the problems in our conflict-ridden world.
A quarter of a century ago, John Brunner wrote a landmark science fiction novel; one of the precursors of cyberpunk. Among other things, it introduced computer "worms" and "viruses" into the noosphere. And one of the central ideas of his vision of the future included public future prediction models. Here's the relevant passage:
HOW TO GROW DELPHINIUMS
It works, approximately, like this.
First you corner a large- if possible, a very large - number of people who, while they've never formally studied the subject you're going to ask them about and hence are unlikely to recall the correct answer, are nonetheless plugged into the culture to which the question relates.
Then you ask them, as it might be, to estimate how many people died in the great influenza epidemic which followed World War I, or how many loaves were condemned by EEC food inspectors as unfit for human consumption during June 1970.
Curiously, when you consolidate their replies they tend to cluster around the actual figure as recorded in almanacs' yearbooks and statistical returns.
It's rather as though this paradox has proved true: that while nobody knows what's going on around here, everybody knows what's going on around here.
Well, if it works for the past, why canít it work for the future? Three hundred million people with access to the integrated data-net is a nice big number of potential consultees.
Unfortunately most of them are running scared from the awful specter of tomorrow. How best to corner people who just do not want to know?
Greed works for some, and for others hope. And most of the remainder will never have any impact on the world to speak of.
Good enough, as they say, for folk music . . .
The Shockwave Rider
The forecasting mechanism Brunner is talking about is the "Delphi Poll." It's a marvelous discovery of the distributed knowledge of the noosphere. As he says, even though nobody knows what's going on, everybody knows what's going on. It's true and it works.
The Delphi polling method was developed at RAND back in the late 1960s. In a nutshell, a Delphi poll is iterative feedback polling with confidence estimates.
Like most things in a nutshell, a little explanation is necessary. Iterative feedback means the poll results are presented to the sample group and the questions asked again. The process is repeated until the results start repeating themselves. As long as the answers keep changing, you continue the process. The confidence estimates mean that each question is really two: what do you think the answer is and how confident are you that your answer is correct. The confidence estimates are included in the results, so that everybody shares the information about the certainty or uncertainty of the answers. In some more elaborate versions, participants can include short comments on the questions, explain the reasons for their answers, discuss their reservations and share remarks about how the process is going.
The questions are usually phrased so they can be answered on a continuous scale, rather than the yes, no, or choose one of several divergent answers typical of polling techniques used to shape -- rather than measure -- public opinion. The goal of the Delphi process is to share knowledge and the participants are the first recipients of the shared knowledge. Again, this is quite different from opinion polling, which was strongly shaped by the wartime need to measure the effects of propaganda campaigns and psychological warfare.
As the Delphi poll proceeds, the answers typically gravitate towards one of two general outcomes. A shared consensus answer will show the results moving towards a single peak response, like the loaves of bread question in Brunner's example. A divided consensus produces usually two but occasionally more peaks in the response. A divided consensus can indicate that the question needs to be rephrased or split into more questions or that perhaps people just don't know.
Most Delphi polls operate on small numbers of participants, usually experts chosen for expertise in special areas of knowledge. There has not been much effort in constructing Delphi pools open to the public, though.
The Delphi process isn't a magic panacea and it doesn't always produce clear or correct results, but a majority of the time properly worded questions can elicit knowledge that was not previously accessible.
The DARPA project was not a Delphi poll, though it shares some aspects with it. Instead, the Political Analysis Market (PAM) was modeled on commodity futures exchanges. In one graphic of a dartboard (at the bottom of this page) the PAM site claimed better results than Delphi, but how they substantiated this claim is unclear. The now-vanished web site restated what is known as the "strong" version of the efficient market hypothesis -- the idea that markets capture all the information to the extent that prices accurately reflect the "true" value of a commodity.
The implication that in an efficient market prices match reality is a misunderstanding, albeit a very widespread one. What the efficient market hypothesis means is that you cannot systematically locate pricing errors in order to cash in. This is a whole different kettle of fish. Bad prices may predominate in an efficient market, but you can't find them.
The place where people go wrong in interpreting the efficient market hypothesis is to assume that it is some sort of inviolable law of nature that guarantees all prices are perfect at all times. A quick search of the web for statements about efficient markets will quickly locate many of these kinds of blunders. In fact, you are more likely to find a misinterpretation of the hypothesis on the web than an accurate explanation.
One way of stating the strong form is that a strong form efficient market is immune to accounting fraud and insider trading. There are two other versions of the hypothesis. The semi-strong form says publicly held information reflects pricing accurately enough that barring things like stock-rigging schemes and insider trading there is no way to "beat the market" by technical (statistical) or fundamental (accounting) analysis. The weak form efficient market hypothesis basically says that past price history is valueless in predicting future prices.
Very few markets have been shown to satisfy the strong form of pricing efficiency -- cheaters prosper and market-rigging like the Enron scam is common. Some markets are exploitable in the semi-strong sense. And most markets meet the weak form hypothesis - the past doesn't reliably predict the future. You can't read tomorrow's headlines in today's paper. The problem with the efficient market hypotheses is that the inefficiencies that can be located are often not enough to overcome transaction costs like taxes, brokerage fees and such.
There are numerous examples, however, of markets (deriviatives are notoriously inefficient) that contain pricing inefficiencies that can be turned to a profit by operators with sufficient speed, skill, capital and access to information. So the bottom line is the efficient market theory is broadly true and narrowly false. Monkeys throwing darts at stock listings are often better and rarely worse than the "experts." And you can't get tomorrow's prices from today's paper.
But there are some situations where markets are so bad at setting prices that the fast and clever can systematically and repeatedly make a killing. Derivatives are the classic example of markets which are uniformly sprinkled with violations of the semi-strong hypothesis. So why did DARPA design in deriviatives?
The collapse of the PAM scheme was sudden and the collateral damage reportedly includeds Adm. Pointdexter's departure from DARPA. The suddeness of the collapse is a pity, since the scheme deserved a more detailed public examination than it got before being hauled off to the knacker's yard. Long story short, PAM was a dog of an implementation and deserved to be put to sleep before it bit somebody. I could go on at great length about how the PAM scheme was a beautiful example of a market that was neither free, open or efficient. But enough, it's sunk and the Admiral has gone down with it. Sufficient unto the day.
Hearing Aid - the Delphic coracle
In The Shockwave Rider, the problem with corrupt regulation of markets was addressed in an interesting manner. One of the novel's subplots involves a service organization named "Hearing Aid." The future in Shockwave Rider features a society where surveillance is total, except of course, for the favored elites. One absolutely secure means of communication does exist, however. Hearing Aid is a private service organization that provided a public health function: they listen. Anyone can dial ten nines into a phone and say anything they want, for as long as they want, about anything at all. At the end of the call, the Hearing Aid operator says, "Only I heard that. I hope it helped." The lines are secured by a massive computer worm/virus that inhabits the national data grid and defeats all attempts at evesdropping by the authorities.
In a world caving in under the stress of accelerated change, massive corruption and intrusive government, Hearing Aid performs a valuable service. It gives people a chance to unburden themselves without fear of reprisal. Given the high degree of surveilance (including continuous monitoring of the Delphi boards to maximize the "social mollification factor") in the future, not even the sanctity of privileged communication with a priest, therapist, lawyer or doctor is assured. So Hearing Aid is a unique safety-valve for the future-shocked.
It bears a lot of resemblance to Danius Maximus' proposal for a witness protection program for people who suddenly stumble out of the fun-house of the media simulacrum into the harsh daylight of real knowledge. Given the trend in the abolition of civil liberties and privacy rights under things like the Total Information Awareness (brought to you by the same wonderful folks at DARPA, but now renamed "Terrorist Information Awareness" -- making you a terrorist because the snoops are spying on you) and the Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act of 2001 (which pretty much abolishes the concept of legal rights for the accused), there are a lot of people who need a safe place to unburden themselves of their knowledge of the crimes of the state.
Here's a key passage. Speaking are Nick Hafflinger, a computer prodigy who escaped from a government program, joined up with Hearing Aid, was captured and is now being interrogated by Paul T. Freeman, an investigator for the federal Bureau of Data Processing. Mr. Freeman is about to get a nasty suspicion confirmed:
"But he was trusting Hearing Aid, wasn't he?"
"Yes, and that's one of the miracles performed by the service. While I was a minister I was resigned to having the croakers monitor the link to my confessional, even though what was said face-to-face in the actual booth was adequately private. And there was nothing to stop them noticing that a suspect had called on me, ambushing him as he left, and beating a repeat performance out of him. That type of dishonesty is at the root of our worst problem."
"I didn't know you acknowledged a 'worst' - you seem to find new problems daily. But go on."
"With pleasure. I'm sure that if I start to foam at the mouth there's a machine standing by to wipe my chin. . . . Oh, hell! It's hypocritical hair-splitting that makes me boil! Theoretically any one of us has access to more information than ever in history, and any phone booth is a gate to it. But suppose you live next door to a poker who's suddenly elected to the state congress, and six weeks later he's had a hundred thousand-dollar face-lift for his house. Try to find out how he came by the money; you get nowhere. Or try confirming that the company you work for is going to be sold and you're apt to be tossed on the street with no job, three kids and a mortgage. Other people seem to have the information. What about the shivver in the next office who's suddenly laughing when he used to mope? Has he borrowed to buy the firm's stock, knowing he can sell for double and retire?"
"Are you quoting calls to Hearing Aid?"
"Yes,- both are actual cases. I bend the rules because I know that if I don't you'll break me."
"Are you claiming those are typical?"
"Sure they are. Out of all the calls taken, nearly half - I think they say forty-five percent - are from people who are afraid someone else knows data that they don't and is gaining an unfair advantage by it. For all the claims one hears about the liberating impact of the data-net, the truth is that it's wished on most of us a brand-new reason for paranoia."
The Shockwave Rider
Meanwhile, back here in the real world, we find the web playing a role very similar to Hearing Aid, but (sigh) without the facility of being centralized through a face-to-face community. There are advantages to the world wide world. Human contact being the first that comes to mind. The Shockwave Rider is a novel, after all, and we are stuck here in a world in conflict.
The point of all this being that the web is a way we can bear witness to each other and speak truth to power. I'm certain that there are plenty of people in high places who would just love to shut down, spy on, interfere with and generally turn the web into a means of domination like the media. Lord knows, Adm. Pointdexter certainly tried with that abominable Total Information Awareness scam. So far, we appear to be holding our own . . .
A day at the races
In the early 1980's, efficient market theory branched out into a new field: the race track. It turns out that there are massive similarities between the pricing models of markets and parimutuel gambling. Both are markets with considerable risk and uncertainty. The stock, bond and commodity markets are supported by the government in several ways and allow a reasonable expectation of a profit (in the long run.) Horse racing, being a form of pleasure, is taxed considerably more than the financial markets. As a result, they have a negative expectation of a profit (in the long run.)
Parimutuel racing works like this: the bets go into a pool, but are recorded as to what portion of the money was bet on each horse. The track "take" (usually 15-20%) is deducted from the pool as the house cut. The odds (prices) are computed according to the proportion of the money bet on each horse. At the finish of the race, the winning bettors (investors) are awarded the money in the pool. Unlike the commodities and stock markets, the fixed take guarantees the investors (bettors) always get back most of their money. And unlike the commodities and stock markets, parimutuel markets are not prone to booms and crashes.
The efficient market research on horse racing found several biases in parimutuel betting odds (prices.) A market can have biases but still be efficient if the biases are too small to be exploitable for profit. One of the earliest papers, "Horse racing: testing the efficient markets model" by Wayne W. Snyder, showed some biases existed relative to the published odds in newspapers and racing forms and the actual odds determined by the bettors at the track. But the biases in the odds discovered by Snyder were insufficient to yeild a profit. So at least at first glance, horse racing was found to be at least weakly efficient.
A 1984 study in the Journal of Business, "Market efficiency in racetrack betting" by Peter Asch, Burton G. Malkiel (the author of a famous book on efficient market theory, titled A Random Walk Down Wall Street) and Richard E. Quant nicely summarized the state of knowledge about parimutuel pony punting:
There is a systematic tendency to overbet longshots and underbet favorites
The tendency to overbet long-shots is strongest in late races
The utility function of bettors is convex indicating risk love.
Races are "efficient" with respect to the information captured by handicappers' picks and track odds.
A substantial inefficiency exists in place and show betting (see: Hausch, D.B.; Ziemba, W.T.; and Rubinstein, M. 1981 Efficiency of the market for racetrack betting. Management Science 27:1435-52)
The first three finding say that there are biases in the betting structure. The fourth says that these biases are not large enough to make the market inefficient with respect to the handicapping odds published before the races (you can't make a profit betting for or against the handicappers.) The fifth says systematic biases in the place and show bets are frequently large enough to yeild a postive return.
The final result is definitive: parimutuel horse racing fails semi-strong efficiency (meaning statistical analysis reveals opportunities for sure-thing bets) in regards to place and show wagers, but the win market has stood the test and appears to be efficient. To sum up, the ponies are a demonstrably fairer (in sense of transparency and efficiency) than the mumbo-jumbo hodge podge of the commodities and stock markets. They are also more stable.
The comparative stability of horse racing compared to the finanacial markets is due to several factors:
The high transaction cost of betting (the track "take") overwhelms most of the exploitable biases.
Horse racing is much more rigorously regulated.
The wagers are limited in time and frequently repeated.
There are a small number of inter-related bets (derivatives).
The race track market is vastly more transparent than the financial markets. The complexity of the financial markets allows price-rigging schemes like Enron to flourish behind a wall of concealment.
A modest proposal
The DARPA proposal for Political Analysis Markets was flawed. But the notion that knowledge markets can be used to improve policy and public understanding is fundamentally sound. The problems with DARPA proposal were lack of transparency, an inefficient pricing mechanism and the widespread opportunities to profit from crime, collusion and inside information. It lacked the corrective transparency illustrated in The Shockwave Rider by the Hearing Aid scheme.
A parimutuel knowledge market is possible. Here is a sketch of how one such market would provide transparency, regulation and efficient pricing:
A parimutuel pool would be created using Delphi feedback by offering bets based on the degree of certainty. To make the market free and open, the issues could be proposed by anyone willing to pay a "filing fee." The fee would be minimal, just enough to cover the average cost of administering and adjudicating new issues plus a small amount of profit by the operators.
Regulations would govern what constitutes an acceptable new issue. Here are some rough guidelines for tendering a new issue:
It must be clearly decidable on a yes/no basis. The decision criteria must allow unambiguous determination of the question. Issues seeking to determine a quantitative answer would be phrased like "On November 11 will the price of crude oil be below $25.00?"
Where neceessary, the tenderer and/or the stewards may designate the acceptable authority for the information being decided.
It must have a fixed expiration date for calling the bets won or lost.
There must be a cut-off time prior the expiration date after which the betting closes. Since we want to encourage future predictions, there has to be a length of time for the "race" to run.
No issue can be based on the solicitation of criminal acts or the outcome of court rulings. Indictments, arrests, electoral outcomes, outbreaks of hostility, passage of legislation, dismissal from office, etc. are fair game.
The same question can be posed with different cut-off dates. This allows multiple wagers on the same event, but with different time periods providing that the betting periods do not overlap. These series would be the equivalent of tracking polls.
Once an issue is accepted and posted, betting proceeds on a parimutuel basis. Each bet may be valued at an integral number of dollars (i.e. no cents). A fixed proportion of the pool is deducted as the "take" (house percentage.) The take will probably initially fall in the range of 10%-20%. The optimal take can be determined operationally after sufficient statistics have accumulated so that it can be set as low as possible to counteract the measured bias of bettors regarding long shot issues while still covering the operating costs and a reasonable profit.
Bets are placed on a sliding scale of probability ranging from certain to occur to certain not to occur with a few intermediate positions: Certain yes -- very likely -- somewhat likely -- somewhat unlikely -- very unlikely -- Certain no. This range provides the confidence estimate needed for the Delphi certainty dimension and also allows bettors to hedge their "investments." The intermediate positions are assigned fractional values of a "yes" result, i.e. "very likely" means 80% confidence in the answer being yes, "somewhat likely" 60%, "somewhat unlikely" 40%, "very unlikely" 20%. "Certain yes" is 100% confident and "Certain no" is 0%.
A bet on an intermediate position pays a fractional amount depending on certainty. One scheme for apportioning the bets would pay out the certainty figure for that bet plus an apportioned percentage of the losing pools. For instance, a "very likely" bet would get no share of the "Certain yes" pool, 80% share of the "very likely" pool and proportionally smaller shares of the remainder. This scheme encourages betting on the ends of the scale, but makes it possible to hedge bets where no strong certainty emerges.
As the betting proceeds, the amounts wagered on each scale will be continuously updated and displayed as odds in percentage of the total pool bet on each category, as well as the aggregate amounts on "yes" and "no" (constituting the "line.") The payouts for each of the confidence levels in the event of a yes or no outcome will be displayed as well.
On the expiration of the bet, the issue is decided by the stewards according to the decision criteria spelled out in the issue tender. If a decision of the stewards is challenged, a challenge bond equal to 10% of the betting pool must be posted by the parties making the challenge. The steward's decision will then go to an appeal board and if the challenge is upheld, the bond is returned to the challenger. If the challenge fails, the bond is added to the betting pool for distribution to the winners (less the "take.")
Upon the final decision, the bets are redeemed according to the displayed payouts.
A sample example
Suppose the issue was "Were WMD found in Iraq?" The question was posed in early June, the cutoff for bets was June 18 and the expiration date was July 1. The "take" is 15%. The data here reflects the PIPA poll which asks precisely this question.
Were Weapons of Mass Destruction found in Iraq?
The betting line is 24% Yes / 76% No
Payouts per $1 bet (15% take)
"Yes" "Very Likely" "Likely" "Unlikely" "Very Unlikely" "No"
Yes $2.85 $3.23 $2.74 $0.91 $0.26 $0.00
No $0.00 $0.25 $0.74 $0.82 $0.95 $1.17
The correct answer is "No" and the lower payout line is what people would collect. Note the low returns on hedging on the "unlikely" and "very unlikely" bets. This is due to the high degree of confidence in the "No" answer (39%). The spread of the answers with peaks at both ends is typical of strongly divided opinion. These "inverted" opinions with peaks at the ends and a trough in the middle often indicate that the difference of opinion is due to people making their decisions based on different sets of information. The high incidence of incorrect belief in the presence and discovery of non-existant weapons of mass destruction is almost certainly due to misinformation in the media.
If you had bet "No" on this issue, you'd be sitting pretty on a 17% profit!
One of the serious flaws in the PAM scheme was the lack of a free market in issues. The participants in the market were restricted to a set of "stocks" limited by the operators of the market. As some people have pointed out, this only allowed predictions based on previous assumptions. The parimutuel example outlined above would operate as a free market -- the restrictions on issues would be fairly narrow and almost any question would be fair game.
Likewise, the presence of derivatives in the PAM scheme opens the door to substantial inefficiencies without improving the underlying accuracy of the system as a whole. The Information Warfare Site's copy of PAM screens shows (at the very bottom of the page) an example of derivatives. The prices shown are perfectly calculated. However, the "random walk" of the bidding on the derivative prices would almost certainly open up opportunities for sure-thing bets exactly like the derivative inefficiencies exploited by Ed Thorp. The parimutuel market does not allow derivatives and thereby avoids a needless source of pricing inefficiency.
A major problem in policy "markets" these days is the tendency towards groupthink and politicized intelligence. Public opinion polling doesn't address this problem because it is neither free nor open. The issues polled are not determined by the participants and the pool of participants is closed. The sampling bias of polling organizations is due to small pool sizes and different selection rule for choosing who gets asked, leading to the consistant biases noted by Dr. Pollkatz. In The Shockwave Rider, John Brunner offers the example of Hearing Aid as a way of increasing the transparency of information in society. The parimutuel predictions market would offer a similar function. Got a conspiracy theory? Put your money where your mouth is. Got a whistle-blowing situation? Step up to the window and buy your ticket. Think a situation needs some fresh air and daylight? Place your bets.
Currently, the transformation of the politics and the media into entertainment has made emotion the deciding coin of the realm. How about making reality the coin of the realm? Unlike the current game, the parimutuel Delphi pool offers substantial rewards for being in the minority and being right.
There is, of course, a system of government based on people being able to directly influence the policies that govern their lives. It's called democracy. Scott Buchanan has argued that democracy is not just a form of government but it is the basis of all government -- because no government can function indefinitely without the consent of the governed. Maybe it's worth a try.
The Shockwave Rider plebiscite
In The Shockwave Rider, the dramatic resolution comes about by a sudden collapse in the secrecy necessary to maintain corruption in government. The upshot of this new public awareness is a plebiscite calling for a more open and fair government that would abolish corrupt privilege:
THE CONTENT OF THE PROPOSITIONS
#1: That this is a rich planet. Therefore, poverty and hunger are unworthy of it, and since we can abolish them, we must.
#2: That we are a civilized species. Therefore, none shall henceforth gain illicit advantage by reason of the fact that we together know more than one of us can know.
THE OUTCOME OF THE PLEBISCITE
Well -- how did you vote?
The Shockwave Rider (final words)
Update: Here's a 8/17/03 NYT article on an antiquated market with substantial difficulties in establishing prices which could easily be remedied by a Delphi/parimutuel model.