Decision Workshops
If we look carefully at this graph we can see that using judgement or using game theory is worse than just guessing in several cases, and altogether the average is only a couple of percent better than chance. This is not very impressive, especially as it is often the method used to forecast outcomes. In most cases Game Theory does slightly better, but the score of 0% in the "Telco Takeover" scenario lowers the average. It was suggested that Game Theory does so poorly because of the amount of simplification that had to take place to get everything down to numbers lost the fidelity
Results: Role-Play
Role play does considerably better beating every other method in all cases. Also in all cases the modal result (that which came up most often) was the actual result.
All this shows that, despite what some people think, the best way by far to forecast the outcome of a negotiation is to do role play.
Can we do better?
The method of role play including Dilemma Analysis used by Decision Workshops has several differences from that used in the experiment. These are designed to increase the accuracy of the forecast: namely:
Using the Options Table keeps the role-play on subject.. By writing down and displaying the important points of a conflict, this ensures that people do not get distracted into talking about less relevant aspects, concentrating on the actual important decisions.
Using Dilemma Analysis stops players changing their positions too easily. If cards have high scores associated with them (either positive or negative) then people must give plausible reasons as to why they should change their views, they can't just do it on a whim to see what happens.
Using Dilemma Analysis means that personalities are less important. The outcome of a workshop will depend less on who plays what role and more on the structure and power of the confrontation. We should get much the same result, even if players change roles.
Rewarding players for using analogies makes the best use of expertise (see this paper)
Results: Unaided Judgement and Game Theory