(Note: The following was made available by John Barnes to other members of the Science Fiction and Fantasy Writers of America, in the context of a discussion of the best system to use to select the Nebula Award winners. The Nebula Awards are the "Academy Awards" of the SFWA. This article is reprinted with the permission of the author.)
The re-stirring of the Nebula controversy has prompted me to dust off some old skills -- what is called "formal modeling" in political science. Formal modeling is the mathematical study of voting systems, election procedures, committee systems, etc. to determine which, if any, players the system is biased in favor of. The subject is subtle and difficult in some ways, chiefly in seeing some of its furthest-out paradoxes and implications, but the basics are easy, and anyone who can read set theory can read most of the theorems in the field. Those who insist on mastering the whole subject for themselves -- and I think it's terrific story material, and might have a lot of applications to game design -- are hereby referred to Arrow, Social Choice and Individual Values; Downs, An Economic Theory of Democracy; Kelly, Arrow Impossibility Theorems; MacKay, Arrow's Theorem; and Luce and Raiffa, Games and Decisions.
Rather than present theorems here, I'll try to write out the logic of the theorem in a few sentences of more-or-less clear English each time. Where that proves impossible, I shall wave my hands and hope you all trust me. If you don't, please do feel to call or write and I'll try to further convince you.
The material below divides into three sections:
1) What properties of any "collective choice mechanism" (voting system, market, constitution, etc.) can be abstracted and studied mathematically. That's where you get the definitions. Likely to be rough and somewhat disconnected sledding, so do skim it, but please don't skip it, or at least if you find something you don't like later in the piece, please make sure of definitions from Section 1 before you wade into the argument -- I expect this to be messy enough without getting into definitional battles. On the other side of things, if any of this seems to reinforce your more cherished prejudices, please check the definitions to make sure it does.
2) Why no voting systems can work perfectly for all cases, and thus we are always making choices between evils of one sort of another. How different systems can be analyzed to show their intrinsic properties.
3) As-short-as-I-can-make-them analyses of several different voting systems. Because they've all been discussed as a way of replacing the current (Australian/Hare) system, I'll give special attention to weighted voting systems, plurality vote, and required majority/primary systems, plus of course the Australian system itself. Because I think we ought to consider some of the others as well, I will briefly explore straight majority systems, pairwise voting, and shadow-market (transferable vote) systems.
As for my personal recommendation, in my judgement the universe, as revealed by Arrow's Theorem (see below), is sufficiently perverse that I don't like _any_ of these systems but don't believe a better one can be invented. My recommendation is that we all spare ourselves a lot more fuss and feathers by deciding, once and for all, which set of deficiencies we'd rather live with, and then do it.
Acyclicity: choices and preferences are acyclic if they cannot be chained together transitively to produce a cycle (see Transitive, below.)
Arrow's Theorem: the mathematical demonstration by Kenneth J. Arrow which shows that for any system of social choice -- election, market, or representation -- which is both rational and democratic, there is at least one possible structure of individual choices that will cause the system to violate either democracy or rationality. Thus, turning it around, no possible set of rules can produce a democratic and rational result for all possible sets of individual preferences.
Collusion: a system is said to reward collusion if the total effect of a group of voters acting together can be made larger than the effect of the same group of voters voting their individual preferences. A system is said to require collusion if collusion must be present for the system to be determinate.
Democratic: a system of choice is democratic if it is nonimposed, nondictatorial, and obeys the Weak Pareto Condition.
Determinate: a procedure is determine if it always arrives at some answer, no matter how screwy or perverse.
High Consensus: a situation in which virtually all individual preference structures are nearly identical. Thus, in a high consensus, if you disagree with anyone about your first and second choices, chances are his first is your second, and vice versa.
Independence of Agenda: a voting system with multiple elections exhibits independence of agenda if the order in which the choices are presented to the voters has no effect on the social outcome. By definition any system in which only one ballot is taken is independent of agenda.
Independence of Irrelevant Alternatives: a system is independent of irrelevant alternatives if adding alternatives that nobody prefers to the ballot cannot change the outcome of the social choice process.
Low or Scattered Consensus: situation in which virtually all of the possible individual preference structures are present in the electorate in appreciable numbers. Thus in a scattered consensus, any two individuals are about equally likely to have any two different preference structures.
Nondictatorial: a system is nondictatorial if there is no situation in which one individual prefers A, everyone else prefers B, and the system picks A. (Please note that this includes not just pre-selected "dictators," but also dictators created by the voting process itself; e.g., randomly pulling out one ballot from the box and giving the Nebulas out according to that single ballot is dictatorial according to this definition.)
Nonimposed: a system is nonimposed if there are no rules against anything on the ballot winning.
Polarization: situation in which a low number of individual preference structures, (but more than one), are present in the electorate, and the structures are drastically dissimilar. Thus in polarization, any two individuals will be either in close agreement or in drastic disagreement.
Preference structure: the order in which an individual ranks the available alternatives.
Rational: Individual or group choices and preferences are said to be rational if they are determinate, transitive, acyclical, and independent of irrelevant alternatives.
Social Choice: any procedure by which individual preferences are surveyed to produce a single choice for the whole group.
Strategic Voting: voting for something different from what you want because some peculiarity of the voting system will cause the choice to go to what you really want. A system is said to reward strategic voting if it is possible to construct situations in which your maximum chance of seeing the social choice go your way can occur when you vote for some other position.
Transitive: preferences and choices are transitive if (X is preferred to Y) and (Y is preferred to Z) implies (X is preferred to Z).
Weak Pareto Condition: a system obeys the Weak Pareto Condition if unanimity always carries; that is, if everyone votes for A and against B, then A, not B, is always the social choice.
The first things to note above are that democracy and rationality are defined very precisely, but the definitions themselves are loose. The definition of democracy here merely says, in effect, that everyone's vote counts, anything on the ballot can win, and that anything that's unanimous will pass.
The definition of rationality is a bit tougher, but in essence it asks us that the system not defy common sense. Transitivity and acyclicity, taken together, only say that I can't set up a "money pump," which is the economic equivalent of a perpetual motion machine. If you like apples better than bananas, bananas better than tangerines, and tangerines better than apples, then presumably you would be willing to pay some small amount to trade to the thing you liked better (how small doesn't matter). So I could trade you tangerines for apples (and collect money), bananas for tangerines (collecting more of your money), and then apples for bananas (collecting still more of your money). Now I've gotten it back t where it started and can collect more money for another trip around the cycle. So anyone, or any system, that is not acyclic and transitive, is surely irrational by most people's definition. (If you've got cyclic preferences, please let me know -- and have your checkbook ready). For the Nebulas, we don't want a process that could always be somehow cycled one more time to produce a different result that we would all like better.
Determinate simply means it comes up with an answer, even if it's just the celebrated Mr. Noah Ward. I hardly think we want the trouble and expense of an election in a system which is nondeterminate and thus may give us nothing at the end.
Independence of irrelevant alternatives is a subtle point. One traditional example may help: suppose you hate fish. You go to a restaurant and are told the alternatives are steak or chicken. You pick steak. The waiter then says, "Oh, and we also have fish." You then say, "Oh, in that case I'll have chicken." Now, I realize you're all fiction writers and are now happily building 50 scenarios to explain that behavior, but remember we're in high abstraction here; absent all those bizarre circumstances you just thought of, it's pretty weird to react that way. If you like, take any close contest of some years back -- say The Mote in God's Eye vs. The Dispossessed. Would your vote have been changed, between the two of them, if one of the Gor books had been added to the ballot?
So the first thing to note is that rationality and democracy, defined here, are not very restrictive. A lot can be fit in under them.
Three of the other terms up there are ways to do things that many of us don't like. Rewarding collusion means simply, that there is some way that ten clever members who arrange their votes properly have more say in the outcome than ten staunchly independent types who happened to vote identically. It places a premium on people's organizational skills and personal popularity. Strategic voting gives the clever a big break over the honest, and further means that in the event of a crisis it's impossible to determine what people really wanted, as opposed to what they voted for. A system that is not independent of agenda can always be manipulated by someone (trust me on that one -- the theorems take several pages) not in the majority.
Three other terms cover what the voting system may have to cope with. High consensus, low or scattered consensus, and polarization are best thought of as situations. It's worth mentioned that almost any system will produce a not-too-controversial result from a high consensus; results from a low consensus will probably please nobody very much, and results from a polarization will displease at least a substantial minority severely. They're mentioned up above because different systems are differently vulnerable to different varieties of consensus. If you think one type is more likely than another it may have some bearing on what type of voting system you would prefer.
Arrow's Theorem is subtle and complicated in its full form, but a simple degenerate case of it is all we really need here: suppose we define the system of selection as being both democratic and rational, and then see what trouble that gets us into.
We'll work with just a 3-work ballot, and 3 blocs of voters (it's possible for both number of candidates and number of voting blocs to show that what works for N works for N+1, so the extension to the real Nebula ballot is quite appropriate.)
In the 1001 voters in the Silly Factional Writers of America, there are 3 groups:
500 cyberducks, all of whom like Electric Duck better than Space Duck, and Space Duck better than Sword of the Duck. (Perhaps because they hate Sword of the Duck so much they'll vote for anything against it.)
500 duck-fantasists, all of whom like Sword of the Duck better than Electric Duck, and Electric Duck better than Space Duck.
and of course,
Fred, the author of Space Duck, who likes his own book best, and prefers Sword of the Duck to Electric Duck.
Now, what choice will a rational, democratic system make?
Surely not Space Duck -- 1,000 members to 1 like Electric Duck better, and Fred does not get to be a dictator because the system is democratic.
What about Electric Duck? 501 to 500 people like Sword of the Duck better.
So we take Sword of the Duck. The problem? 501 people like Space Duck better! We've just violated acyclicity.
Clearly, simply majority rule won't work here. So we try to use more of the rules to get a result.
We cross of Space Duck entirely, because it would have to be a dictatorial or imposed result to win. Now, that's better -- Sword of the Duck wins 501 to 500. Except that now we've violated the independence of irrelevant alternatives.
Because we're working from the _definitions_ of rationality and democracy, what can be shown is that a structure like this will screw up any rational, democratic decision-making process. That's Arrow's Theorem, in a crude nutshell; you can see it much more elaborately in the sources I've cited above, if you're interested in the real, complicated math for its own sake.
But what interests us here is something quite different. Formal analysis can not only tell us that any system will break down, but how a specific system will break down.
That has some real impact on how we decide we want to choose the Nebula. It may be possible to set up a system that will break down in a way we can live with -- something that won't produce too ludicrous or unpopular a result. Probably the Silly Faction Writers of America could live with either Electric Duck or Sword of the Duck winning, and thus a system that simply, definitely picks one of them may be good enough for the job, even if it doesn't do it perfectly. The two things we don't want are perverseness (giving an award that makes most people say "That's ridiculous -- nobody voted for that"), and any appearance of dishonesty (subject to the problem that dishonesty is defined differently by different people -- is collusion, as defined above, a way for people to conspire against the will of the majority, or does it simply reward the more ardent supporters of a work? Is strategic voting unfair to the truthful or does it provide a powerful tool _against_ things you strongly don't want to win? The answer to both questions, I'm afraid, is "Yes, both.")
Section III, below, explains the breakdown mode and the vulnerability to corruption of each popular voting system. Just for grins, I've explained also what the result would be for the hypothetical case cited above.
The Australian (Hare) Ballot -- the current system.
In this system, a work is required to obtain a majority of total votes cast in order to win. Voting is on a ranked ballot, with decision made by successive elimination; first place votes are tabulated, and if any work has a majority it wins. If not, the work with fewest votes is dropped from the running, and its votes are redistributed according to the second place choices on those ballots. This procedure is repeated until something gains a majority.
In the above example, there would be 500 first round votes for Electric Duck, 500 for Sword of the Duck, and 1 for Space Duck. Thus no work would have a majority. At that point, Space Duck would be dropped from the running, and those ballots (in this case just one) which had Space Duck as first choice would now be assigned based on the second choice -- in this case, Sword of the Duck. The second round result is Sword 501 to Electric 500, so Sword wins.
Notice that 501 people would have preferred the work that was dropped from the running -- so the system definitely did break down.
The test case is highly polarized, obviously, and those are the circumstances in which an Australian ballot is most likely to break down. In general if there are roughly equal-sized blocks of people, with highly consistent preference structures within each bloc (in this limited case preference structures are identical within blocs), the decision will tend to get made by the lower-level choices (2nd, 3rd, 4th choices) of the few eccentrics outside the blocs. In this relatively pure case, the Nebula was awarded based on Fred's concept of the lesser of two evils.
Where consensus is high, but for some reason a majority doesn't turn up on the first ballot, the Australian ballot will almost always give the Nebula to the work that has the plurality. This is because the reassigned votes will tend to favor the front runner in a high consensus.
Where consensus is low, the Australian ballot is extremely sensitive to differences in the number of first-place votes cast for the less popular works. To see that, you need only realize that the order in which works are eliminated has a tremendous effect on the outcome, and thus a vote or two of difference between the last and next-to-last positions after the first ballot can completely alter the outcome of the process. This at least borders on a violation of independence of irrelevant alternatives.
On the positive side, if any work does have a majority of first-place choices the Australian ballot will select it. It will never elevate a genuinely unpopular work (say something with a minor cult following) to the top. It will tend to favor works that are, if not the first place choices, at least the second choices of nearly everyone. This may well minimize post-election grumbling, but it does mean that the Nebulas are currently slanted toward the consensual than the controversial.
There are no rewards for strategic voting or collusions in an Australian ballot, and this is its biggest strength. As long as ballots are collected and counted honestly, there is no way to manipulate the process (for those of you interested, you prove this by demonstrating that any change from the honest ballot (i.e. voting the way you feel) will hurt your preferred candidates).
Weighted voting -- apparently the old system.
In the simplest version we assign three points for each first place, two for each second, and one for each third. The point totals are then Sword of the Duck 2002, Electric Duck 2501, Space Duck 1503, so Electric Duck wins, even though a majority would prefer Sword of the Duck.
Now, according to Jerry Pournelle, the actual former arrangement gave an extra point for a first place, so there would be 4 points for each first place, 2 for 2nd, and 1 for 3rd. This gives us Space Duck, 1504, Electric Duck 3001, and Sword of the Duck 2502.
Consider the smallest number of ballot changes needed to change the result. In the first case, it's 125; in the second, it's 84.
What we're looking at here, of course, is the problem that the assigned number of points per position determines the sensitivity of the system. In fact, in the first case, Electric Duck could have as few as 376 first-place votes to 624 for Sword of the Duck and still win. Furthermore, it's not strictly the number of points assigned per position, but the ratios between those points per position -- and that changes with the number of items on the ballot. In general, the more items on the ballot, the fewer voters would need to change their minds to reverse the results of the election. In close elections, weighted voting will actually violate independence of irrelevant alternatives -- adding a Gor book, say, to the ballot, would cause a different outcome, even though no one voted for it.
Strategic voting is highly viable on a weighted ballot. Suppose that 250 of Sword of the Duck voters (in the first arrangement) were fairly cunning and realized that Electric Duck was their chief rival. They refuse to rank it at all, even though in fact it's their second choice. Now Sword of the Duck wins, because it had enough voters smart enough to lie about their actual preferences.
Weighted ballots are moderately vulnerable to collusion between organized blocs, but only if the full results of the vote are announced. If voting blocs are compact enough, it's possible that supporters of a minor work with no chance of winning could cut a deal for an unexpectedly large point total by helping "put the winner over the top." Thus, if we adopt this system and are interested in preventing collusion, it's vital that vote totals remain secret (as they are today).
Weighted voting will tend to favor the "crossover" work in a polarized situation (the example, of course, is polarized). Thus it may produce a slightly livelier overall pattern of winners than the Australian ballot as long as there's considerable polarization in the electorate. When consensus is low or scattered, weighted voting will often pick the "least offensive to most people" alternative, winning by being many people's second choice. (Based on poll data, if weighted balloting had been used in the US presidential election of 1980, John Anderson would have finished a strong second to Ronald Reagan; in 1968, Hubert Humphrey would have won). Where weighted voting is truly unique, however, is that it will often pick most people's _second_ choice in high consensus; thus it does have the potential to give a perverse result in which practically everyone prefers something else to what won.
In this system, only first-place ballots are cast, and whichever gets the most votes wins. The analysis of such systems is trivial; you can apply Arrow's Theorem to them directly, so we'll just glance at this one.
The obvious problem with the above example is that it produces a tie, so we'll jockey with it a little. Notice that if we change Fred's vote (maybe because he's decided he can't possibly win), Sword of the Duck wins, even though Electric Duck is much more the consensus choice. In fact, plurality systems and any other system where only a first choice is voted for always throw away the preferences of everyone who didn't vote for the winner. They thus produce the "sense of the largest faction" rather than that of the group as a whole.
Because adding more items t the ballot can "siphon off" votes, plurality systems are quite likely to violate independence of irrelevant alternatives. On the other hand, they are immune to strategic voting or to collusion.
Like most systems, plurality works well for high consensus. Low consensus may result in an essentially random outcome; in the polarized case, plurality slightly favors extremist positions and thus is more likely to pick the controversial rather than the consensus decision.
Required majority systems.
In a required majority system, only first choices are recorded. If any work gets a majority of total votes cast, it wins. If not, the top two vote-getters (or in some systems the smallest number of vote-getters whose total votes add up to some percentage, usually 80%) run against each other in a runoff election.
In the example above, after the first election Space Duck gets dropped; then Electric Duck runs against Sword of the Duck, and because Fred has to vote for something else, Sword of the Duck wins a majority in the second election. As always, in an Arrow's Theorem case, the majority would have preferred something else.
Unless, of course, it occurs to Fred and enough Electric Duck voters that by strategically voting for Space Duck in the first round, they can defeat Sword of the Duck in the second. (Defeat it by giving the Nebula to Space Duck,, that is. The cyberducks can't win, but they can exercise sheer spite). This, of course, is only effective if they all get together to do it -- and that's the big problem with required majority systems. They strongly invite both collusion and strategic voting, except in the case of high consensus, where everybody's favorite is likely to win on the first ballot anyway.
Straight majority systems.
Exactly like a required majority system, except that instead of a runoff we declare no award in the event of no majority on the first ballot. Subject, of course, to all the same problems as above, if there's any faction that would rather see no Nebula given than see it go to
The maximum possible hassle, in many regards, for both voter and tabulator, but also uses the maximum possible information in reaching its results. Works are matched against each other in all the possible pairs, and you vote for your favorite within each pair. The votes on each pair are then tabulated, and using transitivity, a chain is constructed from least to most preferred. Most preferred gets it; if most preferred is part of a cycle, then either no award is given or all the other members of the cycle are also given the award (that is, cycles are basically treated as ties.)
Thus in the above example, the ballot would look like:
WHICH DO YOU PREFER? CIRCLE ONE ON EACH LINE
Space Duck / Electric Duck
Sword of the Duck / Space Duck
Electric Duck / Sword of the Duck
The results are then that by 1000 to 1, SFWA prefers Electric Duck to Space Duck; by 501 to 500, SFWA prefers Space Duck to Sword of the Duck; and by 501 to 500, SFWA prefers Sword of the Duck to Electric Duck. This, of course, neatly constructs a cycle, declaring either all of them or none of them the winner.
Pairwise voting is subject to some strategic voting (you can always vote for things you know won't win over your chief rivals) but does not reward collusion. In high consensus, it will produce the same results as majority rule; in low consensus, it tends to find "coalition-building" selections (that is it finds the work that most people think is at least a little better than most of the other things on the ballot), and in polarization it tends to produce cycles, meaning either giving out more than one award or giving out none.
Pairwise voting is a pain to fill out and to tabulate, but I include it here because it has features that many people have expressed an interest in. First and foremost, more than any other system it allows you to vote against a work you don't like. Secondly, it is in some ways unusually well-behaved across a broad range of consensus conditions -- when people all agree, it gives them what they want; when people are of many different minds, it will pick a work whose support is broad and varied; and when people are fighting, it either gives every major faction a prize or frustrates them all. Thirdly, you can vote in such a way that you affect only the standing of works you've actually read; if you don't get to something before the ballot is due, you just don't vote on the lines that include it.
Incidentally, for the Nebulas as presently set up, pairwise voting would require us to have 15 lines per category, for a total of 60 pairwise choices; currently you have no more than 20 lines.
Transferable vote systems (shadow markets).
In this system, you get some fixed number of votes (60 would be a likely number) which you may allocate any way you wish across works in all categories. Thus if you want, you can allocate votes 5-4-3-2-1 in order of preference in each category, or you can cast all 60 for one short story. Whatever gets the most votes in each category wins. It's called a shadow-market because it has a lot of the properties of a free market; people who never read novels and couldn't care less about them, but are passionate about short fiction, get more votes in short fiction, where they want them.
The example above [the Duck novels] doesn't really work for this system because it only applies to a single category.
A shadow market does not reward collusion. There is one highly strategic way to vote in it -- pick what matters most to you and put all your votes on that single work -- but it's quite possible that most people will not be able emotionally to put all their eggs in a single basket. On the other hand, it may mean some unpleasant "coupling effects" will occur between categories -- if, for instance, a novel and novelette that have quite similar audiences are both running, one may draw off support form the other. This might also be seen as an advantage, in that no single artistic movement is likely to sweep all categories.
In a high consensus situation, the shadow market will pick the majority preferences in most but usually not all categories. In low consensus, the results are very nearly unpredictable. If there is polarization in any category, almost all votes will be cast in that category that year, leaving the other categories to be chosen by a small and probably unrepresentative subgroup of the electorate.
A shadow market does not circumvent Arrow's Theorem (any more than a free market does) -- it simply moves it to a larger arena. However, because it operates across categories, it may conceal many of the Arrow-type problems and breakdowns well enough so that they won't be noticed by those within the system.
It has been truly said that if all the social scientists in the world were laid end to end, they would not reach a conclusion. Once again, every choice of voting system is both a selection of benefits to be gained and a selection of losses to be incurred. My strong recommendation is merely this:
1. Ask yourself whether you think SFWA is likely to be a high consensus, low consensus, or polarized organization, or to perhaps move between those states.
2. Ask what's most important to you in a work that gets the Nebula -- that it be a broad consensus choice? Innovative and controversial? "Fair" (whatever that may mean to you)?
3. Look for the system that is most likely to deliver what you want for the state of consensus you think is likely.
4. Accept philosophically that even if you're right, the system is going to break down every now and then.
5. Give up on this stuff, and go write.
More seriously, I do hope this has been of some help to at least some people.