Friday, August 12, 2016

Majors Stitch Up Senate Term Lengths, Film At 11

Admin note: clearance of comments will occur during the evenings and on weekends only for the next two weeks for work reasons.

---------------------------------------------------------------------------------------------------------------------

When the Senate resumes at the end of the month, one of its first items of business will be to allocate the Senators to six-year and three-year terms.  Today both major parties indicated that they will use the order-of-election method, under which the first six Senators who were elected in each state's cutup will receive six-year terms and the remainder three-year terms.  Special Minister of State Matthias Cormann has stated:

 "The important point is obviously this is a function of how many votes and how many preferences you are able to attract. If you are elected in the first six out of 12 then it stands to reason that you were elected earlier and as such you qualify for the longer period."

Labor's Penny Wong has issued a statement saying Labor will support the order-of-election method and saying that it "reflects the will of the voters".

The Electoral Act requires the AEC to conduct a "Section 282 recount" to simulate which Senators would have filled the first six places at a half-Senate election based on the same votes, as an alternative to using the order-of-election method.  However there is no requirement that the Senate use the recount to decide the order.  While the Senate affirmed in both 1998 and 2010 that the recount should be used, those decisions were not binding on the current Senate, and were under the old system anyway.

The Senate can in fact allocate the three and six year terms however it likes.  There is nothing in the Constitution to prevent the Senate from, for instance, giving all the crossbenchers three-year terms, or giving three-year terms to known Tony Abbott supporters or all Senators who opposed Senate reform (for example).  But there would probably be a massive outcry if the Senate did something that was obviously undemocratic.

It happens that both major parties benefit from the agreement to use the order-of-election method rather than S 282.  Senators Lee Rhiannon (Greens NSW) and Derryn Hinch (Vic) are both given three year terms while Scott Ryan (Liberal, Victoria) and Deborah O'Neill (Labor, NSW) will not have to look for work again until at least 2022 (unless there is another double dissolution).  This difference had previously been found by Grahame Bowland and the AEC has advised the Clerk of the Senate that Rhiannon and Hinch would get six-year terms if the Section 282 recount was the deciding factor.

Problems with the order of election method

Section 282 was brought in because the order-of-election method can lead to irregularities.  One of them is that if a minor party gets just over two double dissolution quotas, the minor party will win two six-year terms.  Then with the same vote it would win a third Senator at the next half-Senate election and hence hold a quarter of the state's seats off only 15.4% of the vote.  

When we look at the two different options in the 2016 case, it isn't immediately obvious that either causes such a problem.  Both Hinch and Rhiannon fell short of a DD quota and were the only elected candidate for their party in their state.  It doesn't seem to really matter much whether they are given three year or six year terms.

Yet there are some fairness-based problems with applying the order-of-election method even in this case.

I'll use Rhiannon's case as an example.  Her primary vote was 0.947 quotas (including 1 Greens ticket votes).  Because she was short of a quota on primary votes, she had to wait until the cascading surpluses at the start of the count elected eight major party candidates.  But had she polled a quota in her own right (just another 0.41% of the primary vote) she would have been elected third and won a six-year term.  

On the other side of that coin, Jacqui Lambie polled a primary vote of 1.068 quotas in her own right (including ticket votes).  This meant Lambie was elected fourth by the order-of-election method, and therefore gets a six-year term.  Yet the massive surpluses of the Tasmanian major party ticket leaders Abetz and Urquhart actually meant that seven Senators who won either on primaries or on cascading surpluses had more "support" than Lambie did.  Thus, as a sign of her support relative to these major party candidates, one might say that her having polled a DD quota on primaries was far from conclusive. 

Not only is polling a quota or not on primaries therefore a very arbitrary test of whether a rather easily-elected minor party candidate should get six years or three, but in a really borderline case it is worth bearing in mind that it is the candidate whose primary vote (including ticket votes) determines the term length, not the party.  So a minor party is disadvantaged if it runs a good support candidate who pulls primary votes from the lead candidate.  It is also disadvantaged if minor parties that are similar to it take primary votes from it, even if those minor parties are quickly excluded in the cutup with all votes they have taken from the winning party returned as preferences.

The order of election system is also absurdly granular.  In Tasmania the two major parties have about the same vote (Labor 33.6 Liberal 32.5) and get two six-year terms apiece.  But this would also happen if the figures were Labor 46 Liberal 20, or the other way around.

These issues arise because a double dissolution is an election for twelve Senators in a state.  The order in which one is elected in an election for twelve Senators says nothing conclusive about whether one is one of the six candidates voters would deem most worthy of a long term, because the bar to get over for the latter should rightly be higher.  

Problems with the Section 282 method

The Section 282 method seems like a better option.  It uses the votes cast in the election to simulate what could have happened had the same votes been cast at a half-Senate election.  In this way, it uses the election votes to conduct an election for six places, using the appropriate quota rather than the clearly inappropriate quota for an election for twelve.

There are issues, however, and a lot of them are created by the unexpected interface between the new Senate system and Section 282, which was written under a different system. Because the government left the implementation of Senate reform very late in its previous term (whether to avoid provoking the non-Green crossbench, in the hope of bipartisan support or simply because it was distracted by leadership fighting) a number of finer details were glossed over for the first run of the new system.  The matter of the fairest term-allocation system, which had long been on the back-burner anyway, was one that was so obscure that it wasn't even on most psephologists' shopping lists for things to fix for the new system.

Under the old group ticket system almost every vote was a full ranking of candidates - albeit by a voter who typically had no idea that they were ranking candidates in that way (and with very rare exceptions created by the old below-the-line savings provisions) - but now very few voters rank all the competitive candidates.  In NSW, only about 0.3% of voters went effectively all the way either above or below the line.  Most voters numbered just 1-6 above the line or 1-12 below, which means they tend not to have expressed a preference between all the candidates who were actually elected.  Some votes exhausted without reaching anyone who won.

These problems aren't as acute as they might appear, since parties that get elected tend to receive strong preference flows from voters generally.  In South Australia, where five different parties won at least one seat each, about 30.5% of above-the-line votes included at least four of the winning parties in their top six.  (Thanks again to David Barry's preference explorer.)

(Note added: Indeed, the proportion of votes that are not only usable but remain in the recount seems quite high.  Percentages of votes excluded from the recount based on Grahame Bowland's simulations appear to have been around: 4.6% Victoria, 4.1% Queensland, 3.7% NSW, 3.2% South Australia, 2.5% Western Australia, 1.7% Tasmania.  Exhaust rates during the recount are then estimated at: 4.5% NSW, 4.4% SA and Victoria, 2.3% for Queensland and Tasmania and 0.1% for WA.  So over 90% of vote-values were used.)

Still, when votes cast under the new system are processed in a Section 282 recount, some of them will carry no usable preferences, some will carry a few and some will carry preferences between all the elected candidates.  The effect is that if a voter's vote didn't reach any elected candidate then that voter has no say in which of the elected candidates get a six-year term.  But had the voter been asked to vote in an election between the elected candidates for that specific purpose, they may well have had a view on that.  Given that they weren't informed that they should vote all the way through if they wanted to have a say on term length, it doesn't seem ideal to exclude their vote from the assessment. 

That's not to say S 282 is less fair than the order-of-election system.  I think it's still fairer - just that it's hardly brilliant and it is worth a serious think about whether there could be a better approach than either for the new voting system.

There are also many ambiguities in applying S 282 to the new system.  Some of these existed in the old system as well (The AEC recommended fixing them in 2011 but it never happened).  I'll repost what I wrote on an earlier thread:

------------------------------------------------------------------------------------------------------------------
There is a detailed post up by Dean Ashley (Alaric) dealing with the ambiguity of the Section 282 rules to prepare a recommendation for which Senators get 3 and 6 year terms.  The drafting of these rules is quite a mess and this was a hot topic of discussions I had with a few people following the declaration of the polls today.  There are ambiguities in the wording of Section 282 concerning, for example:

* whether votes that were informal in the original count (eg they had two 1s) can become formal in the S 282 recount (I would think not)
* whether votes that were partially formal in the original count (because of duplications of numbers) can become more formal in the recount if numbers are no longer duplicated (again I would think not but this has never been clarified)
* whether votes that were fully formal in the original count can become informal in the S 282 recount because they do not have enough preferences (again I would think not, hardly the voter's fault)
* whether a vote that has zero usable preferences in the recount because it is entirely for excluded candidates counts towards quota although it in fact exhausts immediately (this is largely a quirk of applying S282 the new system, although it could have happened in the old system too; the AEC's view as of 2011 was no; Antony Green also believes such a vote is excluded).

(PS Another one has come up, and it's a cracker:

* whether a vote that was originally formal both above and below the line, but has zero usable preferences in the recount below the line can have its above the line preferences used in the recount instead.  I think the legislation has not anticipated this application of the new system at all!)
--------------------------------------------------------------------------------------------------------------------

I have received a copy of the AEC document sent to the Senate and it confirms that informal votes do not become formal and that votes with no usable preferences are excluded from the recount quota.  It sheds no light on the more esoteric points.

A myth about the Section 282 system has been that it disadvantages smaller parties.  It does so in Antony Green's example (where a minor party has two DD quotas but parties other than the "big three" have 5.6% of the vote between them).  However these days it is common for both major parties to poll not that much more than two half-Senate quotas in a state.  That creates a situation in which a minor party with just under one DD quota can win the fifth or sixth seat at a Section 282 recount, despite only winning one of the last few seats in the original election.  Thus at this election, it is actually the order-of-election method that disadvantages the smaller parties.

Alternative Proposal

Senator Hinch (who has been all over the place on this one, probably because he didn't initially realise the S282 recount would advantage him) has come up with a suggestion that every party that wins a seat should have at least one six-year term.

I think this is giving the micro-parties too much.  It means that the LDP and Family First, which have only snuck into the last seat in one state each because it was a double-dissolution, would get a six-year term off a vote that would not have got near winning one at a half-Senate election.  Hinch's case is quite different to theirs and S 282 recognises this.

The proposal is also open to rorting; for instance the Greens could deliberately devolve to separate state parties, thus winning themselves a six-year term in every state in which they won a seat.

Major Party Dealings

What is interesting here is that the AFR originally mooted (I am unsure on what evidence) that the major parties were looking at using Section 282 to disadvantage the crossbench.  Whoever had crunched possible numbers for this based on the results even as they stood at that time had absolutely no idea what they were doing. It has turned out that Section 282 actually advantages the crossbench, and, at this point, we have the major parties proposing to ignore it.

The charge of having an eye for the main chance is especially acute here in Labor's case.   Labor attacked the Greens and NXT for dealing with the Coalition to fix massive problems with Senate voting, accusing them of doing so for self-advantage and helping the Coalition in the process.  Yet now we have Labor doing what looks a lot like a deal with the Coalition to fix the Senate term lengths to its own almost indisputable advantage, and helping the Coalition in so doing. (I say "almost" because while the Greens grapple with their own savage case of NSW Disease, there's a case for Labor keeping Senator Rhiannon where she is as long as possible.) 

I doubt the expected decision is going to cause any great outrage, though it may produce some parliamentary theatre.  Those disadvantaged will complain, but it's a fact that the disadvantaged candidates polled less than a DD quota each, and would only have won six-year terms via the recount because of the weakness of the major parties, with exhaust perhaps a contributing factor.  It doesn't have the public-consumption no-go value of a Senator winning off half a percent of a vote or an election being voided and rerun, and by the next Senate election almost everyone outside the pseph community will have forgotten all about it. 

Monday, August 8, 2016

Senate Reform Performance Review Part 2

In this piece I continue the roundup of the performance of the new Senate voting system that I started in part 1.  Part 1 covered proportionality, half-Senate projections, winning vote shares, the impact of preferencing, just-vote-1 rates and exhaust.  This part covers  informal votes, the One Nation question, below the line rates, the two most unusual victories (Lisa Singh and Malcolm Roberts), the impact of the fixed quota, the question of verification and areas for improvement. (There's also a brief note about Inclusive Gregory that isn't relevant to reform, but that I thought I should mention anyway).  Anything else I think of may be tacked onto the end.  I have already discussed donkey-voting, proximity preferencing, the abject failure of most parties' how-to-vote cards, and preferencing patterns by state on the Button Press Week thread. Also, lots of Tasmanian-specific stats can be found on the Tasmanian button press thread.

Informal Senate Votes

The Senate informal vote rate at this election increased from 2.96% to 3.94%.  Given the confusion created when any new system is introduced, this is not a very large increase, but I still believe it's a bit larger than it should have been.  That said, the rate was slightly lower than the rate at the first two elections under the previous Group Ticket Voting system (1984 and 1987).  It also hardly stands out as an outlier compared to previous years, being about the same as at three of the last five elections.

(Before the 1984 changes, informal rates were usually scandalously high because voters had to fill in all the boxes.)

Here is a graph of the changes by electorate (noting that some electorates were heavily redistributed):


The top five electorates for informal voting (position in 2013 in brackets) were Blaxland (1), Watson (2), Fowler (3), McMahon (5) and Werriwa (10). Calwell (Vic) fell from fourth to sixth.  The top three are also the same top three as in 2010, but in 2007 the top five electorates were Victorian.  Now they are all in New South Wales.

Informal Senate voting fell in five electorates, generally inner-city seats: Grayndler, Melbourne, Reid, Higgins and Perth.  The six greatest proportional increases, however, were all in Queensland (Fairfax, Fisher, Wide Bay, Moncrieff, Fadden, McPherson), and Queensland had ten of the top eleven (the odd one out being Indi.)  If you spotted the (not necessarily causal) pattern that the Senate informal vote went up lots in seats Palmer United Party did really well in in 2013, you're not alone.

Given the dire forecasts about how the AEC would not have time to educate voters on how to use the new system properly, the increase is remarkably modest.  All the same I can think of two things that are worth improving to reduce informal votes for next time:

1. The savings provision for below-the-line votes, which requires voters to have the numbers 1 to 6 once and once each, is too stringent.  Voters sometimes accidentally omit or repeat numbers in this range and shouldn't have their whole vote junked for doing so.  An improvement would be to allow any below-the-line vote containing the number 1 once, provided that it had at least 12 boxes numbered with at most two sequence errors in the first 12.

2. Too many voters are voting both above and below the line, causing their vote to either be informal or else potentially prematurely exhaust (more often the latter if they start above the line).  Even if retained as formal some such votes have little relation to what the voter was trying to do.  The ballot paper needs to have a clear and prominent instruction to "Vote above or below the line, not both"

Who Put All Those Hansons In The Senate?

Debate about the role of Senate reform in the election of multiple One Nation Senators has been raging since election night.  While Labor is blaming reform for the One Nation victories, during the 2016 reform debate several Labor MPs actually said that only the Coalition, Labor, the Greens and NXT would be winning seats under the new system.  Yet not only has a minor party outside that list won four seats, but three of the previous non-Green crossbenchers have been returned and Derryn Hinch has joined them.  So much for Labor's claims that the new system would "disenfranchise" a quarter of Australian voters (and there are many more examples of such claims).

Some people did actually claim Senate reform would advantage One Nation before the election (eg Saturday Paper, The Australian, Van Badham).  The argument was that in the absence of parties preference-trading against them (as happened in 1998-2001) all One Nation needed to do was poll a modest primary vote and then they wouldn't need to reach a full quota since the exhaust rate would save them from being chased down by others on preferences.

I pointed out the flaw in the comparison to the old system before the election. Had the 2013 election been a double-dissolution, Pauline Hanson would have won a seat in NSW off 1.22% of the primary vote (a result her party easily exceeded in every state this time around, including in Queensland as surplus over their first quota).  The image of One Nation as a party that was easy to beat under Group Ticket Voting hailed from a very different era in which nearly all votes not for One Nation were for the Coalition, Labor, the Greens and Democrats, making fencing One Nation out on preferences easy.  Since then we have seen a massive rise in the micro-party vote, and not just similar right-wing micros but also some centrist and left-wing ones were sending ATL votes to One Nation.

As for exhaust, results as expected showed that a 3%-ish vote is no guarantee of winning without strong preference flows.  But it turned out that one of One Nation's wins was on raw quota and in the other three cases the party was heavily outperforming parties that might have beaten it to the final seat (overtaking five of them in Queensland).   Moreover, One Nation very nearly crossed quota before the seats were all decided in NSW and WA.  Indeed, far from exhaust stopping One Nation from being caught on preferences as the reform-helps-One-Nation theory claimed, One Nation missed out on a very real chance of beating the Greens in Tasmania because Liberal Party votes exhausted!

By any fair measure, One Nation's primary vote share and strong preferencing performance compared to other micro parties entitles it to the seats it won.  Anyone who disagrees with that has a problem not with the new Senate system, but with preferential and proportional democracy itself.  All the same, the fact that we have four new One Nation Senators and not one (given the votes actually cast) is down not to Senate reform but to the government's strategic decision to force a double-dissolution.  Here's a contingency table of the possible outcomes for One Nation based on the votes cast, depending on the kind of election and whether or not there was Senate reform:


Without knowing what deals might have been done we cannot know for sure how many seats One Nation would have won under the old system - it might even be in theory that at a half-Senate election Hanson herself could have been beaten by some similar right-wing micro-party on a preference snowball.  It's very likely though that PHON would have won at least one at a half-Senate election with a good chance of more, and that they would have won multiple seats at a double dissolution.  Depending on the deals done, their seat haul at a double dissolution under the old system could have even been as high as seven.

The big question is how a discredited relic party that couldn't even get its GVT form in in Victoria in 2013 managed to multiply its vote by ten in South Australia and nearly 17 in Queensland.  My very first assessment of One Nation's chances was way too negative precisely because of their dreadful 2013 result. Some factors in their revival include the demise of Palmer United, a resurgence of anti-Islamic xenophobia, and the publicity given to One Nation by the media on the back of the reform-helps-One-Nation theory (which may have thus become a self-fulfilling prophecy in spite of being false.)  But others over which the left had plenty of control included the nature of the campaign (all very urban, middle-class and modern, nothing to see for the old right or the left-behinds from either major party) and also conspiracy theories about Senate reform.

By opposing reform Labor gave oxygen to claims about its impacts that until then had been the domain of conspiracy cranks and vested interests in the preference-harvesting industry.  The results have shown that these theories had no business in the political mainstream, but Labor insisted on putting them there.  If the increased vote for micros including One Nation (a magnet for conspiracy nuts, after all) was in any way a backlash against Senate reform, then Labor fuelled that backlash.   This puts Bill Shorten in a shaky position from which to blame anyone else.

Below The Line Vote Rates

Below-the-line voting increased to the highest level for at least the last five elections in every state, falling only in the ACT:



The increase was most marked in Tasmania (where BTL rates nearly trebled because of a below-the-line rebellion against party preselections) and New South Wales (where the rate was easily more than double that for the past four elections).

The increase in below-the-line voting resulted in Labor winning a seat in Tasmania that would have otherwise gone to One Nation (see Lisa Singh section below), and caused a different Labor Senator to win than the candidate in line for a seat by ticket order.  That seems to have been the only case of below-the-line voting actually changing a result, but it does show potential for this to someday happen in other states.  Although below-the-line leakage from the Labor ticket in South Australia was greater than Bob Day's victory margin, the proportion of leakage caused by increased below-the-line rates was not.

Lisa Singh's Historic Below-The-Line Win

Lisa Singh's win of a seat from sixth place on the Labor ticket marked the first time a candidate had won a seat out of ticket order since 1953 (at which time party boxes did not exist.)  In the early years of the PR system introduced by the Chifley government, there were a few cases of Tasmanian candidates winning out of party order, usually because their parties had put popular candidates low on the list to try to win more seats.  The 1953 case of Bill Aylett (Labor) was comparable to Lisa Singh's - Aylett was initially deselected, then reendorsed for the bottom position following a death.

Singh's win with 6.12% of the Tasmanian primary vote was achieved entirely on below-the-line primaries and preferences with not a single party vote reaching her.  Under the old system the onerous nature of filling out 58 boxes would have given her no chance of re-election.  The strange thing though is that Labor accidentally benefited by demoting Singh to the so-called "unwinnable" position.  Singh drew votes to the Labor ticket that Labor would never have otherwise received, especially from the Greens.  Without these votes Labor would have had little more than four quotas and would not have won five seats.  The Liberals also could have pulled off the same thing for Richard Colbeck, had their share of all preferences in Tasmania not been so poor.

Singh's victory shows that it was well worth liberalising below-the-line voting as it has given voters an avenue of appeal against party preselections, while still allowing those who want to vote the "party card" to do so easily.   To what extent Singh's win was also caused by the reforms to above the line voting isn't clear.  It might seem that she would also have won off her quota of BTL preferences had only BTL been liberalised (creating a system like the Victorian upper house), but in that case parties from which Singh obtained BTL preferences in the real count may have snowballed ahead of her on ATL group ticket flows.

While this may all seem a specifically Tasmanian thing (because of Tasmania's familiarity with multi-candidate voting) it might someday work in some other state.

Malcolm Roberts And Those 77 Votes

Malcolm Roberts' win from a small One Nation surplus, with a personal primary vote of just 77, has created a lot of misinformed commentary.

Senate reform as such has nothing at all to do with Roberts' victory with such a small BTL vote.  At the only previous double-dissolution under the group ticket system (1987) many major party Senators were elected with very small below-the-line votes - the previously record low 155 for Noel Crichton-Browne (Lib), 168 for Robert Ray (ALP), 208 for Amanda Vanstone (Lib), 201 for Chris Schacht (ALP) and so on.  Had this election been a double dissolution under the old system, it's entirely possible Roberts would have won off exactly the same BTL vote.  He won because voters for other parties directed huge numbers of mainly above-the-line preferences to One Nation.

The main reason for Roberts' new low is that this is the first case we have of a minor party that does not have unusually high BTL rates winning two seats, since party boxes were introduced.

I am not aware of anyone predicting before the election that One Nation might win two seats in Queensland or even mentioning it as a serious chance.  This might be because people thought that if One Nation did reach quota it would be with a very small surplus, and hadn't realised that under the new system a very small surplus can be enough to beat micro-parties.  (This was really the biggest new thing we learned from the results, except that the rates of following how-to-vote cards are much lower than would have been expected.)  The other strange thing with Roberts is that while he was well known before the election as a "climate change denier", revelations of Roberts using "sovereign citizen" language in protest against the carbon tax did not surface in public debate until after the election.

Perhaps it would have made no difference, but there is a question of whether opponents of parties like One Nation are researching their candidates enough in advance to spread public awareness of possible evidence that One Nation could be as crazy a rabble as ever.  In the 2014 Tasmanian state election, the Liberal Party used detailed candidate research (some of it mine, conducted independently of theirs) into the wackiness of the state Palmer United crew in a manner that helped burst a PUP polling bubble and prevent PUP winning any seats in the state.  I also wrote a candidate guide that pointed out some of the pitfalls lurking on the 2016 Tasmanian Senate ballot paper, which may have helped damage Family First's preference flow in the state.  When I look at the reaction to Roberts' eccentricities, it all seems surprisingly post-hoc.

In terms of the mechanics of Roberts' victory, he was competing with other micro-parties (eg Liberal Democrats, Animal Justice, Katters Australian Party, Nick Xenophon Team, Family First and Glenn Lazarus) for the preferences of fellow micro-parties.  The preferences of Labor, Liberal and Green (the parties with the highest how-to-vote card follow rates) were largely irrelevant, and since hardly any voters followed the micro-party cards, they were making their own decisions.

Of these micro-parties, One Nation was the most preferenced among voters for (among others) KAP, GLT, Shooters Fishers and Farmers, and Australian Liberty Alliance - fairly predictable right-wing sources.  But it was also the most preferenced by voters for some of the left micros (Sex/HEMP and Drug Law Reform) and the second most preferenced by NXT voters.  KAP, ALA and Derryn Hinch Justice Party were especially rich preference sources for One Nation. The party's huge profile advantage over the other micros helped it gain massively on preferences and easily haul in the Liberal Democrats (whose preference performance was poor).

Fixed Quota

An issue with the new system as implemented is that vote-values are being locked up unnecessarily in party quotas as votes exhaust out of the system.  In Tasmania, after the exclusion of Richard Colbeck, 6493 votes had exhausted, meaning that any candidate with 24790 votes or more would certainly be elected (the quota was 26090). Lisa Singh had 26094 votes at this point, so was 1304 votes clear of what she needed to certainly win, yet her surplus was only four votes.  Likewise, Liberal candidate David Bushby was 10,291 over quota at this point, but 11,591 over the point at which his election was mathematically certain.

Had there been a progressively reducing quota (as votes exhaust) then Bushby and Singh would have had 1300 votes more apiece at this stage.  The net impact would have been that Bushby, Singh and Bilyk would all have had larger surpluses, and in total this would have caused McKim to win the final seat by more than he did.  So a progressively reducing quota wouldn't have changed the Tasmanian result, but the lack of it came very close to causing One Nation to win unfairly.  Also, even before Colbeck was excluded, there were thousands of vote-values locked up in the quotas for the eight already-elected Senators over and above what those Senators now needed to be sure of election.  An iterative quota, such as in the Meek System, would have freed these vote-values, although as far as I can tell Richard Colbeck would still have been excluded at the same point.

In South Australia, Labor's Anne McEwen was disadvantaged by the fixed quota. She had to compete with the Greens and NXT for left and centrist preferences as they took longer than necessary to be elected, and the top three Labor Senators had vote-values locked up in their quotas beyond what they needed.  As far as I can tell, a progressively reducing quota wouldn't have changed this result, but an iterative quota would have made the final seat so close that I'm not sure which way it would have gone.

There is a fair amount of fear of introducing an iterative quota in Australia because of lack of prior experience of it and impossibility of replication by hand.  However we should at least have a progressively reducing quota to stop some voter intention being wasted.

Magic Expanding Lambie Votes

Nothing to do with Senate reform, but it's worth noting that a very silly example of Inclusive Gregory distortion happened at this election, with some votes cast for the Jacqui Lambie Network in Tasmania increasing in value during the count.  Dean Ashley has found that votes the Liberals received at value 0.064 on the exclusion of Steven Martin were later redistributed at a value of 0.092 as part of the Bushby surplus (when in an ideal system they would have been reduced in value).  The impact on the count was tiny (as the papers were only worth 90 votes total) but this sort of thing still shouldn't be happening.

Verification

Some concerns about verification of the results have been raised by some electoral IT experts and retired electoral administrators.  The AEC outputs files that include details of every formal vote that is included in the count, making it possible for those with formidable IT skills to try to replicate the result.  Grahame Bowland has been testing the preference distributions based on these files and has provisionally found that they match exactly; similar replication attempts succeeded in 2013.

What cannot be so readily verified by the general public is that the AEC's files match what is actually on the ballot papers and that those votes classified as informal were in fact informal.

I spent about 15 hours scrutineering the Tasmanian Senate count and watched the process.  Scanned images of votes came through to human data-processors who would enter in the number they could see on the screen for boxes picked up by the system as containing numbers.  If something strange turned up the operator would look at the whole ballot paper as scanned.  If there was a mismatch between the scan and the human interpretation it was referred to another operator, and there were various chains of command for "raising" difficult votes at a higher level.  There was a separate computer desk where all the really tough cases went, which was easy for scrutineers to watch over.

In theory, the role of scrutineers in the system allows them to watch the human interpretation of votes and challenge any they disagree with.  In practice, this is only practical to a small extent.  By the end of the Tasmanian count the operators had become blisteringly fast, completing manual entry in what I calculated as an average of six seconds per ballot paper - many would disappear from the screen before a scrutineer could see the full preference flow.  The processing of so many votes in a room at once would mean that for every party to watch every vote getting processed, the room would be full of hundreds of scrutineers.  In any case most parties lack the resources to provide more than a handful of scrutineers for the post-count.

What scrutineers can do that is useful under this system is to watch the processing of informal votes, since many borderline calls are likely to crop up there.  They can also keep an eye out for any obvious issues in data entry that might occur on a regular basis.  But they simply can't see everything, and I do agree with the proposal that the matching of ballot papers to recorded preferences (both formal and informal) should be audited to check that preferences were recorded correctly.  For all I know, such a review may already be planned.

Overall the conduct of this election was remarkably smooth compared to pre-election concerns, and even compared to the previous election.

Areas for improvement

The following are some areas I think should be improved prior to future runnings of this system. I may add others.

1. The ballot papers should more prominently indicate that the voter is to vote above or below the line and not both.

2. Savings provisions for below the line votes should be improved to preserve more voter intention.

3. Rotation of parties on ballot papers should be seriously investigated to prevent proximity-preferencing and donkey-vote impacts from affecting close results, at least in double dissolutions.

4. A progressively reducing quota should be introduced.

5. Display of Senate results in the Virtual Tally Room should be improved so that there is one display of votes for each state that only includes those booths that have been fully processed as party votes and votes for specific individual candidates.  (Following and explaining the Tasmanian count was a nightmare because of the lack of such a feature and the releasing of misleadingly low interim  percentages for individual candidates.)

6. Inclusive Gregory for surplus distributions must be replaced by Weighted Inclusive Gregory, or with some other system that does not cause its distortions.

7. New South Wales should, if resources permit, amend its Upper House system to better encourage the use of preferences.

Some voters have asked me for a view on scrapping ATL voting boxes altogether and just going to candidate-only voting.  Based on the record of the system that ran from 1949 to 1983 (which was exactly that, but with compulsory preferences) it seems that cases of voters overturning party preselections were relatively rare - this happened in Tasmania at the first few elections, but not after that.  That was probably a consequence of single ticket orders within each party, so voters tended to vote down the list.  To get something more radical going in terms of individual candidates, it would be necessary to not only scrap above-the-line voting but also introduce Robson Rotation within party columns.  Ideally I think it would be a good thing if Senators had to work hard for preference shares and campaign individually but I think there's a long discussion ahead of us in terms of practicality and voter education before we could get to that point.  So that is not on my shopping list at the moment.

However major parties that think such a system would be bad for them should probably think again.  As we saw with Singh and could have seen with Colbeck, splitting up a party's vote between candidates rather than having it flow down the ticket creates extra paths to possible wins for the big parties.

How To Vote Cards

How to vote cards (and another free plug for David Barry's magnificent efforts in making ATL preference flows explorable) were generally very unsuccessful at controlling voter preferencing choices at this election.

There seems to have been a Palmer United Party how to vote card in Victoria that achieved a 51.6% follow rate (1-6 numbered in exact order), although I haven't seen it.  If that is so, it was the highest in the nation.  Apart from that the highest recorded anywhere was 39.7% for the Coalition in Victoria, and the Coalition averaged about 30% per state in the mainland states.  Labor averaged about 14% and the Greens perhaps 10%; multiple cards make it difficult to estimate the Greens figure correctly.  No other party averaged anything like 10% though some right-wing parties (Rise Up Australia, Australian Christians, Australian Liberty Alliance and Family First, for instance) got around that rate in some states.  Rates for left-wing micros varied from a few percent to fractions of a percent and in one memorable case zero.

A few commenters have suggested that just looking at the strict 1-6 follow rate underestimates the success of cards because they may have been influential on voters without the voters necessarily following them fully.  There's bound to be something in this, but I think it's easy to overestimate it, for the following reasons:

* Where left-wing parties preferenced like-minded but obscure micros second, it wasn't just that voters didn't get to the end of the card; the vast majority didn't even follow it to 2.

* A similar thing was observed with attempts by the Shooters, Fishers and Farmers to preference religious right parties.

* Even for the stronger flows, the gaps between the 1-2 flows between the top two parties and the full 1-6 card flow weren't massive, and could have been largely caused by normal voter intention rather than the influence of cards.

* Once a voter followed a card 1-4, they strongly tended to follow 5 and 6 as well; only a small proportion of voters went their own way from that point.

This is not to say that cards have no impact, and indeed had the Liberals in South Australia not preferenced Bob Day on their card he would probably not have won.  But it does seem that micro-parties that lack the ability to put boots on the ground outside booths also have little ability to impact upon preference flows.  Also, left-leaning voters especially like to make their own decisions.

This may all be partly a response to the novelty of the system and it may be that over future elections we will see HTV cards become more widely followed.

Note: Part 1 has been updated with a proportionality comparison with the 1987 DD.

Saturday, August 6, 2016

Senate Reform Performance Review Part 1

The results of the Senate election are all in and so it is timely to review how the new Senate system performed in full detail.  No system should be expected to perform flawlessly on its first attempt.  However, the new system has generally exceeded the expectations of its most ardent supporters (save any who supported it for the wrong reasons) and made its opponents' pre-election arguments look very silly indeed.

Two major models of reform were canvassed in the Senate reform debate that ran through the last term of parliament.  The original JSCEM model released in 2014 allowed for fully optional preferencing above the line with semi-optional preferencing (six squares for a half-Senate election, twelve for a double dissolution) below the line.  The revised model released in 2016 initially allowed for semi-optional preferencing (1-6) above the line, but essentially maintained compulsory full preferencing if voting below the line.  After many complaints from the psephosphere (and I especially give credit to Michael Maley and Antony Green here) the final version as amended allowed semi-optional preferencing (1-12) below the line as well.  This change, in the end, allowed Tasmanian voters to overturn the contentious demotion of a sitting high-profile Senator.

Monday, August 1, 2016

Gaynor's Clueless Call For Senate Recount

A reader today drew my attention to this piece by Bernard Gaynor claiming apparent irregularities in the Tasmanian Senate count, and calling for a recount, even including a petition to the AEC to that effect.  Gaynor's concern is that as a result of this outcome Australia will have one less Pauline Hanson and one more Sarah Hanson-Young.  Luckily for those who think one of either or both in the Senate is one too many, both these Senators are actually unique. Neither is affected by the outcome in Tasmania.

In this piece I'll examine the basis for Gaynor's call in some detail, but firstly I should point out that the call for a recount is in itself misconceived.  The reason for this is that it comes too late.  The time for calling for a recount in a seat is before the declaration of the poll (S278 Commonwealth Electoral Act).  In 2013, requests for a recount in Western Australia were granted (on appeal) because of a much closer apparent margin (14 votes) at a tipping point during the preference distribution and the declaration of the poll was delayed while the recount was conducted.  I followed the progress of that recount in an article at the time.

The Senate: Button Press Week

This week if all goes according to plan, the buttons will be pressed on the remaining Senate races in ACT, WA, SA (Tuesday), Victoria (Wednesday), Queensland and finally NSW, and the makeup of the Senate should be known.  I'll be posting some comments about these races and the results as time permits, but I'm rather busy this week, so feel free to add thoughts about any of the races in comments.

I haven't had the time I would have liked to analyse these races in detail, and really you have to sample preferences in scrutineering to have a really confident handle on what's going on in a given state under the new Senate system.  You also can't do it on the night, as you have to know what the main races are to know what you need to look for.  I used scrutineering-based modelling to successfully predict the result in Tasmania (though there was very nearly an upset for the final seat) and it surprises me that I have not seen any detailed public attempt at a scrutineering-based model for any other state.

Saturday, July 30, 2016

Eric Abetz And The 2016 Tasmanian Liberal Result

Advance Summary

1. The Tasmanian Liberal result at the 2016 federal election was very poor.

2. Not only was the Liberal primary low, but in the Senate race the Liberals failed to obtain preferences even from "right-wing" parties.

3. Liberal ticket leader Eric Abetz has offered a number of reasons for the party's poor results, some of which are clearly false.

4. In fact the Liberal ticket would have had more chance of winning five seats had a below-the-line campaign for Richard Colbeck been more successful, not less.

5. Although Senator Abetz polarises opinion and was the candidate most frequently placed last on Senate ballots, no conclusions about his popularity can be drawn directly from this result.

6. Historically, there is evidence that the Liberal Party performs worse in Tasmania (relative to the nation) when Senator Abetz is on top of the Senate ballot, but the difference is not quite statistically significant and could be caused by other factors.

7. There is no evidence in historic results that having Abetz on top of the ticket is an asset to the party's fortunes.

8. Tasmania is a historically Labor-leaning state, but this does not alone explain the poor result at this election.

----------------------------------------------------------------------------------------------------------------------


Wednesday, July 27, 2016

Tasmania Senate 2016 Button Press And Analysis

Result

Order 1 Abetz (Lib) 2 Urquhart (ALP) 3 Whish-Wilson (Green) 4 Lambie (JLN) 5 Parry (Lib) 6 Polley (ALP) 7 Duniam (Lib) 8 Brown (ALP) 9 Bushby (Lib) 10 Singh (ALP) 11 Bilyk (ALP) 12 McKim (Green) 
Colbeck (Lib) was passed by McKim and McCulloch and excluded
5 Labor 4 Liberal 2 Green 1 Lambie

I would like to thank all the AEC staff in Hobart for all their help with information on the count and congratulate them on their efficient and punctual delivery of the Tasmanian outcomes.