Alex Hern explains why the phrasing of the question to be asked in the Scottish referendum should be concerning for anyone who wants a fair vote.
On Wednesday afternoon, Alex Salmond announced that the question the SNP would put to the Scottish people on 24 June 2014, the 700th anniversary of the battle of Bannockburn, would be:
Do you agree that Scotland should be an independent country?
As many picked up on, this question is may be “simple, straightforward and clear”, but it’s not quite as fair as Salmond suggests it is. By phrasing the question as “do you agree…” rather than the more neutral options of “do you agree or disagree…” or simply using “should”, there is likely to be a small but significant increase in the amount of people voting yes.
Time and time again, textbooks on survey construction warn against phrasing questions the way the SNP have, because it will lead to biased responses.
The CDC lists leading questions in its catalog of survey biases (pdf):
Different wording of the same question can guide or direct respondents toward a different answer… The preferred phrasing is, “Do you agree or disagree that . . . ?”
Taylor-Powell (1996) writes:
Biased questions influence people to respond in a way that does not accurately reflect their positions. A question can be biased in several ways:
(1) when it implies that the respondent should be engaged in a particular behaviour;
(2) when the response categories are unequal or loaded in one direction;
(3) when words with strong positive or negative emotional appeal are used, such as “freedom,” “equality,” “boss,” “bureaucratic,” etc.
Here are some examples of biased questions:…
3. Do you agree that funding for Extension in your county should be increased?…
This is a leading question. A better question would state:
Do you agree or disagree that Extension funding should be increased?
And Janes (1999) warns questioners against leading questions, saying:
Let them tell you what they think.
Salmond is not the first referendum writer to phrase his question thus, however.
The wonderfully obfuscated question asked of Quebec in 1995 read:
Do you agree that Quebec should become sovereign after having made a formal offer to Canada for a new economic and political partnership within the scope of the bill respecting the future of Quebec and of the agreement signed on June 12, 1995?
While the question which decided Malta’s accession to the EU was:
Do you agree that Malta should become a member of the European Union in the enlargement that is to take place on 1 May 2004?
What else do these referendum questions have in common? They were all written by the group campaigning for a yes vote. The question may still be simple, straightforward and clear, but it’s looking less and less likely that it’s fair.
See also:
• Progressives need a positive vision for Scotland – Ed Jacobs, January 26th 2012
• Hughes makes the case for an English Parliament as Salmond faces fresh scrutiny – Ed Jacobs, January 23rd 2012
• Win or lose, Scottish independence referendum heralds a revolution in UK politics – Ed Jacobs, January 16th 2012
• SNP: Cam’s “economic uncertainty” argument is nonsense; we’ll stick to our timetable – Humza Yousaf MSP, January 9th 2012
• Referendum on Scottish independence? Time to ‘bring it on’ – Ed Jacobs, June 5th 2011
25 Responses to “Salmond’s Scottish referendum is a textbook example of a leading question”
Anonymous
“Do you agree or disagree that Scotland should be an independent country?”
If that question were put instead of the one suggested by the Scottish Government, would a Yes vote mean that one agreed or disagreed? Voters would arguably be unsure. An unclear question would give an unclear answer, which would be disputed.
Incidentally, even at this early stage the most recent opinion polling is showing Scottish support for independence to be moving up to the danger zone in which a majority becomes possible, while it is apparent that English opinion is against further substantial powers for the Scottish Parliament within the UK and would rather that Scotland left.
The anglo-union would appear to be done for, which is the conclusion which other countries are beginning to come to.
John Ruddy
Thats why the Electoral Commission should decide on the question. They are the experts in looking at questions for reasons of ease of understanding, impartiality etc.
Interesting YouGov poll in todays Daily Record – the SG’s preferred wording gives independence a 10% advantage.
Alastair MacIver
"Salmond’s Scottish referendum is a textbook example of a leading question" http://t.co/q0Icg5XQ #indyref
Newsbot9
I’m sure you’re just fine with secret funding in referendums as well.
Because, after all, it doesn’t make a mockery of democracy or anything, no. Or rather, your hatred of democracy is showing.
Newsbot9
So two rights make a wrong? Keep up the attack on democracy!