Alex Hern explains why the phrasing of the question to be asked in the Scottish referendum should be concerning for anyone who wants a fair vote.
On Wednesday afternoon, Alex Salmond announced that the question the SNP would put to the Scottish people on 24 June 2014, the 700th anniversary of the battle of Bannockburn, would be:
Do you agree that Scotland should be an independent country?
As many picked up on, this question is may be “simple, straightforward and clear”, but it’s not quite as fair as Salmond suggests it is. By phrasing the question as “do you agree…” rather than the more neutral options of “do you agree or disagree…” or simply using “should”, there is likely to be a small but significant increase in the amount of people voting yes.
Time and time again, textbooks on survey construction warn against phrasing questions the way the SNP have, because it will lead to biased responses.
The CDC lists leading questions in its catalog of survey biases (pdf):
Different wording of the same question can guide or direct respondents toward a different answer… The preferred phrasing is, “Do you agree or disagree that . . . ?”
Taylor-Powell (1996) writes:
Biased questions influence people to respond in a way that does not accurately reflect their positions. A question can be biased in several ways:
(1) when it implies that the respondent should be engaged in a particular behaviour;
(2) when the response categories are unequal or loaded in one direction;
(3) when words with strong positive or negative emotional appeal are used, such as “freedom,” “equality,” “boss,” “bureaucratic,” etc.
Here are some examples of biased questions:…
3. Do you agree that funding for Extension in your county should be increased?…
This is a leading question. A better question would state:
Do you agree or disagree that Extension funding should be increased?
And Janes (1999) warns questioners against leading questions, saying:
Let them tell you what they think.
Salmond is not the first referendum writer to phrase his question thus, however.
The wonderfully obfuscated question asked of Quebec in 1995 read:
Do you agree that Quebec should become sovereign after having made a formal offer to Canada for a new economic and political partnership within the scope of the bill respecting the future of Quebec and of the agreement signed on June 12, 1995?
While the question which decided Malta’s accession to the EU was:
Do you agree that Malta should become a member of the European Union in the enlargement that is to take place on 1 May 2004?
What else do these referendum questions have in common? They were all written by the group campaigning for a yes vote. The question may still be simple, straightforward and clear, but it’s looking less and less likely that it’s fair.
See also:
• Progressives need a positive vision for Scotland – Ed Jacobs, January 26th 2012
• Hughes makes the case for an English Parliament as Salmond faces fresh scrutiny – Ed Jacobs, January 23rd 2012
• Win or lose, Scottish independence referendum heralds a revolution in UK politics – Ed Jacobs, January 16th 2012
• SNP: Cam’s “economic uncertainty” argument is nonsense; we’ll stick to our timetable – Humza Yousaf MSP, January 9th 2012
• Referendum on Scottish independence? Time to ‘bring it on’ – Ed Jacobs, June 5th 2011
25 Responses to “Salmond’s Scottish referendum is a textbook example of a leading question”
Cate Vallis
Salmond’s Scottish referendum is a textbook example of a leading question, writes @AlexHern: http://t.co/YLXNpv7r
Even children are taught Salmond “Do you agree” question is biased | Political Scrapbook
[…] agree that Scotland should be an independent country?”Technical literature on survey design is clear that questions phrased in this way result in a “small but significant increase in the amount […]
Charles Addison
“Salmond is not the first referendum writer to phrase his question thus”
The Devolution Referendum (set by Westminster) asked two questions:
”Do you agree that there should be a Scottish Parliament?” Yes/No.
”Do you agree that a Scottish Parliament should have tax raising powers?” Yes/No
But suddenly, this is no longer acceptable as a model! Why didn’t you say earlier…
There is a consultation (well, two consultations) going on so plenty of time to thrash this out. Submit your suggested wording.
Duncan Hothersall
"Salmond’s Scottish referendum is a textbook example of a leading question" http://t.co/q0Icg5XQ #indyref
George Eaton
Great stuff from @leftfootfwd and @PSbook on Salmond's loaded referendum question http://t.co/5OjpcQun and http://t.co/Jy0inWYz