"North Wales Police Authority has not received this correspondence. Get in touch and I will be happy to look into what went wrong. Our contact details are as set out on our website.
We have not misrepresented anything. The basis of the survey was made explicit: working with the other Police Authorities in Wales we have given an opportunity for the public to express a view. The timetable for consultation was very short, but this is the fault of the Home Office. We submitted interim results by the Home Office deadline, but then we re-opened the survey so that others can have an opportunity. We have now closed the survey and South Wales Police Authority will be analysing the results so that we can publish them.
There is a difference between consultation and research. “Opinion polls” or “Representative Surveys” use sampling techniques to find out what the general public think. Statistical theory suggests that by setting quotas for gender, age, work status, social class and spreading those selected for interview randomly across an area, we are able to question about 1,000 people across an area and find out what they think with a margin of error up to 2 or 3 percentage points either way.
Consultation is when we invite people to express a view. By its very nature, consultation attracts more responses from those people who take an interest in a particular subject. Those responding will not necessarily be representative of the wider public.
We never suggested that this was an opinion poll. The press release was explicit: we have invited people to express a view on the Government's proposals and offered to pass this on to Government. The all-Wales analysis undertaken by South Wales Police Authority does not twist the results - we said they showed that "According to the survey, 41 per cent were opposed to the introduction of a commissioner while less than 30 per cent were in favour". You have misinterpreted the results and said that 45 percent are either in favour or somewhat in favour. It would be equally valid to say that 56% of respondents were either against or somewhat against. We have given full information on the respondents including where they live, gender and ethnicity. The information given on ethnicity is the same as the information given on gender.
North Wales Police Authority has been clear about its own views on the Government's proposals. We are not opposed, but we do have serious concerns. We recognise that there is room for improvement and we are open to change, but we believe that changes should be focused on ensuring high quality, accountable policing, with the involvement of a wide range of stakeholders in agreeing policing priorities and agree that this is best achieved through governance arrangements which promote a consensual, involving approach with appropriate safeguards. See www.nwalespa.org for full details."
Whereas I thank Mr Michael for getting in touch via this blog, and am happy to accept his description of the poll as a 'consultation' rather than a 'representative survey', I continue to take issue with this part of his reply:
"The all-Wales analysis undertaken by South Wales Police Authority does not twist the results - we said they showed that "According to the survey, 41 per cent were opposed to the introduction of a commissioner while less than 30 per cent were in favour". You have misinterpreted the results and said that 45 percent are either in favour or somewhat in favour. It would be equally valid to say that 56% of respondents were either against or somewhat against."
Mr Michael's argument entirely rests on how you interpret the following results:
My interpretation is that the number of respondents either "in favour" (29 percent) or "somewhat in favour" (16 percent) comes to a total of 45 percent -- a larger number than those against (41 percent). However, according to Mr Michael, the pinkish wedge in the top left labeled "somewhat" does not mean "somewhat in favour" but actually "neither in favour or against". Accordingly Mr Michael asserts that it is equally valid to say that the total number of respondents against the proposal is 57 percent, including "against" (41 percent) and "somewhat against" (16 percent).
I'm sorry, Mr Michael, but this is clearly not the case. Even your own Executive Summary of the survey (page 2) states that the 16 percent figure represents "somewhat in favour":
If, as you assert, I have "misinterpreted the results", then I'm afraid that the author of your own report has also "misinterpreted the results".
It seems clear to me therefore that my interpretation is the correct one and more respondents favour the proposal for elected police commissioners than are against it -- in direct contrast to how the results are presented. I will leave it to readers to decide whether the SWPA and NWPA have tried to "twist" the results of the survey to create a false impression of the level of support for the Police Commissioner proposals in Wales.
I also note, Mr Michael, that your reply does not address the fact that there was only 82 respondents to the poll from North Wales -- yet following a briefing by the NWPA this is how the Daily Post reported the interim survey results on 18th September:
"OPPOSITION is growing to the idea of having an elected commissioner in charge of police in North Wales. The interim results of a survey show a majority of people are against the proposal which is part of a package of reforms being put forward by the Government"
Can you kindly explain how a survey of just 82 people throughout North Wales -- which shows even according to your own executive summary that more people are either "in favour" or "somewhat in favour" than against -- gets written up as "OPPOSITION is growing to the idea of having an elected commissioner in charge of police in North Wales"?
28 comments:
get in there druid. They just keep digging the hole don't they? Congrats on the blog aswell.
Dazzler
I too tried to understand how Mr Michael arrived at his figures.
Clearly 'somewhat' refers to 'somewhat in favour' given the context created by the question.
The only other way I could see an interpretation of a no vote at above 50% was by counting the 'Don't know', & 'Don't care' as part of the No vote.
This would bring the figures up to 55% saying no, but would be a blatent distortion of the figures.
Either way it demonstrates a disturbing lack of mathematical ability and would probably not merit a pass in GCSE maths.
The clarity of the grammar isn't much better.
"Statistical theory suggests that by setting quotas for gender, age, work status, social class and spreading those selected for interview randomly across an area, we are able to question about 1,000people across an area and find out what they think with a margin of error up to 2 or 3 percentage points either way".
Surely he means 'randomly selected' and not 'randomly spreading those selected for interviews' as the text suggests.
I wonder what 'null-hypothesis' was used when calculating the statistical significance (p-value)to arrive at the 2-3% margin of error?
If this is an example of the caliber of unelected representatives then it can only lend weight to the 'elected commissioner' tendency.
Druid and company,
Sorry for the late arrival at this development.
Firstly, I'm going to send the report to a fully-qualified academic statistician for comment. It'll take a few days, but I'll pass on the results.
I think the only comment I can make about the NWPA's interpretation of the result of the 'survey' is that it is astonishing and extremely troubling. As a previous poster notes, the question is "are you in favour of...", to which possible answers as "Yes", "Somewhat", "No" etc. The scale clearly slides towards the negative, so saying now that "Somewhat" refers to "Somewhat Against" is blatant nonsense. If this was the wording offered to respondents (which I know it was), then we can conclude the respondents were being entirely misled in what they were being asked, making the results even more meaningless than they already are.
Secondly, NWPA says this was not a poll. We can argue about semantics and ask what the purpose of the 'survey' was. Much the same as conducting a poll, I would say. Also, if this truly were not in any sense a poll, why does NWPA then try to cite polling methodology to justify the way they undertook it? The aim was to get a picture of what people thought. The proper way to do this is an independent poll. Not a self-selecting, non-indepedent survey.
Yes, 82 respondents from north Wales. Just 7 from Anglesey. To say this is a representative sample for any purpose is ridiculous. Are they claiming to know people in Cardiff and the Valleys have the same views as those on Anglesey based on such data? If so, how do they know? The sample size is far, far, far too small to be able to tell. Yet the result is pooled, masking any regional differences. Are they suggesting we should have only one, all-Wales PA if this is the way it works? I suspect not!
And finally, it's good that NWPA have responded - after my letter and subsequent expression of concern to SWPA. Also good to know this blog is getting to the heart of local issues and getting supposedly responsive and accountable bodies to, erm, be responsive and accountable.
I should also add for Mr. Michael's benefit that I did not state the PAs have 'twisted' the results. As is evident, I point out the flaws in the way they do their survey/poll, and how this will affect the outcome. I do say there is bias evident in the wording of their report, but then, he admits to this.
But the effect of all this is, indeed, to twist the results.
The old plod gets it wrong...HAHA!
Agree with the detail of your post Druid, but I do not believe that you can blame Mr Micheal for the actions of the Daily Post.
The press use the "opposition is growing" card if they receive 1 person who comments against a proposal.
Anon 20.57
Indeed. One has to blame Mr Michael's for Mr Michael's post, coming as it does a month after the Daily Post's article.
Anon 20:57 - with respect I'm afraid I have to disagree. Although I accept that the press like negative stories more than positive ones, in this case it's quite clear that the way the Daily Post wrote it up was exactly how the NWPA wanted them to, and how they were briefed.
If the survey/poll/consultation were to exhibit a sense of balance then the categories for agreement or disagreement with the proposal would include 'somewhat against'
As the NWPA argument stands I can't see what significance 'somewhat agree' brings to the whole, apart from obfuscation of the interpretation.
If there were only 7 respondents from Anglesey and I was one of them, how many 'readers' on here also responded? That would be no more than 6 which I find hard to believe.
Red Flag - remember that the survey in question was an 'interim' survey. The results on which it was based were collected sometime before around the 9th September (see this post: http://goo.gl/1JgX). The survey was then re-opened some weeks later. I don't know if you submitted your views before 9th Sept or not -- but according to the report on the interim survey (i.e. ending 9th Sept), only 7 respondents identified themselves as from Anglesey.
"the categories for agreement or disagreement with the proposal would include 'somewhat against'"
I agree with the general view there, but according to my dictionary, which agrees with common sense, the term 'somewhat' is not properly used with a negative term. 'Are you cold?' 'Yes, I'm somewhat cold'. It's not correct to say 'I'm somewhat not cold', is it? Especially when your written report confirms what you meant to say!
But, in case I'm accused of being pedantic, let's not get distracted by too much detail. Let's ask: why did the police authorities want to do this survey? Was it to gather valid, representative views from the public that they could then rely on and announce to the politicians and public via their report and media? Yes, it was.
If not, the authorities surely aren't saying they deliberately set out to gather what is almost certainly unreliable data using certainly flawed methods with a view to misleading politicians and public alike. Are they?
Remarkably, I set out with grave reservations about an elected commissioner. After this debacle, I now see the unelected police authorities are not very accountable, not responsive unless kicked, and not able to be honest and say 'we messed this up in our rushed opposition to change' when clearly shown to be in the wrong.
That is not accountable representation of the people.
From Dictionary.com (check others, they define it in much the same way):
Survey.
>a formal or official examination of the particulars of something, made in order to ascertain condition, character, etc.
>a statement or description embodying the result of this: They presented their survey to the board of directors.
>a sampling, or partial collection, of facts, figures, or opinions taken and used to approximate or indicate what a complete collection and analysis might reveal: The survey showed the percentage of the population that planned to vote.
Sounds a bit like 'poll' to me:
>a sampling or collection of opinions on a subject, taken from either a selected or a random group of persons, as for the purpose of analysis.
Mr. Michael says:
"Those responding will not necessarily be representative of the wider public"
So why did they report in their own document - then repeated by the Daily Post (so it wasn't the paper getting it wrong at all) - that "We" (that being the public" 'don't want an elected commissioner'?
Did the NWPA and SWPA mean to say that they were happy to knowingly put forward unrepresentative views as the basis for moving forward?
This survey and response is spherical: it's rubbish no matter how you look at it.
"Opinion polls” or “Representative Surveys” use sampling techniques to find out what the general public think"
So, do the NWPA and SWPA think that only a certain group of society are worth listening to? And if so, who are they, and why are their views any more valid than the views of the general public they admit thy aren't seeking in using this 'survey'?
Did they set out to obtain views that supported and gave credibility to their own? It certainly seems so.
TGC: Very good observation about the 'slightly not agree' - quite right. I must have been thinking of 'Likert' scale questionaires (on a scale of 1-5, etc.)
I too have gone from a position of neutrality, (don't know), to 'slightly agree' purely on the strength of the statistical manipulation being attempted.
To Anon 19:27 yesterday:
"The old Plod gets it wrong"
The NWPA are not 'the old plod' as they are an independent body, or so we are led to believe.
As set out in the report published on our website, the question you are talking about (7) was “are you in favour of a directly elected Police and Crime Commissioner”. The options given and the responses were Yes 29%, Somewhat 16%, No 41%, Don’t know 13%, Don’t care 1%. Our press release summarised this as 41% against and 29% in favour, which is fair. If the people who said “somewhat” wanted their position recorded as yes or no then we can only assume that they would have ticked those boxes instead. I certainly wasn’t presenting the results as meaning “57% are against” – I am saying that to do so would be as silly as arguing that 45% are in favour.
The purpose of the questionnaire? To give people an opportunity to express their views, which we could take into account in formulating our responses and communicate on to MPs.
The purpose of the media release? To publicise the survey and encourage people to participate. If it got you and some of your correspondents interested, then we achieved our purpose.
As I said before, North Wales Police Authority is not opposed to the idea of a directly elected commissioner but we have serious concerns about the lack of checks and balances in the Government’s current proposals. We are engaging positively in seeking to improve these proposals and ensure it works effectively. If you or any of your contributors have any points to make on this substantive issue, get in touch by emailing us at PoliceAuthorityWebSite@north-wales.police.uk
Mr Michael
"Our press release summarised this as 41% against and 29% in favour, which is fair. If the people who said “somewhat” wanted their position recorded as yes or no then we can only assume that they would have ticked those boxes instead."
I'm afraid that when you find yourself in a hole, you should stop digging. In the context of the question “are you in favour of a directly elected Police and Crime Commissioner?”, "somewhat" clearly means "somewhat [in favour]". Therefore you have effectively split the "yes" vote into two groups and then simply disregarded one group ("somewhat", or 16% of respondents) so as to arrive at the result you want. The more you ask us to believe that white is in fact black, the more you erode our trust in the accountability of the NWPA and thus defeat your own purpose.
Oh dear. Where to start? On a positive. That Mr. Michael is engaging (so far as we can ascertain it is in fact him on the blog!) with the public in an open way. This is to be encouraged, as only open debate will suffice in a pluralist democracy. So, 'well done' for that, if nothing else.
As for the facts, Druid is far more eloquent and to the point than I could ever be. There is no point arguing about the 'somewhat' vote, when your own report, as extracted in the main blog post, clearly says the NWPA also intended - and took - to mean 'somewhat in favour'.
As for the purpose of the 'questionnaire' (now not a survey, poll, or consultation, apparently), well Mr. Michael says is was to:
"give people an opportunity to express their views, which we could take into account in formulating our responses and communicate on to MPs."
Which entirely confirms, given the flawed way the survey was done, the view that the NWPA and SWPA either did not care or else did not pay sufficient attention to ensuring that the data gathered was representative and reliable. Those two features are rather crucial when the results are emblazoned on newspaper straps and reports to influence MPs.
It seems we have reached an impasse. The responses are on the record. All I will say is that Mr. Michael should be very wary of labelling any contributor of this blog as 'silly'. Time and again, such remarks have come back to bite those making them. I'll start with my own contact with the Home Office, as the Assembly advised is appropriate...
"If the people who said “somewhat” wanted their position recorded as yes or no then we can only assume that they would have ticked those boxes instead."
So 'somewhat in favour', as you record in the report is what it did mean and that any sensible person would take it to mean, didn't mean that at all, then?
If you don't know if it meant 'somewhat in favour' or 'somewhat not in favour', what was the purpose, precisely, of having it as an option?
Can I politely suggest to the authority that polling people's views and getting a proper answer is much more difficult than getting someone to stick up a questionnaire on the internet. That's why we have specialist polling organisations with extremely clever statisticians running them, and why the police authorities would have been far better advised to use them.
Here are a few choice extracts with comment from the NWPA/SWPA Report:
PA: "A survey was carried out jointly by the four Police Authorities in Wales,"
Comment: So it is a survey, not just a consultation?
PA: "a survey was published asking for public opinions on the changes"
Comment: Again, it's a survey, and one would hope that only a meaningful, representative
and reliable survey would be accepted by the authorities and the government.
PA: "In total, 884 people took part in the survey. This is an excellent level of response when
considering it was only available for four weeks"
Comment: It's significantly (in scientific terms) less than 1000, the minimum accepted poll number IF the poll is representative. But the NWPA readily admits it wasn't representative, so all we can conclude is that the survey/poll, call it what you will, is not reliable as an indicator of what the population in general think. There is no point arguing about the actual numbers coming out of the survey, because without being representative, it doesn't actually mean anything! There is no reason to think the response rate is 'excellent'. Proper polls are conducted within a carefully-selected time period, usually gathering all the data in one day so that the variation of opinion with time (arising from news
coverage and so on) doesn't confuse the result. Having taken place over 4 weeks, there was enough time, for example, for people to pick-up, absorb and be influenced by blog posts such as this.
PA: "However, with the exception of people aged under 25 and 75 and over, the age range of respondents was fair. Pleasingly, 3.1% of the respondents were from a minority ethnic background."
Comment: So, as an exercise in the round, the 'age range' was not fair. Why is a 3.1% rate of ethnic minorty responses
'pleasing'? The 2001 census - our latest offfical 'snapshot' says we had 2.1% ethnic backgrounds in Wales. Which
means that the survey has likely over-represented one group of people significantly. That is not fair (and no, this isn't a racist argument for those unable to follow complex debates). It proves our contention - that the survey was flawed from the outset.
PA: "Knowledge of Representatives
75% Chief Constable
59% Local Councillor
68% MP
"49% knew who their Police Authority Member was""
Comment: Is it really the case that more people know who the Chief Constable is (I assume the question was about the name, not the position) than their local Councillor and MP? It seems
surprising, but I'm willing to allow a proper poll to prove me wrong.
PA: "Those who stated that a PCC would not increase accountability in the police were more likely to have a better understanding of the role of the current Police Authority."
Comment: Which might suggest that the survey was attracting a non-representative group of respondents. What further questions to allow for this were asked?
PA: "Younger age groups
felt the introduction would not lead to a better police service (83% of 16-24 year olds,
78% of 25-34 year olds, 77% of 35-44 year olds). However, older age groups felt the introduction would improve the police service (61% of 65-74 year olds, 70% of 75 or above year olds)."
Comment: The sentence is constructed in such a way that it seems to suggest the older groups, because they had the same view as the PAs, were actually the ones who were 'correct' in those views. Is the 83% figure
to be dismissed as misguided?
Sorry - the last paragraph is of course wrong - it was included from a scratch pad in error. My apologies.
From the NWPA web site:
"Three of the 17 Members are female and 5 of the Members are fluent Welsh speakers. One Member is aged between 40-50, 6 Members between 50-60 and 10 Members are over 60 years of age."
This immediately gives rise to concerns over how well it reflects the population at large, and conflicts with the NWPAs own statement on equality:
"North Wales Police Authority has both a legal and moral duty to promote equality and fairness ...internally amongst staff, members and volunteers or externally in its dealings with the communities of North Wales."
So, on the data, rather less than equal, then? Perhaps we should be worried about 'institutional unfairness and non-representativeness'?
On a slightly personal note: I note there are 3 JP members. The last answer I had from a JP (C'Fon Criminal Justice Centre Open Day) about the large difference (around 20%, current statistics are not easy to find) between conviction rates in Magistrates' Courts and Crown Courts was that "juries don't understand the law, so people get away with more upstairs". That is a very curious reply, because the rule of law expects that it is a fair trial that people get in court, not appearing in order to "get away" with things. It tends to confirm that Magistrates become jaundiced over time. Maybe this has crossed over to the Police Authorities?
Mr. Michael,
You asked that I or anyone "get in touch by emailing us at policeAuthorityWebSite@north-wales.police.uk"
I sent a letter, which you say was not received. That may well be true, so a pity. I sent an e-mail to SWPA, who say their Staff Officer has sent a reply, even though I've not received such a reply, nor sure why a Staff Officer would get involved; if I sent it to the wrong place, then surely the letter should have been forwarded to the PA and the police staff have no more to do with it?
Neither have I received a reply from SWPA; I was told early last week that a reply was "being prepared". But only after I pointed out that a blog was covering the story, and after a month since the letter was sent. Despite another week going by, I've still had no reply.
Do you think it fair to say that, under the circumstances and all the evidence, that it looks to us as though the PAs don't think they should respond to reasonable points put to them by the public? It would be a little surprising and insensitive if you did not.
"49% knew who their Police Authority Member was""
Could one of the 49% enlighten me as to who is the PA member for Ynys MOn, as I don't have a clue?
"as I don't have a clue?"
Nor me! Most of us have an active interest in politics on here, but I very much doubt if half of us know the names of our PA member. It seems laughable.
I think you will find it's Cllr Peter S Rogers
"I think you will find it's Cllr Peter S Rogers"
Thanks. But we didn't really want to know. We were wondering how many people knew this at the time of the Police Authority asking. They say 49%. Hands up who's surprised at that number?
UPDATE: SWPA said they were preparing a response to my critique two weeks ago. I've still not received a response. It looks like they make a habit of misleading the public.
Post a Comment