SiteMap •  Links • Search

 

ELECTION POLLS AND THE OCTOPUS ORACLE OF OBERHAUSEN

While football still has to catch the fancy of Pinoy sports fans, the world watched the final stages of the 2010 FIFA World Cup last  week in South Africa with such great passion. We can just imagine the Oles reverberating all  over Spain right now and only God knows for how long confetti will rain in Spain to celebrate their very first World Cup! Congratulations Spain! 

We also know now what a vuvuzela does! And we are sure you have heard about Paul, the  German  octopus oracle of Oberhausen who has become such  a worldwide celebrity?   Paul had  correctly picked the winner of all five German World Cup matches leading into the semifinals. To the horror of the Germans, Paul picked  Spain to beat  Germany in the semifinals, and he of course turned out to be right!. Oh how the Germans wanted to turn prescient  Paul into adobong pusit! What one gets these days for being honest! For making good forecasts! Probably not wanting to be cooked just yet, in the battle for third place Paul picked Germany over Uruguay and Paul was again right! In the finals, he picked Spain over the Netherlands and the  Spaniards  shouted lustily Vamos  Paul!  The rest as they say, is history! We do not know what statistical techniques Paul uses but for sure, his is a much much better record than that  of some of our madame manghuhula!   Maybe the Dutch should have filed a case with the International Court of Justice  in the Hague to ban Paul’s forecasts? Just like what some losing  candidates wished to do during our May 2010 elections!  Incidentally, even if we do not get to watch Ronaldo and Ramos and Casillas and Iniesta as much as we do Nadal and Federer and Serena and Maria and Venus, we thought waking up at 2 this morning and watching  the finals was a, well, a glorious experience! You agree?

Talking about forecasts, since almost two months have passed after our very successful shift to automated elections,  maybe we can  hope that  we  have regained our fair sense of objectivity in looking at opinion polls! Understandably, people take different positions in the never ending controversy whether election polls should be banned. Actually,  Statistically Speaking has written ² a few times about election polls and pollsters, including practices in other countries.  In fact,  on May 5, 2001 our Supreme Court ruled in favor of election polls. ³  The Supreme Court commented that, “to sustain the ban on survey results would sanction the censorship of all speaking by candidates in an election, on the ground that the usual bombasts and hyperbolic claims made during the campaign can confuse the voters and thus debase the electoral process.”  It thus decided that §5.4 of RA. No.9006 (Fair Election Act),  which provides, “Surveys affecting national candidates shall not be published fifteen (15) days before an election and surveys affecting local candidates shall not be published seven (7) days before an election.” constitutes “ an unconstitutional abridgement of freedom of speech, expression, and the press.” In the same light, the Foundation for Information, an Amsterdam-based organization for the protection of the right to obtain and make use of information avers “ the freedom to carry out and publish the results of public opinion polls is a fundamental democratic right. It is just as fundamental as the freedom of the media to publish comments and opinions on the issues of the day." Add to this what the Council of Europe says ⁴ on measures concerning the treatment of opinion polls by  both the print and broadcast media:

            "Regulatory or self-regulatory frameworks should ensure that the media, when disseminating the results of opinion polls, provide the public with sufficient information to make a judgment on the value of the polls. Such information could, in particular:

- name the political party or other organization or person which commissioned and paid for the poll;

- identify the organization conducting the poll and the methodology employed;

- indicate the sample and margin of error of the poll; and

- indicate the date and/or period when the poll was conducted.

All other matters concerning the way in which the media present the results of opinion polls should be decided by the media themselves.

Any restriction by member States forbidding the publication/broadcasting of opinion polls (on voting intentions) on voting day or a number of days before the election should comply with Article 10 of the European Convention on Human Rights ⁵, as interpreted by the European Court of Human Rights.

Similarly, in respect of exit polls, member States may consider prohibiting reporting by the media on the results of such polls until all polling stations in the country have closed."

Those who oppose election polls contend that they are being used for trending and that they  influence voters and therefore are detrimental to candidates who cannot afford to conduct or commission them. In the literature on election polls, it is  well recognized that election polls can in fact influence voters in various ways such as  the bandwagon effect, the underdog effect, the demotivating effect, the motivating effect or the free will effect ⁶.  In this regard, some countries have banned the conduct of election polls within a certain period from election day. Others ban the release of the results of exit polls  until after precincts have closed.

On the other hand, there are those who oppose bans on election polls, notably the pollsters themselves. They invoke Article 19 of the Universal Declaration of Human Rights ⁷ which states, "Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers." In addition, a paper entitled "Who’s Afraid of Election Polls? Normative and Empirical Arguments for the Freedom of Pre-Election Surveys” ⁸ claims that "the effects of election polls are  minimal and can be seen as completely harmless." The author adds that "poll results prior to the election tend to get drowned out amidst the many other statements on the outcome of the election and that the significance of election polls for the average citizen is overestimated by both politicians and social scientists." 

Exit polls are usually conducted as voters exit the poll precincts and the results are released after all the polling places had closed. But different pollsters have different practices. In the 2008 U.S. presidential elections, the exit polls were able to predict that Obama had won handily before 11 pm of the election day. In the 2009 presidential elections in Indonesia, the exit polls also correctly predicted that Yudhoyono had won but they released the results even before the closing of the polling places. In the Philippines, SWS released its exit polls results for the 2010 elections on 17 May while Pulse Asia released theirs on 13 May.  Maybe time for SWS and Pulse Asia to automate too?

For sure,  the debate whether to allow or to ban election polls and the publication of their results is far from settled. But as Statistically Speaking has previously cited ⁹, in a 1996 survey on election poll policies and practices of 78 countries, 30 countries had some restrictions, 47 countries had absolutely no restrictions and 1 country did not respond.   In the case of the Philippines, our Supreme Court has spoken in favor of election polls. So why do  some of us  continue to disrespect decisions penned by the highest court of the land?

During the campaign period for the May 2010 elections, it was amazing how the same complaints about opinion polls got repeated many times over. As we have said in the past, we wouldn’t be surprised though if in fact, the “complainants” actually knew about and understood the science of election polling. But of course, there are illegitimate pollsters and the public must guard against them. In this regard, responsible journalism can contribute a lot by publishing results only of surveys that pass standards of quality and professionalism. The media must completely ignore pollsters who are not transparent and are not willing to share methodological details that can be scrutinized by trained statisticians.  Is this censorship? We do not think so.

We are not lawyers and do not have expertise in interpreting legal provisions but as a statistician, we are all for responsible election polling and responsible media reporting of election poll results. Meaning that legitimate and transparent pollsters who have proven track records should not be harassed when they do their work professionally and that the media should never publish results of polls conducted by fly-by-night pollsters with questionable credentials and unprofessional practices.

We submit that If only we would seriously study the history of opinion polls, including the experiences and practices in different countries, we would not have a hard time agreeing on two things: first, that banning opinion polls has no place in a democracy and second, that legitimate pollsters are quite good in predicting election results.

Having previously dwelt on the theoretical, statistical basis why politicians and the public in general had better believe the results of scientifically-conducted election polls, this article will once again  show proofs why.

In the 2004 and 2007 elections our two most respected pollsters in the Philippines, namely the Social Weather Station (SWS) and Pulse Asia did quite well. However, both SWS and Pulse Asia fared better in predicting the results of the senatorial elections in 2004 compared to 2007. Statistically Speaking thus challenged them to go read their books again and made a number of recommendations ¹⁰ :

1. Conduct the last pre-election poll a week before the election;
2. SWS should improve its exit poll design ¹¹, especially for NCR (on-site interviews might be better than home interviews);
3. SWS and Pulse Asia should hire two different firms to do their field operations to make them truly independent of one another. 

So how did SWS and Pulse Asia do  in the 2010 elections?

We will look at four polls: the BW-SWS survey  conducted on 2-3 May 2010 jointly by Business World (BW) and the SWS; the Pulse Asia survey conducted 23-25 April 2010; and the TV5-SWS Exit Poll  and the ABS-CBN-Pulse Asia Exit Poll both  conducted on 10 May 2010. We will refer to them as the BW-SWS survey, the Pulse Asia survey,  the TV5-SWS Exit Poll and the ABS-CBN-Pulse Asia Exit Poll.

All four surveys correctly predicted the winner of the Presidential election and all 12 winners of the Senatorial election.   In fact,  the BW-SWS survey,  the TV5-SWS Exit Poll and the ABS-CBN-Pulse Asia Exit Poll  correctly predicted all the winners in the Presidential, Vice-Presidential and Senatorial elections in 2010!  Yes, all the winners! Just like Paul the octopus! (Table 1)

The Pulse Asia survey missed the winner of the Vice-Presidential race. It  failed to capture the surge of voter preference towards Binay in the last two weeks leading to election day. We  note that the survey was conducted on 23-25 April, more than two weeks before election and earlier than the BW-SWS survey. Because the timing of the survey is an important aspect in assessing surveys,  we therefore reiterate our recommendation to conduct the survey as close to the election day as possible. Maybe Pulse Asia is not as rich as SWS?

The two exit polls did very well in predicting the percentages of votes cast for the winning  presidential and vice-presidential candidates, missing by not more than 2 percentage points. (Table 1)

However, the Pulse Asia survey was relatively  off: 39.0% for Aquino instead of 42.1% and 28.0% for Binay instead of 41.6%. The BW-SWS survey was also off  for Binay: 37% instead of 41.6 %. (Table 1)

All four surveys correctly predicted within one percentage point  the percentage of votes cast for 3 out of 8 winning senatorial candidates. For BW-SWS, it was Revilla, Estrada, and Defensor-Santiago; for Pulse Asia, it was Estrada, Enrile, and Sotto; for the TV5-SWS Exit Poll, it was Drilon, Enrile, and Guingona; and for the ABS-CBN-Pulse Asia Exit Poll, it was Drilon, Enrile, and Osmeña. Overall, the exit polls did better than the pre-election polls. (Tables 1 and 2)

The worst predicted percentages of votes for senators are as follows: for the BW-SWS survey, it was Enrile, 36.0% instead of 41.3% and Guingona, 22% instead of 27.1%; for the Pulse Asia survey, it was Lapid, 21% instead of 29.1% and Marcos, 27.7% instead of 34.8%; for  the TV5-SWS Exit Poll, it was Recto, 27.1% instead of 32.8%; and for the ABS-CBN-Pulse Asia Exit Poll, it was Revilla, 56.8% instead of 51.5% and Estrada, 55.4% instead of 50.0%. (Table 1)

Using the same methodology previously adopted by Statistically Speaking ¹²,  the Exit Polls did better  than the pre-election surveys. This indicates how important  the use of sound exit poll methodologies is! It may be noted that SWS is now interviewing voters as they exit the polling places and may have learned from its mistakes in 2004. However, Pulse Asia still does its exit poll interviews at the voters’ homes nice to note that both pollsters have posted metadata on their methodologies at their respective websites, a best practice in statistics.  (Table 2)

Overall, SWS and Pulse Asia  did very well in its 2010 election polls, better than in 2004 and 2007. Better in terms of predicting the winners and the percentages of votes cast for the winning candidates. The BW-SWS survey got 94%, Pulse Asia survey, 86%,  the TV5-SWS Exit Poll, 96%  and the ABS-CBN-Pulse Asia Exit Poll, 96%. In the UP grading system, I would give them, 1.25, 1.75, 1.0, and 1.0, respectively. Ang galing di ba? Eh ano pa ba naman ang gusto natin para tayo maniwala?

  Some observations:

1. SWS and Pulse Asia are now using different sets of interviewers, which make them truly independent of each other unlike in the past when they subcontracted the field work to the same organization.
2. The 2010 TV5-SWS exit poll interviews were conducted as voters “exited” from the precincts unlike in previous SWS exit polls when interviews were conducted at home after the voters had returned to their residence.
3. Looks like SWS has more resources than Pulse Asia to do surveys – in terms of number of surveys conducted and in terms of proximity of the surveys to election day.
4. The TV5-SWS exit polls results were published May 17, or a week after elections while the ABS-CBN –Pulse Asia exit polls results were released on May 13. This is rather late and hopefully in 2013, our pollsters will try to finish and release their work earlier.

We now repeat what we said before. As a statistician, we would be happy for SWS and Pulse Asia to do consistently well in their survey work. We are therefore happy at the resounding success they had in the 2010 polls and we hope they will continue improving their work. We also hope the candidates and the voters will learn to manage the SWS and Pulse Asia results better. For their own good.

Meanwhile, our congratulations also go to our champion boxers  Nonito Donaire,  Brian Viloria and Ana Julaton for their latest  victories! And if you still have not been  counted in the 2010 Census of Population and Housing, please call the NSO at telephones numbered 713-2789 and 716-9368 !

 

Reactions and views are welcome through email to the author at ra.virola@nscb.gov.ph

________________

¹ Secretary General of the National Statistical Coordination Board (NSCB) and Chairman of the Statistical Research and Training Center (SRTC). He holds a Ph. D. in Statistics from the University of Michigan in Ann Arbor, U.S.A. and has taught mathematics and statistics at the University of the Philippines. He is also a past president of the Philippine Statistical Association. This article was co-written by Noel S. Nepomuceno,  Information Technology Officer II of the NSCB. The authors thank Lina V. Castro, Cynthia S. Regalado, Jessamyn O. Encarnacion,  Candido J. Astrologo, Jr., Ma. Libertie V. Masculino, Albert Garcia, Andrea Baylon, and Jeffrey Enrado for the assistance in the preparation of the article. The views expressed in the article are those of the authors and do not necessarily reflect those of the NSCB.

² See Statistically Speaking, http://www.nscb.gov.ph/headlines/StatsSpeak/default.asp

³See Statistically Speaking, On Election Polls: Part III,  http://www.nscb.gov.ph/headlines/StatsSpeak/050604_rav_polls.asp

⁴ Recommendation No. R(99) 15, adopted by the Committee of Ministers on 9 September 1999.

⁵ Article 10, Freedom of Expression: 1. Everyone has the right to freedom of expression. This right shall include freedom to hold opinions and to receive and impart information and ideas without interference by public authority and regardless of frontiers. This Article shall not prevent States from requiring the licensing of broadcasting, television or cinema enterprises.  2. The exercise of these freedoms, since it carries with it duties and responsibilities, may be subject to such formalities, conditions, restrictions or penalties as are prescribed by law and are necessary in a democratic society, in the interests of national security, territorial integrity or public safety, for the prevention of disorder or crime, for the protection of health or morals, for the protection of the reputation or rights of others, for preventing the disclosure of information received in confidence, or for maintaining the authority and impartiality of the judiciary.

⁶ See Statistically Speaking, On Election Polls: Part III http://www.nscb.gov.ph/headlines/StatsSpeak/050604_rav_polls.asp  

⁷ Adopted and proclaimed by the United Nations General Assembly on December 10, 1948.

⁸ By Prof. Wolfgang Donsbach, University of Dresden, Germany for the Foundation for Information.

⁹ See Statistically Speaking, On Election Polls: Part III http://www.nscb.gov.ph/headlines/StatsSpeak/050604_rav_polls.asp

¹⁰ See Statistically Speaking, On Election Polls: Part IV: Who Did Better – SWS or Pulse Asia? http://www.nscb.gov.ph/headlines/StatsSpeak/071204_rav_polls.asp

¹¹ On 11 July 2004, the Philippine Daily Inquirer carried an article on the SWS admission of errors in the Exit Polls for NCR, Davao Region, SOCCSKSARGEN, and ARMM.

¹² See Statistically Speaking, On Election Polls: Part IV: Who Did Better – SWS or Pulse Asia? http://www.nscb.gov.ph/headlines/StatsSpeak/071204_rav_polls.asp

 

Table 1.

	BW-SWS		PULSE ASIA		TV5-SWS 2010 Exit Poll		ABS-CBN - PULSE ASIA Exit Poll		COMELEC
	May 2-3, 2010		April 23 to 25, 2010		May 17, 2010		May 13, 2010
	% Votes		% Votes		% Votes		% Votes		% Votes
Noynoy Aquino	42.0%		39.0%		43.30%		41.1%		42.08%
Joseph Estrada	20.0%		20.0%		26.40%		29.2%		26.25%
Manny Villar	19.0%		20.0%		14.70%		16.0%		15.42%
Gilberto Teodoro	9.0%		7.0%		10.30%		8.8%		11.33%
Eddie Villanueva	3.0%		3.0%		3.40%		2.7%		3.12%
Richard Gordon	2.0%		2.0%		1.40%		1.0%		1.39%
Nicanor Perlas	0.1%		0.3%		0.13%		0.2%		0.15%
Jamby Madrigal	0.2%		0.1%		0.23%		0.3%		0.13%
John Carlos de los Reyes	0.3%		0.2%		0.15%		0.2%		0.12%
Jejomar Binay	37.0%		28.0%		42.50%		42.7%		41.65%
Mar Roxas	37.0%		37.0%		39.20%		37.4%		39.58%
Loren Legarda	12.0%		20.0%		11.50%		13.7%		12.21%
Bayani Fernando	4.0%		3.0%		2.90%		2.4%		2.89%
Edu Manzano	3.0%		3.0%		1.80%		2.5%		2.30%
Perfecto Yasay	1.0%		1.0%		1.40%		0.8%		1.04%
Jay Sonza	0.3%		0.3%		0.20%		0.3%		0.18%
Dominador Chipeco, Jr.	0.2%		0.05%		0.43%		0.2%		0.15%
	% Votes	Rank	% Votes	Rank	% Votes	Rank	% Votes	Rank	% Votes	Rank
Revilla, Ramon “Bong” B.	51.0%	1	52.8%	1-2	54.45%	1	56.8%	1-2	51.5%	1
Estrada, Jinggoy E.	50.0%	2	50.0%	1-3	51.01%	2	55.4%	1-2	50.0%	2
Defensor-Santiago, Miriam P.	45.0%	3	47.9%	2-3	47.61%	3	47.2%	3	45.7%	3
Drilon, Franklin M.	39.0%	4	38.4%	4-6	42.28%	4	41.4%	4-5	41.8%	4
Enrile, Juan Ponce	36.0%	5	40.6%	4-6	41.95%	5	42.1%	4-5	41.3%	5
Cayetano, Pilar Juliana US.	34.0%	6	38.1%	4-6	37.65%	6	38.7%	6	36.0%	6
Marcos, Ferdinand Jr. R.	31.0%	7	27.7%	9-10	33.54%	7	33.1%	7-8	34.8%	7
Recto, Ralph G.	29.0%	9-10	30.1%	7-10	27.10%	10	30.4%	9-11	32.8%	8
Sotto, Vicente III C.	30.0%	8	32.3%	7-9	28.41%	8	33.9%	7-8	31.4%	9
Osmeña, Sergio III D.	29.0%	9-10	32.2%	7-9	27.36%	9	30.5%	9-11	30.7%	10
Lapid, Manuel M.	26.0%	11	21.0%	11-18	25.84%	12	30.4%	9-11	29.1%	11
Guingona, Teofisto III D.	22.0%	12	23.0%	11-15	26.66%	11	25.4%	12-13	27.1%	12

Note: The Comelec percentages were derived by dividing the votes received by 37,685,706, the number of actual voters from COMELEC.

Data Sources: Social Weather Stations (SWS), PulseAsia Inc., Commission on Elections (COMELEC)

 

Table 2.

	BW-SWS		PULSE ASIA		TV5-SWS 2010 Exit Poll		ABS-CBN - PULSE ASIA Exit Poll
	May 2-3, 2010		April 23 to 25, 2010		May 17, 2010		May 13, 2010
	Winner Points	% Votes	Winner Points	% Votes	Winner Points	% Votes	Winner Points	% Votes
Noynoy Aquino	50	20	50	14	50	18	50	20
Jejomar Binay	50	12	0	0	50	20	50	18
Revilla, Ramon “Bong” B.	50	20	50	18	50	16	50	10
Estrada, Jinggoy E.	50	20	50	20	50	18	50	10
Defensor-Santiago, Miriam P.	50	20	50	16	50	18	50	18
Drilon, Franklin M.	50	16	50	14	50	20	50	20
Enrile, Juan Ponce	50	10	50	20	50	20	50	20
Cayetano, Pilar Juliana US.	50	16	50	16	50	18	50	16
Marcos, Ferdinand Jr. R.	50	14	50	6	50	18	50	18
Recto, Ralph G.	50	14	50	16	50	10	50	16
Sotto, Vicente III C.	50	18	50	20	50	14	50	16
Osmeña, Sergio III D.	50	18	50	18	50	14	50	20
Lapid, Manuel M.	50	14	50	4	50	14	50	18
Guingona, Teofisto III D.	50	10	50	12	50	20	50	18
SUB-TOTAL Senators only	600	190	600	180	600	200	600	200
SUB-TOTAL Overall	700	222	650	194	700	238	700	238
SUB-RATING Senators only	100%	79%	100%	75%	100%	83%	100%	83%
SUB-RATING Overall	100%	79%	93%	69%	100%	85%	100%	85%
GRAND TOTAL Senators only	790		780		800		800
GRAND TOTAL Overall	922		844		938		938
OVERALL RATING Senators only	94%		93%		95%		95%
OVERALL RATING Overall	94%		86%		96%		96%

Sources of basic data: Social Weather Stations (SWS), PulseAsia Inc., Commission on Elections (COMELEC) 

 

 

Posted: 12 July 2010.

 

Statistically Speaking

Main Page

Related Links

Press Releases

Announcements

For the Record

E-Newsletter

Media Services

Events

 

	E-mail the webmaster	Terms of Use					Home • Top of Page
		1997-2012, National Statistical Coordination Board Makati City, Philippines