On February 13, a noteworthy diplomatic meeting occurred. Nancy Powell, the U.S. ambassador to India, met with Gujarat Chief Minister Narendra Modi, the chief minister of the Indian State of Gujarat, at his home in the new state capital of Gandhinager. The talks were the highest-level meeting between Modi and any U.S. government official since the State Department revoked Modi’s visa in 2005, following allegations that he and his State government had failed to protect minority Muslims from attacks by Hindus during the communal riots that swept across his Gujarat state in 2002.
The sudden thaw in the relationship comes as India heads into the 2014 Lok Sabha (Lower House of Parliament) elections to be held this April and May. Modi’s Hindu-nationalist Bharatiya Janata Party (BJP) has named him as its candidate for prime minster should the BJP gain an electoral victory. By meeting with Modi, American officials are signaling that they view a BJP victory in the coming elections as a real possibility, and are hoping that they can improve the damaged relationship with Modi as much as possible prior to the election.
Unsurprisingly, U.S. officials and India watchers around the world are closely monitoring the torrent of election polls and public opinion surveys streaming out of the Indian media to see where the BJP stacks up against its rivals as the elections draw near.
In examining these polls, however, observers ought to be mindful of the fact that election polling in India is a notoriously unreliable exercise. It suffers from the political biases of the polling agencies and news outlets that produce the polls. A more serious challenge to reliability comes from operational problems inherent in India’s mammoth electorate, complex demographics, daunting geography and poor infrastructure, all of which make accurate polling an immensely labor intensive, expensive and often-dubious process.
A Monumental Task
Unlike the United States, where state electorates divide themselves relatively neatly into Reds and Blues, Indian states have their own idiosyncratic grouping of both national and regional political parties. The Indian Congress Party and the BJP, the two principal national parties, exert influence nationwide, but their power has waned in recent decades in favor of regional parties. These parties generally represent certain caste, linguistic, ethnic or class groups, groups which themselves are often uniquely in a particular state.
Indeed, India has rarely demonstrated a pan-Indian, national voting pattern, except when a single emotive issue develops momentum, such as in the sympathy vote following the 1984 assassination of Indira Gandhi. In general, past elections have tended to turn on local issues and identity politics. While a few national issues such as inflation, anti-incumbency and national security are consistently of concern for many Indian voters nationwide, they have tended to play a secondary role in determining how citizens actually vote. Local politics are still the name of the game in India; and, unfortunately for the pollsters, Indian local politics are extremely hard to predict.
This predominance of local politics, local issues and local parties, has given each of the 543 Indian parliamentary constituencies its own distinctive political color. Since national polling is most accurate when a survey sample can serve as a statistically meaningful representation of the national whole, and because India’s constituencies are so diverse and cast their votes for such different reasons, it makes a proper sample incredibly difficult to construct.
To account for the diversity in the electorate, polls must be taken in most – if not all – of the 543 constituencies. This is extremely expensive and generally unworkable. Constituencies in India are also numerically huge (most have more than a million people) and often physically challenging for poll-takers to access. But technology is not a panacea for the pollsters: despite India’s rapidly growing telecommunications and Internet industries, the vast majority of Indians still live without phone or Internet access.
This lack of communications infrastructure has made face-to-face, door-to-door surveying the preferred method of polling. Agencies send data collectors personally to survey village and city halls, bazaars and town courtyards, schools and universities. Unsurprisingly, this method is not the most efficient or cost-effective way to do polling.
To get a meaningful number of interviews, in a majority of constituencies, a polling agency would need to employ a virtual army of pollsters. But because no single polling agency in India has the manpower or the funds to do meaningful door-to-door polling in a majority of constituencies, polling agencies must extrapolate data from one constituency to another; or, in some cases, to extrapolate data from a few constituencies to forecast an entire state. Agencies examine the socioeconomic composition of a constituency, look at the castes and religious communities represented, and use the data from that area to calculate and predict the results for another location with similar demographics. But since each area has its own distinctive set of issues and parties, extrapolation of data based exclusively on caste or socio-economic considerations is bound to be flawed on a larger scale.
Anything But Evenhanded
An additional source of poll unreliability stems from the relationship between India’s political parties and the polling agencies and media outlets. Indian news outlets – which ultimately sponsor the polls – tend to be ideologically slanted. Many have long-standing historical ties to political parties. A number of large, national dailies, such as the Hindu, Aaj Tak, and the Times of India, are either owned or operated by political party heavyweights. The Hindustan Times, for instance, was founded by long-time Congress Party supporters, the Birla family. K.K. Birla and G.D. Birla, who at different times owned and edited the paper, have both served as Congress Party MPs and provided a large amount of funding for the party. The Pioneer newspaper, another influential daily, is today partly-owned and edited by a standing MP of the BJP, Chandan Mitra. This political connection has at times colored the polls the papers release.
Media is by no means separated from politics in the United States either, as any cursory look at Fox News and the political leanings of Rupert Murdoch would demonstrate. But the U.S. has trade bodies such as the American Association for Public Opinion Research and the National Council on Public Polls that issue guidelines on the dissemination of opinion poll results. And as a result, pollsters in the U.S. are usually forced to reveal their sample size, methodology and margin of error.
No such institutional oversight exists in India. Apart from the CNN-IBN-The Hindu-CSDS poll, which put out a detailed note on the methodology, none of the polls even mention the number of respondents for their polls (at least, not in any of their public disclosures in newspapers).
Moreover, news outlets do not even conduct the pre-election surveys they publish themselves. Instead, they contract private polling agencies to do the polling for them. Some of these companies, like C-Voter and MARG, are exclusively election polling groups. Most, however, are conventional commercial marketing research firms. These commercial marketing agencies are not often ideologically tied. Yet, because they are contracted by papers that do have political links and are better disposed to certain parties, there is often explicit or implicit pressure on polling agencies to skew their results in favor of the sponsor’s party of choice. Agencies that want to continue their contracts into the future and the papers want to have results that promote their party, forcing agencies at times to choose to ignore certain data, or to extrapolate statistics in a way sympathetic to the sponsor’s chosen party.
Making matters even more difficult, the Indian voter is in general tight-lipped with his or her responses to pollsters. This has even been the case in more innocuous surveying like census data collection. Pollsters report that many voters do not think that polls or surveys are innocent, and that their responses to election polling will come back to harm them through retribution by the one party or another. Many opt instead to refuse to answer the questions, others just offer up an answer they think the pollster would like to hear. So even if polling were less of a biased, logistic nightmare, pollsters would have little confidence that people were answering truthfully.
A History of Inaccuracy
The lead up to the Lok Sabha election of 2004 highlights the unreliability of national polling. Nearly all pre-election polls suggested a robust victory for the BJP and its allies. The polling agencies and media outlets were left red-faced when the results showed a Congress Party upset. Granted, 2004 was ten years ago and polling methods have improved since then. Indeed predictions for the 2009 election were more accurate, with most polls correctly picking a slim Congress Party coalition victory. The 2009 election demonstrated that pre-election polls, when taken in aggregate, can provide some valuable insight.
But while polling organizations may claim that they now employ the most sophisticated statistical modeling tools available, polling’s history – and India’s unique blend of confounding factors – has proven it unwise to take Indian polling as failsafe predictor of voter preference. The best the India-watchers can do is wait until the ballots are counted.
Jonah Force Hill is an international affairs consultant and a former Fellow of the Belfer Center at the Harvard Kennedy School of Government.