India’s Statistical System: Past, Present, Future

The statistical system of a country acts as its mirror. It generates the statistics that allow observers to see how well a country is performing on key socioeconomic parameters such as per capita income, inflation, poverty, life expectancy, and average years of schooling. In most countries, a single agency or a handful of agencies produce the bulk of official statistics. The work of these and other peripheral agencies is typically regulated by a national statistical office that ensures statistical standards are in line with international norms. The statistical system provides citizens an impartial view of the state of their country’s progress. It enables policymakers and investors to make informed decisions.

India’s official statistical system, as we know it today, began taking shape during the British Raj (1858–1947). Colonial efforts to develop the statistical system were driven by an imperative to track a key market for English products; hence, trade statistics were much more well-developed compared to statistics on domestic economic production or socioeconomic development. Several official committees suggested reforms to correct the lopsided development of the official statistical system in British India, but most of their recommendations weren’t implemented.

It was only after India’s independence in 1947 that a serious effort was made to revamp India’s statistical infrastructure. The globally renowned statistician P. C. Mahalanobis led this drive and was backed by Jawaharlal Nehru, India’s first prime minister. The Mahalanobis model of data collection relied largely on random sampling and inspired similar initiatives elsewhere in the developing world.

With Mahalanobis’s death in 1972, India’s statistical system lost a powerful champion who had ensured its relevance without compromising its autonomy. Other changes in the post-Mahalanobis era diminished the statistical system. Growing insularity, the lack of investments in computing resources, and the declining influence of the Planning Commission (which had earlier been a pillar of support for statisticians) eroded the statistical system’s effectiveness over time.

By the turn of the twentieth century, India’s statistical crisis had become too severe to be ignored. In early 2000, the central government appointed a high-level commission led by the former central banker C. Rangarajan to review the statistical system and suggest ways to improve it. Some of the commission’s recommendations were implemented but only in a half-hearted manner.

The modest reforms initiated in the wake of the Rangarajan commission’s recommendations failed to address the deep-rooted crisis the system faced. The development of the statistical system remained stunted, impacting the credibility of data releases.

Meanwhile, the political pressures on the statistical system grew as data assumed a major role in public discourse. A weakened statistical system failed to assert its autonomy in the face of such pressures. The past decade has seen a number of statistical controversies even as the statistical system struggled to reform itself.

Today, India’s statistical system faces a major crisis. Producers and users of official statistics have stated that the lack of a clear road map to address this crisis worries them as much as the crisis itself.

This paper argues that a Statistical Reforms Commission should be set up to address the roots of India’s statistical crisis. The proposed commission must outline the legal framework that would underpin a revamped statistical authority. In addition, this commission should frame a new statistical architecture that is able to meet the emerging needs of data users. It should prepare a national statistical strategy document after taking into account the concerns of all key stakeholders.

Without wholehearted reforms, India’s statistical system will fail to deliver the kind of high-quality, high-frequency datasets that Indian citizens, policymakers, and investors expect from it today.

Organisation Through Neglect: Understanding Field Administration in India

This paper analyses the structure and processes of field administration in India and matches these with the outcomes on the ground. It highlights features of the administrative structure, human resources, and organisational culture that result in the sub-optimal delivery of social services and policy implementation. Recognizing the importance of historical antecedents, the paper provides a brief account of how field administration has developed over time. Subsequently, it presents the findings of a case study of a district situated in Madhya Pradesh. This is followed by a delineation of the key areas for reform and some possible strategies, though these need to be formulated after rigorous debate.

The paper traces the roots of the present-day field administration to the colonial era, when the existing decentralised and diffused field administration system was changed to achieve the government’s goals to maximise revenue from land and forests and maintain order. The district became the key administrative unit in the field, and the district collector (DC) the overall administrative head. After Independence, government goals shifted dramatically, and socio-economic development became a central concern. However, the basic administrative structure was retained, while several departmentalorganisations were added.

Frontline Functionaries in India: The Absent Policy

In the context of a high level of dissatisfaction with the delivery of government services in India, this paper traces the evolution of policy regarding key actors in this process— frontline functionaries or street-level bureaucrats. Frontline functionaries form a special subset of the bureaucracy as they come in close contact with citizens, and negotiate government policy with specific citizen needs. The paper provides a historical background illustrating the continuity of colonial era ideas with the current policy regarding frontline functionaries, and also examines it against broader developments in policy and the administrative structure. The policy regarding frontline functionaries is scrutinised against their three roles: as government employees, as professionals or skilled workers in specific fields, and as workers serving the community.

The paper shows that frontline functionaries, only loosely connected to the government before colonization, subsequently became government employees, as the government sought to increase the efficiency of revenue collection from land and forests, and to maintain order. Because the colonial government minimized administrative costs, it paid frontline functionaries very poorly, placed them near the bottom of an extremely hierarchical bureaucratic structure and provided them with minimal promotion avenues. Frontline functionaries were not considered worthy of serious responsibility, expected to follow orders, had little education and training and were treated harshly by senior officials. Consequently, their performance was unsatisfactory. They were often corrupt and exploited the community.

India’s Regulatory Shift: An Examination of Five Agencies of the Post- Liberalisation Era

This paper explores the design of Indian regulatory agencies established post-liberalisation from an administrative perspective. Regulatory agencies were set up to replace state inefficiencies, and to discipline profligate state agencies, even as much as they were a response to state-market reorganisations and the challenge of privatization. Regulation provided an opportunity for upper levels of the Indian bureaucratic state to recast their power, with the idea that it would provide a framework for economic rationality, independence and technical specialization to take centre-stage. In actual practice however, the design of each of the regulatory agencies is shaped largely by pre-existing legal frameworks and institutions, and the agencies have remained quite tied in with their counterpart departments and on retired bureaucrats. However, in spite of these limitations, these agencies have some common features imbued by legislative mandate and organisational design which are unique in the context of the Indian state. They have focus and stability, a degree of functional independence, and most importantly, a concentration of power, which enables them to think through and implement complex policy transitions from multi-year and context-specific perspectives. The paper builds on learnings from a series of conversations with regulatory agency chairpersons in order to identify what regulatory governance is, in terms of the powers and mandate of the regulatory agencies and what makes them distinctive from the rest of public administration.

Working Paper Series: Cooperative Federalism—From Theory to Practice

Federalism in India is at a crossroads. Over the last three decades, India has undergone significant social, political, and economic transformations which have created new fault lines in India’s precarious federal compact. Economic liberalization and political decentralization have fundamentally shifted the dynamics of union-state economic relations, creating the space for a renegotiation of fundamental frameworks of fiscal federalism. However, institutional inertia and, later, the emergence of a single party majority government at the center have called into question how deeply the new language of “cooperative federalism” has been embedded into actual practice.

The response to these newly emergent tensions in shaping India’s federal architecture must be both political and institutional. Indeed, there is a need for a deeper analytical engagement with the dynamics of contemporary federalism in India and a framework within which to debate the institutional and political solutions to the challenges as they unfold. This new working paper series on cooperative federalism—the product of a collaborative research program led by the Centre for Policy Research and the Carnegie Endowment for International Peace—aims to fill this gap.

The research agenda underlying this new series has three parts: documenting the shifts in the economic realities of states and the evolution of centre-state dynamics; analyzing governmental institutions for managing federalism; and examining what institutional structures might be necessary for managing new or emerging priorities.

Working Papers:

1. Statutory Regulatory Authorities and the Federal System in India – K.P. Krishnan, Amrita Pillai, and Karan Gulati
2. Resetting the Fiscal Architecture: Lessons for India – Kandarp Patel and Anoop Singh
3. Towards an Inter-State Migration Council – Chinmay Tumbe
4. Institutions of Political Mediation: The Office of the Governor – Chakshu Roy and M. R. Madhavan
5. The Adjudication of Federal Disputes – Raeesa Vakil
6. Parallel Streams: Political Federalism and Economic Integration in India – Niranjan Rajadhyaksha and Prakhar Misra
7. The Goods and Services Tax Council: Dialectics and Design – Haseeb A. Drabu
8. Federal Aspects of India’s Emerging Internal Migration Governance Frameworks – Mukta Naik
9. The Rise and Resilience of Fiscal Transfers Amidst Party System Change – Suyash Rai and Milan Vaishnav

Poverty of Circumstances and Poverty of Opportunity

Ordinary people in rural Bihar and in Patna and Delhi have virtually no opportunity to be tested for a hidden talent, be it for athletics or singing or chess or mathematics or art. Not one of more than 800 young people we interviewed has ever competed at the national, state or district level in any of eight activities of different kinds. Alternative careers pass them by. Future champions can remain unidentified and unrewarded. Something better is necessary to make equality of opportunity less a slogan and more a reality. Examples from across the world show what can be done.

Land Rights For Urban Slum Dwellers: A Review of the Odisha Land Rights to Slum Dwellers Act, 2017 and the Jaga Mission

Theorists of economic development have emphasized the importance of clear, legally guaranteed property rights in ensuring economic development. Legally enforceable rights over land are thought of as a solution to incentivize investment in land, ensure optimal use of resources and create the conditions for market led-growth. Land rights (either through titling or restricted rights over land) have long been considered as a precondition for development because they increase access to formal credit. In the specific context of urban slum dwellers, the added benefit of land rights is that these rights can be used to ensure tenure security, help provide municipal services and other social benefits.

Based on these presumptions in 2017, the Odisha Land Rights to Slum Dweller Act (OLRSDA) and the Odisha Municipal Corporation (Amendment) Act (OMC) were enacted. These enactments along with the implementation of the Odisha Liveable Habitat Mission, (also known as the JAGA mission) kickstarted a process of urban redevelopment. The policy measure of providing land rights to slum dwellers has been replicated in Punjab under the “BASERA” scheme and is under consideration in Himachal Pradesh.

In this paper, I have analysed Odisha’s legislative and policy-based interventions to alleviate urban poverty through land rights. My key findings in this paper are as follows: (i) land rights under the OLRSDA are not sufficiently integrated within the existing legal framework and there exist several ambiguities. As a result, the land right granted cannot be used as an economic resource and offers limited increase in tenure security; (ii) land rights are beneficial yet insufficient to lead to the proposed goals of increasing access to municipal services and access to subsidies; (iii) government intervention is required at each stage of the slum redevelopment process to ensure the achievement of these goals.

The State of India’s Pollution Control Boards – A Series of Papers

State Pollution Control Boards (SPCBs) and their counterparts in union territories, the Pollution Control Committees (PCCs), are responsible for several key functions to curb air pollution. These include granting consents, setting emission and effluent standards, monitoring compliance of industry with these standards, and enforcement action in case of violations. However, they have often come under fire for failing to effectively deliver on their mandate. Over the years, several studies have revealed that SPCBs and PCCs do not have the resources and capacity to perform the functions assigned to them under various laws. In ‘The State of India’s Pollution Control Boards’, we explore some of the key institutional issues faced by SPCBs and PCCs in the Indo-Gangetic Plain through a series of papers:

  • “Who has a seat at the table?” – Examines the composition of the Boards and their ability to engage in policy-making and take decisions in furtherance of their statutory goals.
  • “Who is at the helm?” – Analyses the qualifications of the Board’s leadership, the length and stability of their tenure, and whether they are well-placed to guide the Boards’ functioning.
  • “Who is in the field?” – Evaluates the adequacy of the Boards’ capacity – particularly technical capacity – to perform critical functions like consent granting, inspection, monitoring, and enforcement.
  • “Are they in the green?” – Analyses the fiscal health and autonomy of the Boards, examines spending patterns and evaluates their capacity to effectively utilise finances (released on April 27, 2023)

Links to all PDFs are below:

    1. Executive Summary of the series
    2. Who has a seat at the table?, Annexure
    3. Who is at the helm?
    4. Who is in the field?
    5. Are they in the green?, Annexure (April 2023)
    6. All RTI responses for the study

Note: This page was updated on 31/10/2022 (addressed minor errors) and on 27/04/2023 (uploaded fourth working paper).

Frontline Workers and the Covid-19 Pandemic

The Covid-19 pandemic proved to be a global disaster that took countless lives, many of whom were frontline functionaries deployed to contain this crisis. As a result, the Indian state, unlike before, had to acknowledge the risk that accompanied their workers in this crisis. They promised them vaccination, health & life insurance, amongst other material benefits to protect, insure and motivate such cadres. The paper attempts to study these promises, in particular its vaccination and insurance policies, to better understand what it meant to be a frontline worker in this crisis. Did the Indian administration account for all at-risk workers and their context, or did they simply exclude many of them in their policies ? The essay, at its core, will answer this question as it examines the Indian state’s capacity to celebrate its frontline workers.

Band-Aid Solutions Won’t Staunch India’s Power Crisis

India is experiencing yet another power crisis. In the last week of April, the daily peak power shortage rose to 10,778 megawatt (MW). Power deficits reached 5% of overall supply at the national level, while a few states experienced steep deficits ranging up to 15%. To deal with the scarcity, distribution utilities (discoms) resorted to loadshedding — rationing supply hours for households and economic activities, usually unannounced. Such power supply disruptions not only hurt economic activities and productivity, but also impinge on essential services and human well-being. Though India claims to have achieved power adequacy (officially, the annual national power deficit has been below 0.5% for the last six years), periodic shortages remain a predictable phenomenon. Coal fuels more than 70% of the electricity generated in India. Monsoon-induced coal production slowdowns and consequent power shortages are more or less annual events in India. Since 2006, there have been 10 instances when more than 50 power plants reported critically low coal stock. India experienced such a power crisis in October 2021. However, an early summer power shortage is unprecedented. While the situation may appear to have improved in May with a small drop in demand and a ramp up in generation, the system is running under acute stress. 1 As electricity demand is projected to grow, these shortages could occur more frequently through the year if they are not addressed adequately now.

What explains this early summer power shortage? How is it different or linked to monsoon shortages? As we answer these questions, we trace the structural roots of persisting power shortages in India. We analyse immediate government responses to the crisis and their consequences and identify priorities for a long-term solution that are consistent with energy transition. In doing so, we argue that current short-term fixes for India’s power crisis neither address rising uncertainty in the electricity system nor are aligned to a transition towards a 21st century energy system.