How The PMEAC ‘Peer-Reviewed’ And ‘Desk-Rejected’ Arvind Subramanian’s GDP Study
Following Arvind Subramanian’s sensational claim that India significantly overestimated its GDP growth rate between 2011 and 2016, a number of economists including the members of the Prime Minister’s Economic Advisory Council have come out with opinion pieces providing an array of objections to his findings. The most curious of these objections is that the paper is not peer-reviewed. For those who are not very conversant with this academic term, peer-reviewing is the ultimate test of establishing an academic idea and it involves experts in a field scrutinising an academic work and then giving their view on the acceptability of the results of the study.
In any peer-reviewed academic journal, there are broadly three possible outcomes to this process.
- First, in rare cases, experts think highly of the study and immediately ‘accept’ the results.
- Second, a process usually called ‘revise and re-submit’ implies that the experts think that the idea of the study is useful but they suggest certain additional or alternative research that would make the results acceptable.
- The third case is one where the experts do not find the study to be useful at all and this results in a ‘desk-reject’ of the study.
These three options make up the peer-reviewer’s trilemma. In order to evaluate Subramanian’s study and the PMEAC’s objections to it, it makes sense to look at this issue in terms of these three options.
Is Subramanian’s study good enough to be immediately acceptable, or is it useful in terms of its motivation but needs additional work, or is it a completely misplaced idea that has no use?
To answer these questions, it’s useful to recollect the main ideas and methods in his study and the objections raised against it.
Subramanian develops his argument in his paper in three steps. The first step demonstrates that the relationship between the GDP growth rate and the growth rate of seventeen economic indicators that are popularly used as proxies of the state of the economy, changes drastically after 2011, the year that India adopted a changed methodology for estimating GDP. He shows that for most of these indicators, the correlation between growth of GDP and these indicators changed from positive to negative after 2011. He concludes that such drastic changes in the relationship between these variables are indicative of the fact that there were errors in the estimations of GDP growth according to the new methodology. An important implication of his conclusion is that he did not think that such drastic changes were possible due to the transformation of the Indian economy. Of course, any change from positive to negative correlation need not necessarily be considered as drastic, and only statistical tests can establish whether it indeed is so.
Subramanian’s second step undertakes such statistical tests by running cross-country regressions. In this exercise, he estimates a relationship between country growth rates and four variables, i.e., credit, exports, imports and electricity consumption, for a large number of countries including India. Next, he shows that for the period 2002-2011, the Indian GDP growth rate is completely explained by this estimated relationship.
For the period 2011-2016 however, not only is the Indian growth rate not explained by this relationship completely, but it is so far off from the relationship that it can be termed an ‘outlier’. This, according to Subramanian, statistically confirms an over-estimation of the growth rates in India, post 2011.
Finally, in his third step, Subramanian uses the difference between the predicted growth from the estimated relationship and the official growth rates to calculate the magnitude of overestimation of the GDP growth.
The members of the PMEAC have raised a number of objections to this study. Like any careful peer-reviewer, they have put forward their critique to each of the three steps described above.
First, the seventeen indicators used in the first step have been questioned for their appropriateness. It is pointed out that most of the indicators are related to industry and are thus incapable of capturing growth coming from agriculture or services.
The deliberate exclusion of tax-based indicators, which are considered strongly correlated with growth, have been viewed to be a weakness of the study.
The fact that the indicators are measured in terms of volume, whereas GDP is measured as value-added, has been mentioned. Next, critiquing Subramanian’s second step, a number of these opinion pieces have pointed out that the relationship between growth and its drivers can and do change from time to time. This has been suggested to be the main reason behind the regression where Indian growth is found to be an outlier.
Specifically, it has been pointed out that in recent times, three of the four regression variables, i.e, credit, imports and exports, have ceased to be indicators of growth in India. Particularly, weaknesses in the banking sector and the role played by public investment and consumption as the main drivers of growth in India in this period, have made credit growth less relevant. Similarly, imports and exports have become less correlated with growth in this period as domestic sectors have become more dynamic. It has been pointed out that similar changes have made China an outlier in the regression exercises as well. Finally, pointing to the weaknesses in Subramanian’s first two steps, the common conclusion of these opinion pieces seems to be that the conclusions of Subramanian’s study are unwarranted.
How do we assess Subramanian’s study in light of the PMEAC’s critique? Going back to the peer-reviewer’s trilemma, it is clear that the objections put forward by the members of the PMEAC are significant, and so it is difficult to accept Subramanian’s paper ‘as it is’.
The important question is whether the study has some useful aspects that justify more research in this direction or is it an academic cul-de-sac. A careful inspection of the objections to the study suggests that the second part of Subramanian’s paper based on cross-country regression exercises will find it difficult to stand up to scrutiny.
Subramanian himself accepts that the four variables may be inadequate and have been chosen mainly due to their easy availability.
Any further research in the cross-country framework would always be contentious due to the extremely heterogeneous nature of economies, which would make any growth estimation questionable. On the other hand, Subramanian does make an important point in the first part, where he points to the breakdown in the relationship between a large number of economic indicators and GDP.
While this may be due to the inappropriate choice of indicators or a structural change in the economy, one is still puzzled by the fact that a large number of well-accepted indicators not only stop being positively correlated (in terms of their growth rates) with GDP growth but actually become quite strongly negatively correlated with the latter, a peculiar behaviour, to say the least. Perhaps future research will throw more light on this issue. One thing is clear though. The PMEAC has desk-rejected Subramanian’s study.
Sabyasachi Kar is a professor at the Institute of Economic Growth, University of Delhi.
The views expressed here are those of the author and do not necessarily represent the views of BloombergQuint or its editorial team.