Issues and Limitations on Current Methodology

A. Reliability of provincial poverty estimates/Level of disaggregation of poverty statistics 


Issue: High coefficients of variation for poverty incidence estimates for some provinces 

NSCB Technical Staff response:

Based on a research for which results have been presented to the NSCB Executive Board, the coefficients of variation for provincial level estimates produced by the methodology for some provinces are too high (See Annex TN 200307-SS1-02-01). Because of this, the view of some is that the current methodology for poverty incidence estimation should not produce provincial estimates but only regional and national estimates.

The NSCB acknowledges the limitations of the methodology including those due to sampling issues on the Family Income and Expenditures Survey (FIES) conducted by the National Statistics Office (NSO) from which the income and expenditure data used for poverty estimation are gathered. Some of these limitations are caused by budget constraints, which limit the sample size of the FIES and affect the degree of precision of the provincial poverty statistics.

However, the NSCB is of the view that while efforts should continue to be exerted by the Philippine Statistical System (PSS) to produce official statistics with low coefficients of variation, statistical offices must also exert best efforts to be relevant to users’ needs.
The past few years, the NSCB has been receiving requests/demands for provincial (and even municipal) level estimates of poverty incidence. It is in recognition and appreciation of these user demands that the NSCB takes the position that notwithstanding the levels of coefficients of variation of the official poverty statistics (Computation of coefficients of variation (CVs) was done by the NSCB technical staff using the Stata software. There are concerns, however, with the approximations used in the software. The NSCB tried communicating with the developers of STATA and requested for the algorithm used in the computation of the CV. Unfortunately, the NSCB request has not been granted), provincial level estimates of poverty incidence must be released to the public, with the accompanying caveats/footnotes on the limitations of the estimates. However, if key policy-makers and other users give definitive indication for a preference not to be provided at all with official statistics with “unacceptable” levels of coefficients of variation, then the NSCB will re-consider its position, since given the limited resources available for statistical activities, the NSCB Technical Staff has been gearing its efforts more towards demand-driven initiatives. 

It may be noted that even some developed countries like Australia and the United States have released to the public, statistics with CVs higher than ten percent. To quote from the Australian Bureau of Statistics’ publication on Crime and Safety, Australia, for instance, “For all tables in the publication, only estimates with relative standard errors (RSEs) of 25% or less, and percentages based on such estimates, are considered sufficiently reliable for most purposes. However, estimates and percentages with larger RSEs have been included and are preceeded by an * to indicate they are subject to high SEs and should be used with caution. Estimates with RSEs greater than 50% should not be regarded as reliable, and are indicated by the symbol **.” In this regard, it is instructive to read a discussion on the suppression of statistics by statistical offices (See Annex TN 200307-SS1-02-02 & Annex TN 200307-SS1-02-03). 

The issue on acceptable levels of CV was also raised during an ADB forum held on 9 June 2003. The speaker, Professor Graham Kalton of Westat, who presented “Small-Area Estimates of School-Age Children in Poverty in the U. S.”, told the group that his presentation was a result of the U. S. Department of Education’s need for basis to allocate over $7 billion given every year to support education programs meant to meet the needs of educationally disadvantaged children. The allocations were to be based on the small area estimates of the numbers and proportions of children of age 5-17 in poverty produced by the Census Bureau. These estimates were evaluated by a panel of the National Research Council’s Committee on National Statistics. The presentation made by Prof. Kalton showed average coefficients of variation for two sets of estimates of the number of poor school-age children for 15, 226 school districts of 30% and 32% at the total level. By size of school district, the coefficients of variation were 14%, 23%, 28%, 32%, 39% and 64% for one set of estimates and 13%, 22%, 26%, 30%, 36% and 57% for the other set of estimates. Obviously, these are high levels of coefficients of variation, but according to Prof. Kalton, the estimates, nevertheless, were still used by the government. because they were still the best available and the government had to have some basis for the allocation of funds.

During the ADB forum, Dr. Kalton explained that while 10% CV is a common criterion, higher CVs do not necessarily mean that that the data should be suppressed or not reported. When these estimates with high CVs are released to the public, he emphasized however, that it is important to ensure that the users know the limitations of the estimates. 

What the foregoing discussion suggests is that, in the generation of official statistics by statistical organizations that are normally faced with resource constraints, and in responding to data demands by users, high coefficients of variation alone are not sufficient to cause the suppression or non-release of data. What is important is for the statistical agencies to exert best efforts to use scientific approaches in the generation of the official statistics and when such statistics are released, the public must be alerted to the limitations. 

In pursuing the commitment of the NSCB for transparency in the generation of official statistics, several technical meetings have been held on the matter and the new methodology. The results were also presented in a users’ forum sponsored by the National Anti-Poverty Commission (NAPC) and the National Economic Development Authority (NEDA) on January 10, 2003, which elicited a generally favorable feedback from the participants. The methodology was also presented during a joint meeting of the Technical Working Group on Income and Poverty Statistics (TWGIPS) and the Inter-Agency Committee on Labor and Income Statistics (IACLIPS)which also recommended for its approval. 


B. Menu-based approach to poverty measurement


Issue No. 1: Too stringent nutritional requirements 

NSCB Technical Staff response:

Some researchers think that the average daily per capita requirements of 100 percent adequacy for energy (which translates to an average of about 2000 kilocalories) and protein and 80 percent adequacy for other vitamins and minerals are too high and lead to overestimated poverty incidence. These requirements were determined by the FNRI based on its Recommended Dietary Allowances and the demographic profile of the Philippine population. In other countries, the requirements are 2,100 kcal per capita for Cambodia, Indonesia and Vietnam; 2,150 kcal per capita for China and 9,910 kcal for a family of 2 adults and 3 children for Malaysia (Figures were culled from country papers presented during the ADB Review Workshop on RETA 5917 held in Kuala Lumpur, Malaysia, December 2001.). Thus, the Philippine requirement for energy does not seem to be too high. However, the Philippine methodology has additional requirements for the other nutrients which are not explicitly required in other countries. 
Based on the feedback from participants during the January 10, 2003 users’ forum, and during the series of fora conducted by the NSCB in 2000 in six different places throughout the country, the 100 percent requirement for energy and protein and the 80 percent requirement for vitamins and minerals are not considered too stringent by the participants.

However, since the Philippine methodology includes other nutritional requirements on top of the 2000 kilocalories, the subsistence and poverty incidence of the Philippines may in fact be overestimated, when compared to those of other countries.


Issue No. 2: The Philippines is the only country that uses menus in estimating poverty. 

NSCB Technical Staff response:

A variety of methodologies for poverty measurement are available in the literature and different countries follow different approaches primarily because no one methodology has been declared to be superior over the others. To date, no definitive guidelines have been issued by any international organization on how to measure poverty. The ESCAP Seminar on Poverty Statistics in Bangkok last November 1999 (ESCAP, 1999b) acknowledged the diversity in measurement methodologies and made no recommendation about a particular methodology. The view of the participants was that given the inchoate stage of methodological development on poverty statistics, the conditions are not conducive to the formulation of a standard methodology. An expert group on poverty statistics, the Rio Group, was organized by the United Nations Statistical Commission in 1996 and has produced a number of documents on poverty statistics available at its website at www.ibge.gov.br/poverty . It is in the process of compiling a Compendium of “best practices”, but the “best practices” to be compiled are not the kind that will suggest a specific methodology to adopt. And never in the international fora of respected official statisticians did we hear anyone venture to say that it is wrong to use a “menu-based” approach for poverty assessment!

The use of a menu in the derivation of the official poverty line seems to be widely misunderstood. It is emphasized that the menu is just an artifice to derive the cost of the nutritional requirements. It makes it easier to appreciate the fact that the “food bundle” that satisfies the nutritional requirements is in fact, palatable to and/or fit for consumption by human beings! It is not meant to be prescriptive, but the “food bundle” is chosen to satisfy criteria such as cheap, locally available, and if possible, typical of the local eating practices. In other words, the food bundle must not just be a set of food items which satisfy the nutritional requirements – it must be such that when prepared under “normal”, local cooking conditions, the finished product is something that is “normally” eaten by local residents. For example, approximately 4.3 kilograms of instant powdered filled milk or 15.2 kilograms of fresh carabao’s milk will satisfy the set individual nutritional requirements but it does not seem normal for an individual to consume just 4.3 kilograms of instant powdered filled milk or 15.2 kilograms of fresh carabao’s milk in one day. 


Issue No. 3: The use of a one-day menu 

NSCB Technical Staff response:

The official methodology multiplies the cost of a one-day menu by 365 to arrive at an annual food threshold. The fear is that if there is something wrong with the formulation of the menu, the error is multiplied 365 times. A possible area for improvement, therefore, is the use of a set of menus, say, for one week or one month; but this will entail a lot of additional computations because in the one-day menu (with urban and rural dimension) that NSCB is using now, there are already 6 worksheets involved in the estimation of the food thresholds. Thus, using a one-week menu, the number of worksheets per province would be multiplied seven-fold, i.e., 42 worksheets per province and this would really be tedious. While poverty statistics are very important and receive priority attention together with the national accounts, there are other activities in the NSCB work program that the staff has to undertake. Nonetheless, this is an aspect of the methodology that will be given attention for its improvement. 


C. Comparability of estimates across space 


Issue: Comparability across space is lost when different menus are used for different regions/provinces. The use of only one menu (a national menu) has been suggested. 

NSCB Technical Staff response:

As already mentioned, the menu only serves as a tool or an artifice used to give a value to the food requirements based on the same set of energy and nutritional standards for all regions/provinces. It is definitely incorrect to say that different standards are being imposed on different regions/provinces! In the process of valuing these requirements, the choice of the food basket comprising the menu is made by identifying those food items satisfying the nutritional requirements but at the same time are cheap and are available in the region. This consideration enhances the validity of the food threshold obtained – that it corresponds to the minimum income required to be able to purchase the food items satisfying the nutritional standards. If one uses a national menu, or a national food basket that is specific rather than generic (for instance, the basket identifies the specific kind of fish required), the validity of the food threshold as a minimum income required to satisfy nutritional standards is almost certainly lost. Such approach will result in biased estimates of the food threshold, and consequently of the poverty threshold. In places where the nutritional requirements can be met by buying food items that provide these nutritional requirements but are different from and are cheaper than the ones specified in the national menu/food basket, the food threshold would have been overestimated by using a national menu or a national food basket, i.e., an upward bias on the estimate of the food threshold would have resulted. Similarly, in places where a food item in the national menu/food basket is not available in the province/region and has to be substituted by a more expensive food item in order to satisfy the nutritional standards, the food threshold would have been underestimated if the food item in the national menu is used (downward bias on the food threshold)! In this regard, since the approved methodology is based on a regional menu, an even better methodology under the considerations elaborated in this section would be to use provincial menus!


D. Capturing the Different Dimensions of Poverty


Issue: There are other aspects of poverty not captured by the official poverty statistics 

NSCB Technical Staff response:

The NSCB recognizes that poverty is a multi-dimensional phenomenon. The official methodology is concerned with measuring the income-poor but there are other dimensions that obviously cannot be captured solely by the official statistics on poverty incidence. Hence, for effective policy making, it is necessary to look at other social indicators which have been established as correlates of poverty and which are currently available. For instance, one may look at the results of the Annual Poverty Indicators Survey (APIS) of the NSO which generates relevant non-income indicators on water and sanitation, health status and education of the family members, housing, availment of credit to finance the family business or enterprise, to name some. The Human Development Index (HDI) compiled by the NSCB is another indicator that could be looked at side by side with the poverty incidence rates. 

Still another possible source of indicators is the framework on the Minimum Basic Needs (MBN). A component of the Social Reform Agenda, the MBN approach to improve the quality of life is a strategy of prioritizing requirements to ensure that the basic needs of survival, security from physical harm, and enabling needs of the individual, family and community are attended to (UNDP & NEDA).

Based on consultations with national and local government agencies, NGO’s and people’s organizations nationwide, some indicators for 3 major concerns (survival, security and empowerment), were developed such that if a family is unable to meet these basic needs, it is considered to be in a state of poverty. Those who have “higher” levels of deprivation are recommended for poverty assistance.



For additional information on other indicators that could be looked at, the reader is referred to Section 1.4 of the paper Statistical Challenges to Poverty Reduction Strategies in the Philippines, Virola, 2002, Annex TN 200307-SS1-02-04. 


E. Other issues on the methodology


Issue No. 1: Income vs expenditure approach – The use of expenditure from the FIES rather than income is a better approach since expenditure is a better measure of welfare. 

NSCB Technical Staff response:

While indeed expenditure may be considered a better measure of welfare, its use for poverty assessment in the Philippines may not necessarily translate into more reliable poverty statistics. The questionnaire for the 2000 Family Income and Expenditures Survey (FIES) conducted by the NSO is 71 pages long, 42 pages and 1100 data cells of which are on expenditures and 11 pages and 220 data cells are on income. The respondent is asked to remember how much income/expenditure was earned/incurred during the reference period. Remembering how much income was earned during the past six months could produce less error than remembering how much was spent on various items like food, clothing and shelter, education, etc. especially when the reference period is far back in the past. The diary approach to the conduct of household income and expenditure surveys might avoid the recall problems associated with the FIES but when it was tried in 1979, the results were not even published by the NSO.

Nevertheless, the NSCB Technical Staff has done computational exercises using the expenditure approach, the results of which have been discussed with the NEDA and presented to the NSCB Executive Board. The NSCB Technical Staff will continue to explore options related to this approach. 

For additional insights on the FIES income-expenditure issue, the reader is referred to Annex TN 200307-SS1-02-05. 


Issue No. 2: The exclusion of the expenditures on alcoholic beverages and tobacco from the total basic expenditures that “imputes certain moral judgment although such may be justified by enumerating the ill effects associated with consumption of alcoholic beverages and tobacco”. 


NSCB Technical Staff response:

For purposes of defining the minimum income requirement needed so that an individual can be said to have command over the basic requirements for survival, i.e. so that this individual cannot be considered poor, is it necessary to add to the minimum income requirements amounts that are needed to buy what maybe considered as non-basic requirements such as alcoholic beverages and tobacco? If this is done, it will surely lead to an overestimation of poverty thresholds and of poverty incidence. Excluding alcoholic beverages and tobacco from the minimum income requirements is certainly not the same as prohibiting anyone from consuming these items! If the FNRI declares, however, that alcoholic beverages and tobacco are cheap sources of the various nutrients that it has identified to be basic nutritional requirements, then, obviously, they can be included in the food bundle/menu! 


Issue No. 3: Indirect estimation of non-food expenditures 

NSCB Technical Staff response:

Direct estimation, while apparently straightforward, is rarely used in practice by statistical agencies, and the more common approach is to simply scale up the food threshold by some factor which can be chosen in different ways from observed consumption patterns. This approach has been tried before but it can also lead to a lot of questions such as: (a) what is the basic educational requirement for an individual – is it primary, secondary or tertiary, or something else? (b) how will the minimum health requirements be determined? (c) will satisfying the minimum clothing requirements be in terms of buying new sets of clothes per year? how many?, etc.