Back Matter
  • 1 https://isni.org/isni/0000000404811396, International Monetary Fund

Appendix: Details of Simulations Estimating Targeting Implications of Universal Knowledge

In this appendix we describe the methodology used to simulate the implications of universal knowledge for application and acceptance outcomes. In the text we represented the outcomes observed in our data through the following participation process:

P(PART)=P(KNOW). P(APPLY|KNOW).P(ACCEPT|KNOW,APPLY)

where P(PART) is the unconditional probability of being observed participating in the program, P(KNOW) is the unconditional probability of knowing about the program, P(APPLY|KNOW) is the probability of applying conditional on knowing, and P(ACCEPT|KNOW, APPLY) is the probability of being accepted into the program conditional on knowing about the program and applying for the program.

We are interested in estimating the probability that a household that knows about the program will apply, i.e., P(APPLY|KNOW). One approach would be to regress the observed application outcome on household characteristics for those households that reported knowing, and then applying this model to households that didn’t know. However, to the extent that knowledge acquisition reflects household decisions, one expects that those households that report knowing about the program are a selected sample, e.g., that they have unobserved characteristics that make them more likely to apply. Therefore, estimating P(APPLY|KNOW) on the sample of those reporting knowing is likely to give an upwardly biased estimate of P(APPLY|KNOW) so that some correction for this bias is required.

One can think of the knowledge acquisition stage as being determined by household decisions based on the perceived costs and benefits of acquiring knowledge as well as by the program implementation strategy, e.g., the advertising strategy, the latter being exogenous to household decisions.16 Similarly, one can think of a household’s decision to apply as reflecting the perceived costs and benefits of applying. If one controls for the benefits a household would receive if accepted, then the coefficients on the remaining variables can be interpreted as capturing differential costs of applying. However, the problem is that the variables that determine the cost of acquiring knowledge are likely to also determine the cost associated with applying. To correct for possible sample selection bias one needs to have some independent variation in the knowledge (selection) equation, i.e., a variable that affects the probability of applying only through its impact on the probability of acquiring knowledge. Information on the intensity of advertising in the block in which a household resides (e.g., resources devoted to advertising) would be an obvious candidate for such a variable. Although we do not have such information we do know that advertising was concentrated in blocks with the highest poverty rates as estimated by the program’s proxy means algorithm. Therefore, we include the block poverty rate as an explanatory variable in the knowledge regression equation and expect higher poverty to be associated with higher knowledge (via a greater concentration of advertising resources on these blocks).

The first four columns of results in Appendix Table 1 present the regression results of a Heckman sample selection estimation approach. The results are from a two-step estimation of a linear probability model. This provides consistent estimates and facilitates easy interpretation of results but it was also the case that more complex full-information models based on this model and a probit model failed to converge, presumably reflecting the low percentage of households that do not apply conditional on having knowledge. As expected, the coefficient on the block poverty rate in the knowledge equation is positive and highly significant, indicating that poor households living in non-poor blocks are less likely to know about the program. The significant coefficient on the inverse Mills’ ratio reinforces our concerns regarding the need to correct for sample selection bias.

Appendix Table 1.

Results for Conditional Application and Acceptance Outcomes and Consumption Model

article image
Note: Transfers are in thousands of pesos. Consumption quintile dummies were included in the Heckman acceptance regression but were all insignificant.

The predicted probabilities from this model are used to identify households that are likely to apply for the program conditional on universal knowledge. Appendix Table 2 presents the implications for application outcomes. When we use the traditional 0.5 probability as a threshold for identifying those households that apply we find that all households apply reflecting the fact that all households have a predicted probability above 0.6. This is not unusual in samples with a small percentage of households not applying. One approach in such circumstances is to use the average application rate for the selected sample as the cutoff threshold (see Wooldridge 2005 for further discussion)—the second column of Appendix Table 2 shows that 88.6 percent of households that knew actually applied. Using 0.886 as our threshold we find that 89.2 percent of the sample apply. However, one expects that those that previously did not acquire knowledge are less likely to apply compared to those that did. This pattern is consistent with our estimates, which show that the 81 percent of the latter applied compared to 95 percent of the former. Thus, the fact that the proportion of applying across all households increases from 87 percent to 89 percent is worrying. We therefore use an alternative approach that chooses the threshold such that the proportion of households that previously knew (i.e., for which KNOW=1 pre reform) that applied post reform is the same as the proportion that applied pre reform (i.e., 88.6 percent). This threshold was derived as 0.908 by trial and error. Under this approach, the proportion of all households that apply under universal knowledge is 81 percent and the pattern across subgroups is as expected with nearly 91 percent of those that previously applied now applying and only 68 percent of those who previously did not acquire knowledge now applying.

Appendix Table 2.

Application Outcomes Under Universal Knowledge

article image
Note: KNOW=1 refers to those households that in the pre-reform situation knew about the program. APPLY=1 refers to those households that in the pre-reform situation both knew of and applied for the program.

The issue of sample selection bias also arises when estimating the acceptance probability. We view acceptance as being based on administrative selection. We know from discussions with program officials that since a larger number of households that turned up at the program offices were classified as poor under the proxy-means test than was budgeted for, program officials had to use additional criteria to ration program places. One obvious candidate for rationing is the score itself—among those classified as poor under the proxy-means system, one selects households with the highest poverty score. We therefore include quintile dummies based on this score, classifying households as extremely poor, moderately poor, quasi-poor, quasi-non-poor, and non-poor to allow for strong nonlinearities. Another possibility is that program officials attach a higher weight to components of the score than is implicit in the score itself. To capture this we include the variables included in the score. Other household characteristics associated with the overall motivation of the program (e.g., having infants or children of school-going age, or being classified as indigenous) are also included. In addition, we include as an additional regressor a variable capturing the proportion of poor households visiting the program office that received a verification visit. According to program officials, once the quota for beneficiaries was reached, verification visits were suspended so this variable should pick up the intensity of the budget constraint. In the unconditional application selection equation we include the same variables as before but now also include the instrumental variable used in the knowledge selection equation above since the unconditional application outcome depends on both the knowledge acquisition outcome and the conditional application outcome, i.e., P(APPLY)=P(KNOW)P(APPLY|KNOW). Thus P(APPLY) is estimated on the full sample to get an unbiased estimate of P(ACCEPTAAPPLY) for all households. Note also that the level of benefits a household would receive if accepted is included in the selection (application) equation but not the acceptance equation reflecting the fact that acceptance is based on administrative selection and not on household decisions. This variable thus also acts as an additional instrumental variable in the application selection equation.

Columns 5–8 in Appendix Table 1 present the results of a Heckman two-step linear probability acceptance regression. In the application selection equation the coefficient on the block poverty rate is again positive and highly significant. Similarly, the effect of transfers is significantly positive and decreasing. The probability of being accepted is much higher for those with lower compared to higher proxy-means scores consistent with the score being used to ration program places in the presence of a tighter budget constraint. The coefficient on the number of children of primary school age is also significantly positive and the probability of being accepted is also negatively associated with housing conditions as captured by the number of rooms suggesting a higher weight being given to this by program officials than under the score. Note, however, that the inverse Mills’ ratio is significant only at around the 90 percent level.

To identify households that are accepted under the reformed universal knowledge program, we assume that the program size is still binding at 30 percent of the population. The 30 percent of households with the highest probability of being selected (conditional on applying, which determined by the earlier simulation of P(APPLY|KNOW)) are assumed to participate so that newly applying households can now “bump out” households that were previously selected. We also compare this to a situation where only the proxy-means score was used to select households. Finally, we compare both of these to selection based on an alternative proxy-means algorithm based on the consumption regression presented in the final two columns of Appendix Table 2.

References

  • Atkinson, Anthony, 1970, “On the Measurement of Inequality,” Journal of Economic Theory, 2: pp. 24463.

  • Blundell, Richard, Victoria Fry, and Ian Walker, 1988, “Modelling the Take-up of Means-tested Benefits: The Case of Housing Benefit in the United Kingdom.” Economic Journal, 98:5874.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Coady, David, 2006, “The Welfare Returns to Finer Targeting: The Case of the PROGRESA Program in Mexico,” International Tax and Public Finance, 13:21739.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Coady, David, and Rebecca Harris. 2004, “Evaluating Transfer Programs Within a General Equilibrium Framework,” Economic Journal, 114:77899.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Coady, David, Margaret Grosh, and John Hoddinott, 2004a, “Targeting Outcomes, Redux,” World Bank Research Observer, 19(1):6185.

  • Coady, David, Margaret Grosh, and John Hoddinott, 2004b, The Targeting of Transfers in Developing Countries: Review of Lessons and Experiences. IFPRI and World Bank: Washington DC.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Coady, David, and Emmanuel Skoufias, 2004, “On the Targeting and Redistributive Efficiencies of Alternative Transfer Instruments,” Review of Income and Wealth, 50(1):1127.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cowell, Frank, 1986, “Welfare Benefits and the Economics of Take-up,” TIDI Discussion Paper no. 89, STICERD, London School of Economics.

    • Search Google Scholar
    • Export Citation
  • Duclos, Jean-Yves, 1995, “Modelling the Take-up of State Support,” Journal of Public Economics, 58:391415.

  • Deaton, Angus, 1997, The Analysis of Household Surveys: A Microeconomic Approach to Development Policy, John Hopkins University Press: Baltimore.

  • Deaton, Angus., and Cristina Paxton, 1998, “Economies of Scale, Household Size and the Demand for Food,” Journal of Political Economy, 106(5):897930.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Heckman, James, and James Smith. 2003. “The Determinants of Participation in a Social Program: Evidence from a Prototypical Job Training Program,” IZA Discussion Paper no. 798, Bonn, Germany.

    • Search Google Scholar
    • Export Citation
  • Lanjouw, Peter, and Martin Ravallion, 1995, “Poverty and Household Size,” Economic Journal, 105:141534.

  • Martinelli, Cesar, and Susan Parker, 2006, “Deception and Misreporting in a Social Program,” Forthcoming in the Journal of the European Economic Association.

    • Search Google Scholar
    • Export Citation
  • Micklewright, John, Aline Coudouel, and Susan Marnie, 2004, “Targeting and Self-targeting in a New Social Assistance Scheme,” Discussion Paper no. 1112, Institute for the Study of Labour (IZA), Bonn, Germany.

    • Search Google Scholar
    • Export Citation
  • Moffit, Roger, 1983, “An Economic Model of Welfare Stigma.” American Economic Review, 73(5):102335.

  • Pudney, Stephen, Maria Hernandez, and Ruth Hancock, 2002, “The Welfare Cost of Means-testing: Pensioner Participation in Income Support,” Unpublished manuscript, Department of Economics, University of Leicester.

    • Search Google Scholar
    • Export Citation
  • Ravallion, Martin, 2008, “How Relevant is Targeting to the Success of an Antipoverty Program?Forthcoming in World Bank Research Observer.

    • Search Google Scholar
    • Export Citation
  • Skoufias, Emmanuel, Benjamin Davis, and S. de la Vega. 2001, “Targeting the Poor in Mexico: An Evaluation of the Selection of Households into PROGRESA,” World Development, 29(10):176984.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wooldridge, John, 2005, Introductory Econometrics: A Modern Approach. South-Western College Publications.

1

We thank Iliana Yaschine, Bernardo Hernandez and Citlalli Hernandez for very helpful clarifications on the program as well as untiring assistance with our data questions. We are also very grateful for comments received.

2

Self-selection involves households or individuals making decisions on whether or not to apply for a program based on their perceived private costs and benefits of participation. Administrative targeting involves government officials deciding who among those applying should be deemed eligible based on household socio-economic characteristics. For a more detailed discussion of alternative forms of targeting, see Coady, Grosh, and Hoddinott (2004b).

3

For developed countries see Heckman and Smith (2003), which combines data from a number of different sources to investigate the sources of inequality of participation among different groups of eligible individuals for the Job Training Partnership Act in the USA. However, data limitations resulted in both application and acceptance outcomes being combined into a single stage.

4

See Skoufias, Davis and de la Vega (2001) and Coady (2006) for more detailed discussions of the design of Progresa and an evaluation of its targeting performance. Coady, Grosh and Hoddinott (2004) find that Progresa was of the better targeted transfer programs in their sample, along with many other similar programs in Latin America. Note that our evaluation of the targeting performance of Oportunidades focuses on targeting of households “within” the urban areas where the program expanded into, and not on targeting relative to the national income distribution.

5

For our analysis we derive the sampling weights by merging all CENSUS households with SAMPLE households and comparing populations in the sampling categories. We focus on the SAMPLE survey since only this collected consumption data and data on the targeting process (i.e., knowledge, application and acceptance). The sample weights are adjusted for nonresponse in the SAMPLE based on the above four stratified sampling categories - of the 10,527 sampled households, only 9,817 households completed the SAMPLE questionnaires.

6

See Coady and Skoufias (2004) for a more detailed discussion of this approach.

7

Fixing the budget enables us to avoid the issue of how the marginal cost of funds changes with the size of the budget. We are essentially assuming that the budget is financed in a given way and is independent of how the budget is spent. For an empirical example of incorporating the cost of public funds across alternative financing instruments, see Coady and Harris (2004).

8

Note that from a social welfare perspective, since households are assumed to optimally choose their allocation of time between labor supply (or consumption) and leisure, we are indifferent between how households “consume” the transfer and thus can ignore the income effect on labor supply. We also abstract from the issue of disincentives (e.g., where households change labor supply in order to become eligible for the program or devote resources to increasing their probability of being deemed eligible through deceit or lobbying).

9

This is strictly only true for “small” transfers such that welfare weights are approximately constant. More generally, equation (2) needs to be multiplied by an additional term capturing the impact of changing welfare weights. This term will be maximized for the optimal transfer program that differentiates transfers across households such that incomes are equalized at the bottom of the income distribution (subject to a budget constraint and only households with incomes below this new “minimum” income receiving transfers).

10

Note that it is logical to evaluate the stages in sequence (i.e., NEUTRAL then KNOW then APPLY then ACCEPT) given the inherent sequential nature of the participation process and the fact that we expect targeting to improve across stages. But clearly one could give demographically differentiated transfers at all stages instead of uniform transfers. However, this would mean that the transfers under ACCEPT would be the same as those under DEMOG. Since we expect (and find) that moving from uniform to demographically differented transfers increases targeting performance, comparing ACCEPT (=DEMOG) to a uniform transfer to accepted households would result in a decrease in targeting performance at this final stage. Therefore, given our focus participation outcomes and on the relative contribution of each stage to targeting performance, it makes more sense for us to assume uniform transfers for the first three stages (KNOW, APPLY, ACCEPT) followed by a shift to differentiated transfers at the final stage (DEMOG).

11

For uniform transfer programs of equal size in terms of number of beneficiaries, improved targeting performance comes automatically with improved coverage of the target poor population. More generally, for a fixed budget, improved targeting performance can be associated with either lower or higher coverage of the poor since these can be offset by a differences in the structure of transfers.

12

Throughout the paper we will use “income” to denote our adopted household welfare measure (i.e., per capita consumption) and “welfare” to denote social welfare.

13

For example, if we choose as our reference a program that was better targeted than NEUTRAL, then the improvement in targeting performance under KNOW would be smaller. It is also the case that targeting based on knowledge may not be desirable in that one may not want participation to be determined by ignorance. Since the qualitative nature of the results do not change when we use KNOW as our reference, we discuss the results when NEUTRAL is our reference because it facilitates a discussion of the implication of knowledge for targeting outcomes.

14

See Ravallion (2008) for a discussion of the targeting differential and how it relates to other indicators, and Coady and Skoufias (2004) for a comparison of the index used in this paper with other similar indicators.

15

For a constant program size, N, there is obviously a well defined relationship between the PPR and the NPR, with a unit increase in the PPR implying a decrease in NPR equal to the ratio of the share of the poor in the population to the share of the non-poor (or, equivalently, the poverty rate divided by one minus the poverty rate). Therefore, for such a program, welfare will always increase with PPR.

16

For more detailed discussions on modelling program participation decisions within a cost-benefit framework, see Moffit (1983); Cowell (1986); Blundell, Fry and Walker (1988); Duclos (1995); and Pudney, Hernandez and Hancock (2002).

Targeting Social Transfers to the Poor in Mexico
Author: Mr. David Coady and Susan Parker