Innovative Development: Lottery Day in Eastern Mongolia

An MCC contractor displays a pingpong ball during the lottery designed to randomly select herder groups for project benefits.

Washington — Dornod province lies on the high plains of eastern Mongolia. About 40,000 of the province’s 75,000 citizens live in the capital city of Choibalsan; the rest live in villages or on the steppe. In late September 2011, Choibalsan was the site of an innovative approach to select project beneficiaries: a lottery.

As part of the U.S. Millennium Challenge Corporation’s five-year, $285 million compact with Mongolia, local governments are leasing pastureland to groups of herders. Millennium Challenge Account–Mongolia, the organization established by the government of Mongolia to implement the compact, is supplying these herder groups with wells and materials to build fences and animal shelters. Herders also receive training on livestock management, rangeland productivity, business skills and improved dairy production.

The peri-urban component is part of the Property Rights Project, one of five in the Mongolia Compact, and is expected to benefit 6,000 people and raise household incomes by about $15 million over the next 20 years, MCC explained in a press release.

The leasing program operates in five areas and aims to improve herder-group productivity and incomes through better range-management practices and the reduction of the number of livestock to a level that allows for sustainable use of the range. The program is considered a pilot effort that, if successful, could be replicated in other parts of Mongolia.

The question of how to choose who would receive land leases, training and infrastructure was crucial during project design. There were a number of possibilities: choosing herders with the best skills and experience, selecting all groups within defined geographic areas, or conducting a random selection through a lottery.

WHY USE A LOTTERY?

The lottery was chosen to promote fairness and because it allows for an effective statistical evaluation of the leasing program’s impact, MCC said. Other evaluation approaches have limitations that compromise their ability to draw accurate conclusions about impacts. For example, evaluations that only compare results achieved by the group of beneficiaries before and after a project cannot account for what would have happened if the project had not been implemented.

In the case of Mongolia, the rapidly expanding economy and the strains on herder income because of the harsh winter storm of 2009–2010 severely distorted the ability to measure impacts of the leasing program using a before-and-after approach. Even evaluations that try to compare a beneficiary group to a nonbeneficiary group with similar characteristics (such as income, age or education) are not able to account for other characteristics that are difficult to observe and measure but affect the results (such as entrepreneurship or motivation). By randomly assigning a sufficiently large number of the target population into control and treatment groups, each group will have similar characteristics — whether or not they are identified — and the observed results can be attributed to the leasing program.

The use of a lottery is also fair. Outside parties have a difficult time interfering with or changing the results of a properly designed lottery conducted in a public setting. The selection of participants using objective criteria, such as skills, wealth or years of experience, presumes these factors are reliable indicators of herding success. And while sometimes these criteria can be reliable, the differences in criteria between potential leasing program participants can be small and do not provide clear guidance. Subjective indicators such as leadership ability are important but can be hard to quantify.

SELECTING THE BENEFICIARIES

The lottery was put to the test for the first time on September 29, 2011, in Choibalsan; herders filled the main auditorium of the city’s technical school for the selection. Almost 120 herder groups met the project's criteria for participation, and 60 of them were selected to receive leases, training and infrastructure.

The lottery was a series of eight drawings: seven organized by each local district for traditional seminomadic herders and one drawing organized for a special group of dairy farmers. This process ensured herder groups from each district and some dairy farmers would be selected.

A clear plastic box sat on a table on the auditorium stage. The master of ceremonies read participants’ names for each drawing and announced how many would be chosen. As each group’s name was announced, a pingpong ball with that group’s assigned number was placed into the box. Each ball was shown to the audience, and a video projector enabled everyone in the auditorium to see the number.

Once the balls were in the box, the person holding the box rotated it several times, placed it on the table and opened a small door to release a ball. The ball was held up for all to see and projected onto the large screen, eliciting a cheer from the winning group.

Half of the audience went home disappointed, but they accepted the results. Many congratulated the winners. This acceptance is a testament to the effective organization and implementation of the lottery and to the several months of advance work to explain the selection system. This work included managing expectations so everyone who entered the auditorium knew that they — just like everyone else in attendance — had a 50-50 chance of being selected.

The day was a success — not only for the lottery winners, but also for fairness and transparency, the MCC said.

Mongolian herders watch the lottery to learn if they’ve been selected for land leases, training and infrastructure.