Lean Research: Rigorous, Respectful, Relevant, and Right-Sized Research in Development

Lean Research is a framework to guide and improve the practice of field research with people and communities in the contexts of international development and humanitarian work. Research in these contexts is often conducted to understand and improve the impacts of various program interventions in the lives of populations facing poverty, vulnerability, and other challenges. Yet the impact of research activity on the lives of research subjects, communities, and local partners is often ignored.

Lean Research emerges from the work of scholars, practitioners, and donors who recognize research as a form of intervention into the lives of study subjects, particularly those experiencing poverty and vulnerability, and who call for this intervention to produce benefit and positive impact for those involved. By incorporating the principles of rigor, respect, relevance, and right-size into the research process, Lean Research seeks to minimize burden on research subjects while maximizing the value of both the research process and outputs to stakeholders.

Drawing from human-centered approaches to development and design, Lean Research places the experience of the human “research subject” at the center of decisions about research design and implementation. By creating a respectful and enjoyable experience for human subjects in the context of research questions that are relevant to key stakeholders, including participants, Lean Research seeks to increase the quality of information gathered through research, improve the usefulness of research findings for stakeholders, and enable both the research process and outputs to benefit study subjects and their communities, as well as donors and decision-makers.

The Lean Research Framework

Lean Research offers a guiding framework for conducting research and evaluation built upon four principles of good research practice. In order for research to reduce burden on participants and maximize value for stakeholders, it should be:

1. Rigorous, regardless of methodologies employed
2. Respectful towards research subjects, implementing partners, and others engaged in the research process
3. Relevant to research subjects, partners, and decision-makers
4. Right-sized, in terms of protocols and costs compared to the potential usefulness and impact of the study

The four principles of Lean Research are not new, but are often pitted against each other as trade-offs. Lean Research emphasizes conducting research in ways that reflect and exemplify all four principles and challenges researchers to identify opportunities to implement them in an integrated fashion. As a broad framework and approach to social science research, Lean Research can be applied regardless of whether the methods are quantitative, qualitative, or mixed.

Drawing lessons from lean manufacturing as well as human-centered design, Lean Research focuses on continual improvement of the research process, particularly at “touch points” where researchers interface with research subjects, local partners, and others engaged in research production. It seeks to improve research outcomes by improving the experience of those most directly involved in research production—particularly the research subjects—and by orienting the process around their insights and priorities. It challenges researchers to improve upon existing practice and opens a space to innovate new ways of approaching familiar steps in the research process.

Lean Research Principles

Rigor

Lean Research is conducted according to the highest standards of the research methodology that is best suited to the specific nature of the study. Research must adequately address issues of both internal and external validity and ensure accurate reporting of results while protecting sensitive subject data. High standards of rigor ensure the integrity of the research process and results, a pre-condition for research that is respectful of participants’ time and usable by research stakeholders.

Respect

Respectful research places the dignity and delight of the human subject at the center of the research experience. It offers a clear, intelligible informed consent process, in which research subjects feel truly free to reject participation without fearing negative consequences. If they decide to participate, subjects find the experience enjoyable and meaningful. Subjects have the opportunity to review and refute research findings and feel that their contributions to the research have been appropriately valued.

Relevance

Relevant research has clear value to stakeholders and addresses priority issues and questions for research subjects, study communities, as well as donors and decision-makers. Research findings are understandable and accessible to research subjects, practitioners, and policymakers. Research studies and results are framed in ways that can inform action and decision-making at various levels of authority, and stakeholders commit to use findings to inform action.

Right-Size

Research is right-sized when the research scope and methods are well-suited to the research objectives and the priority of the research questions to stakeholders. Right-sized research is only as time-consuming, burdensome, and costly as it needs to be, and all unnecessary questions, activities, and protocols are removed.

Case Example: Root Capital

Root Capital, a social investment fund serving rural farming communities, conducts program evaluations to understand and improve client outcomes and service delivery. While data from evaluations is used to improve programs, participating in the data collection process takes time from entrepreneurs who need that time to maintain and improve their businesses, the objective Root Capital is working to support. After clients expressed reluctance to continue participating in evaluations, Root Capital staff realized that “too often, data collection for impact evaluations, regardless of the intent, feels extractive to research participants.”

In response, the evaluation team now integrates questions that clients propose into their research process. As they conduct program assessments, staff members now generate data that enables lending clients to make more informed business decisions. Encouraged by the value this offers to the enterprises that Root Capital serves, their impact team has overhauled their approach to program evaluation and is developing a “client-centric” model that aligns with Lean Research principles.

From A Client-Centric Approach: Impact Evaluation that Creates Value for Participants by Michael McCreless (Root Capital Working Paper, June 2015).

Guiding Questions for Conducting Lean Research

Lean Research does not provide a set of rules to follow, but rather a guiding orientation to encourage innovation and continual improvement in research practice. From the way in which research questions are selected through implementation and dissemination of findings, there are opportunities to better align the research process with principles of rigor, respect, relevance, and right-size.

Is Our Research Rigorous?

1. How do we know that our research adheres to the highest standards of our discipline or field of practice with regard to research and instrument design, data collection, cleaning, and analysis?
2. What steps are we taking to ensure the internal validity of the research?
3. If applicable, what steps are we taking to ensure the external validity of the research?
4. How are we designing and implementing our research process to ensure that the research is reproducible?
5. What steps will we take to clearly, accurately, and transparently report all relevant research results to stakeholders?
6. How are we protecting the data of the people who participate in the research?
7. If the research is an impact evaluation or trial, is it registered with AEA’s social science registry? If the research is a Random Control Trial, is it registered with 3ie’s RIDIE?
8. Will the research be reproduced or verified by an independent party? If there are no current plans for this, is the research conducted in a way that it can be easily verified?

Is Our Research Respectful?

1. What are we doing to engage the research subjects, members of their communities, or similar populations (where appropriate) in the design of our study and our informed consent process?
2. How are we designing the informed consent process to ensure that research subjects receive all the information that they need in a way that is understandable to them in order to decide if they wish to participate in the research or not?
3. What actions are we taking to ensure that the human subject feels truly free to reject participation in the study or to drop out of a study once it has started without fearing or experiencing negative consequences?
4. What actions are we taking to create an environment in which research subjects can enjoy and find meaning in the experience of participating in research?
5. Are we appropriately using existing information and knowledge that local host institutions may have?
6. Have we determined culturally appropriate forms of compensating subjects and host institutions for their time and expenses?

Is Our Research Relevant?

1. What secondary research have we done in order to assure us that primary research on the topic we are proposing is actually needed?
2. What process are we using to identify the research priorities of the research subjects and their communities?
3. What steps are we taking to understand what aspects of the research local host institutions find most relevant?
4. Have we identified stakeholders in advance of the research project who have given input into how they would like to receive and use research findings?
5. Are the research subjects and host institutions able to clearly articulate the value of the proposed research study?
6. What steps will we take to communicate and share the research findings in ways that are understandable and accessible to all stakeholders?
7. Have we allocated time and budget to the process of disseminating research results?
8. Have decision-makers agreed or expressed interest in using research findings in advance?
9. Are we planning to share de-identified study data?
10. What approach will we use to understand the impact that the research has had?

Is Our Research Right-Sized?

1. What criteria are we using to assess how large and costly it is reasonable for the study to be?
2. How are we assessing which activities and questions are essential?
3. Have we determined the acceptable interview length from the perspective of participants?
4. If the research involves sampling, how are we selecting the sample?
5. How are we planning to train and compensate enumerators?
6. How are we ensuring that enumerators experience the research process as respectful and meaningful?
7. What steps will we take to provide study subjects with opportunities to review and refute findings?

Institutional Credits

The Lean Research initiative is jointly organized by faculty and researchers at:

• Massachusetts Institute of Technology (MIT)
• Tufts University

Contributors:

Jenny Aker (Tufts University)

Laura Budzyna (MIT D-Lab)

Alex Counts (Grameen Foundation)

Benjamin Crookston (Brigham Young University)

Chris Dunford (Freedom from Hunger)

David Evans (World Bank)

Rachel Gordon (Tufts University)

Bobbi Gray (Freedom from Hunger)

Ann Hastings (Microfinance CEO Working Group)

Roxani Krystalli (Tufts University)

Michael McCreless (Root Capital)

Pedro Reynolds-Cuéllar (MIT)

Rebecca Smith (MIT)

Guy Stuart (Microfinance Opportunities)

Asya Troychansky (Root Capital)

Julie Zollman (Bankable Frontier Associates)

Elizabeth Hoffecker Moreno (MIT)

Kendra Leith (MIT)

Kim Wilson (Tufts University)