Improving the CBCRP through Evaluation
California taxpayers deserve to have the funds they provide for breast cancer research spent wisely. That’s why the California Breast Cancer Research Program is conducting a multi-year, formal evaluation of the entire program. Evaluation helps the Program target research dollars where they will do the most to reduce and end the suffering caused by breast cancer. Over the past several years, the CBCRP has evaluated several of its award types: the Community Research Collaboration awards, the Postdoctoral Fellowship awards, the Dissertation awards, and the Innovative, Developmental, Exploratory Awards (IDEAs). The results of these evaluations were used by the CBCRP’s Breast Cancer Research Council to set priorities. These evaluations are available in print to the public and can also be viewed on the Program Web site.
Postdoctoral Awards Evaluation
During 2009, the CBCRP published the second evaluation of the Program’s Postdoctoral Fellowship Awards. These fellows—including graduates having recently completed their Ph.D.s, physicians continuing research activity, and individuals in transition to breast cancer research from another field—receive CBCRP financial support to obtain their postdoctoral training under a designated mentor experienced in breast cancer research. The evaluation found that these awards are meeting important goals set by the CBCRP, including increasing the pool of scientists engaged in breast cancer research. The Postdoctoral awards also allowed the fellows to leverage millions in additional funding for breast cancer research, assuring that the lines of inquiry they are pursuing will go forward in the future. The evaluation suggested possible ways to improve the Postdoctoral awards, including requiring the mentor to have documented expertise in breast cancer research.
Evaluation Leading to Improvement
The results from this evaluation and previous evaluations are contributing to the CBCRP’s current three-year priority setting process, which will be completed in 2010. Previous priority-setting evaluation processes have led to major improvements in the type of research the CBCRP funds. Examples include:
- Three evaluations of the CBCRP’s Community Research Collaboration awards led to the Program making several improvements. The CBCRP conducted a multi-year outreach and training effort that increased the number of community organizations and scientific researchers collaborating on breast cancer research questions of interest to communities of California women. Grant amounts have been increased and the application process has been made more user-friendly. For more on the CBCRP's Community Research Collaborations, see the section of this annual report titled “Collaborating with Breast Cancer Activists and California Communities.”
- A previous three-year priority-setting process led the CBCRP to invest 30 percent of its funds in the Program’s Special Research Initiatives. The initiatives are designed to answer crucial questions about the role of the environment in breast cancer, and to uncover the reasons why some groups in California bear a greater burden of the disease. For more on the CBCRP’s Special Research Initiatives, see the previous section of this report titled, “The CBCRP’s Strategy for Allocating Research Funds.”
- The CBCRP council used results from an evaluation of the Program's Innovative, Developmental and Exploratory (IDEA) awards to make these grants more effective at meeting the goal of jump-starting research that may not succeed, but has high potential to lead to breakthroughs. Researchers now apply for a smaller initial IDEA grant to test their hypothesis. If they succeed, they can apply for a larger grant to push the research forward. This change means the CBCRP is able to fund research to test more new ideas and provide additional support if they show promise.
- A past evaluation of the Program's Translational Research Collaborations found that offering this type of award was not leading to the kind of research the CBCRP had envisioned. This award required two scientists from two different research disciplines to collaborate on moving a proven discovery from basic research toward a practical application to improve breast cancer treatment, detection, or prevention. Since requiring the collaboration was not leading to projects that actually translated research findings into practical applications, the CBCRP focused instead on clarifying its definition of "translational research" in a way that is meaningful to researchers in diverse disciplines engaged in breast cancer research. Applicants for these awards are now required to identify the practical outcome of their research, describe how they are resolving barriers that have kept the research concept from being turned into a practical application, and ensure that previous research has laid the groundwork for putting the concept to practical use.
CBCRP staff and the Program's council informally evaluated how CBCRP-funded research gets translated into new medications, new detection methods, new programs to support patients, policy changes, or other actions that have an impact on breast cancer. As a result, applicants for CBCRP research grants are now required to describe the steps necessary to translate their research project into action that impacts the disease. This has enabled the CBCRP to target its limited funds toward research most likely to lead to progress against breast cancer.
