Improving the CBCRP through Evaluation
California taxpayers deserve to have the funds they provide for breast cancer research spent wisely. That’s why the California Breast Cancer Research Program continually evaluates our work. Evaluation helps the Program target research dollars where they will do the most to reduce and end the suffering caused by breast cancer.
Evaluating the CBCRP's Funding Strategy
During 2010, the CBCRP completed a multi-year, formal evaluation of the entire program that led to a new funding strategy for the coming five years. This evaluation began with the CBCRP's highest decision-making body, the Breast Cancer Research Council, asking the question, "How can we best leverage California resources to make an impact on breast cancer?"
The council engaged in an intensive data collection and evaluation process that included:
- Conducting literature searches and interviews with scientists the CBCRP has funded to identify the impact that CBCRP-funded research has had on breast cancer research;
- Analyzing the ways in which CBCRP-funded research has leveraged funding for breast cancer research from other sources and increased breast cancer research expertise;
- Comparing the CBCRP's research portfolio with those of other breast cancer research funding agencies, nationwide and around the world, to determine where there may be overlap;
- Conducting surveys with breast cancer advocates, breast cancer researchers, breast cancer health professionals and policy makers, and combining the survey results with analysis of California breast cancer statistics to identify opportunities for the CBCRP to fill critical gaps in research.
The evaluation revealed that the CBCRP has been highly effective. The Program has succeeded in funding California-specific research, including research that involves the state's diverse communities. The CBCRP has also made strides in probing areas of breast cancer research that have not been well-studied. In addition, the CBCRP has funded research that has improved access and services for populations of California women who have lacked access and services. The Program has stimulated innovation in breast cancer research, leveraged funds from other sources, and impacted policy around health education, health information, and the environment-breast cancer connection.
The evaluation also showed that the CBCRP was falling short in meeting some Program goals. Despite strong support for research into breast cancer prevention by advocates, scientists, and the public, the CBCRP has not been able to fund much research in this area. A second shortcoming was that some of the grants that the CBCRP has been making duplicate those offered by other research funding agencies.
The context for this evaluation included the CBCRP's declining source of revenue. In 2011, the Program's revenue is expected to drop 15% from the average revenue of the past five years.
The Breast Cancer Research Council used this evaluation to develop a bold new strategy for putting breast cancer research in California in the forefront of the field. The strategy, which will guide the next five years of grant-making, focus CBCRP resources in areas where the program has had the most impact, and increase research in the crucial and under-studied area of breast cancer prevention. For more on the new funding strategy, see the sections of this annual report titled "The CBCRP's Strategy for Allocating Research Funds" and "Answering Urgent, Neglected Questions: Program Initiated Research." For a fuller explanation of both the evaluation process and the new funding strategy, see "New Funding Strategy for the California Breast Cancer Research Program: The Way Forward," a publication available on the CBCRP Web site.
Evaluating the Special Research Initiatives Process
During 2010, the CBCRP evaluated the process used to select research for the first ten Special Research Initiatives. This process involved an in-depth literature review of previous research, guidance from committees of national experts in the breast cancer field, a survey of California resources that might be used to conduct breast cancer research, and opportunities for the public to submit research ideas. The goal of this process was to select research projects that would push forward progress in two areas:
- The environmental causes of breast cancer;
- The reasons why some groups of women are more likely to get breast cancer, or die from the disease, based on characteristics such as their race, ethnicity, or where they live.
The evaluation revealed that the Special Research Initiatives research project development process had worked quite well to fund critical, neglected areas. Still, there is room for improvement in developing future program initiated research. Recommendations included:
- Continue a process to identify crucial, under-researched questions;
- Clarify the roles of committees of experts in the breast cancer research field;
- Consider alternative methods for involving breast cancer advocates and the public in the process
- Allow more time for the CBCRP's Breast Cancer Research Council members to review and consider recommendations for research projects.
One immediate result of the evaluation is that the CBCRP designed and funded the Partnership to Advance Breast Cancer Research grant to carry out an intensive, but streamlined process for developing future program-initiated research. The CBCRP is continuing to evaluate the Special Research Initiatives priority setting process and will find additional ways to optimize it.
Additional CBCRP Evaluations
In the past several years, the CBCRP has conducted evaluations on components of the Program. Results from these evaluations have been used to improve the CBCRP. For example, three evaluations of the CBCRP's Community Research Collaborations have led to changes in the way we make these grants. These changes have increased the number of community organizations collaborating with scientists to research questions of interest to communities of California women. Results of past CBCRP evaluations are available in print and on the CBCRP Web site.
