Update: Pilot Testing the SISC Metrics
With funding from the USDA-NRCS Conservation Innovation Grant, the Stewardship Index for Specialty Crops is ready to begin the process of pilot testing some of the sustainability performance metrics drafted by the Metrics Review Committee workgroups (MRC Workgroups). The purpose of this memo is to inform all SISC Metrics Review Committee participants of our progress to date in developing appropriate metrics for the specialty crop supply chain and to outline the expectations, process and anticipated outcomes of the pilot-testing phase of our work together.
The Draft Metrics (Release 1.0) that are ready for pilot testing have been posted to the SISC website for your review. In an effort to get your feedback on the current version of the Draft Metrics, and to evaluate the metrics development process used to date, the Steering Committee will be emailing a survey to all Metrics Review Committee participants in the next few days. We hope you’ll take the opportunity to assess our progress to date and offer your input as we move forward.
Our Work to Date
The Stewardship Index for Specialty Crops was launched in December of 2008 with the goal of constructing a standard method for measuring sustainability performance across the supply chain for specialty crops. In the wake of food safety efforts that led to ‘audit exhaustion’ when growers try to comply with multiple buyer requirements, specialty crop industry leaders decided to work together to pro-actively create agreement on yardsticks for measuring sustainability that will be accepted by multiple buyers.
Since the metrics development process started in February 2009, 43 webinar meetings across 9 metric areas have taken place. More than 420 persons have signed up to participate in the Metrics Review Committee (MRC) process with varying degrees of involvement. Some participants have chosen to passively monitor developments, while others are actively engaged in the metrics development in the areas of highest interest to them and their company or organization. Some metric areas have higher participation (pesticides) than others (air quality).
The draft metrics are now ready to proceed to pilot testing in 8 of the 11 metric areas: Energy; GHG (non-farm only); Air Quality; Pesticides (on-farm only); Water Use; Soil, Nutrient & Water Quality; Biodiversity; and Waste. The remaining metrics are still under development.
Piloting the Draft Metrics
The Draft Metrics were developed by relying on the best available science and the experience of participants in each of the MRC Workgroups. A lot of good thoughts and ideas were contributed to the effort. These ideas went to the Coordinating Council in July 2009 for approval to be piloted. Important questions about the real world applicability and value of these tools to growers and their supply chain partners remain to be answered. This is why it is critical to put the draft metrics to the test through the process of pilot testing. How will these metrics hold up in the real world of farming and non-farm operations?
A pilot test is a key step in any scientific process of invention. The idea of ‘pilot testing’ comes from aviation and refers to the job of ‘test pilots’ who would fly newly designed or modified aircraft according to a flight plan whereby the results could be measured and the design could be evaluated and improved. In the same way, we now need to give these metrics a ‘test flight’ in the fields and operations by our ‘test pilots’ and ask them for feedback in order to improve the metrics design.
What we hope to learn from the pilot tests:
The pilot testing phase of the metrics development is all about learning what works and what doesn’t. It is a time for us to be open-minded, curious, inquisitive and challenging of our assumptions. Pilot testing is a time for exploration and discovery and requires tolerance of ambiguity that is inherent in an unprecedented and complex undertaking such as the SISC.
The SISC Pilot Program is focused on learning the answers to these key questions:
- Data Availability: How readily available is the data for completing the metrics? Are growers currently collecting the data? How is the data collected? Is it easily accessible?
- Practicality: Do the metrics make practical sense to growers?
- Usefulness: Will the metrics provide growers with useful information to better monitor and manage their farming operation?
- Feedback: What other concerns, ideas and feedback do growers have about the sustainability metrics? What is the risk that the metrics will create unintended consequences or undesired incentives? Will the metrics address the needs of their buyers?
Pilot process and timeline:
SureHarvest staff have begun to meet with pilot participants in the pilot test to review the Pilot Binder (login to SISC website, see “Pilot Information” tab). These meetings are a way to “pilot the pilot” to be sure that staff have prepared the pilot process appropriately before data collection begins. The pilot process is overseen by the Pilot Committee, under the direction of the Coordinating Council.
The timeline will be different for each crop, however, the plan is for data to be collected through the 2010 cropping season. SureHarvest will collect the data sheets and surveys beginning in October with a plan for compiling data by November (or sooner) and reporting back to participants in December. The Coordinating Council will then evaluate the pilot test results, assess and share the lessons learned with MRC Workgroups, determine next steps for improving the metrics, give guidance to the MRC Workgroups and make a plan for 2011. Because the timing is different for each participant, some participants may receive updates as additional metrics are included or metrics are refined according to earlier feedback.
We want to recognize the following organizations for their time, energy and leadership in serving among our test pilots! While there will ultimately be nearly 100 pilot participants, we are publicly listing these organizations because they are providing in-kind support as cooperators in the USDA-NRCS Conservation Innovation Grant funding the pilot tests.
- Stemilt Growers
- SunWest Fruit Company
- Del Cabo/Jacobs Farms
- Wada Farms
- Driscoll’s Strawberries
- Stonyfield Farms
- Del Monte Foods
- Western Growers Association
- California Sustainable Winegrowing Alliance
- National Potato Council
The draft metrics will be tested in the following cropping systems: Processing tomatoes, Potatoes, Strawberries/berries, Winegrapes, Leafy greens, other fresh market veg and processing vegetables, Citrus, Apples, Pears, Herbs and Stone Fruit.
Other possible cropping systems for piloting under discussion with interested growers and companies include: Fresh market tomatoes, Avocados, Almonds, Hazelnuts, Mushrooms and Nursery Trees.
Anticipated Outcomes of the Pilots:
The most important outcome we anticipate from the pilot process will be learning. At the end of this year, we should expect to have greater knowledge about what works and what doesn’t. We hope to gain insight and experience that will provide us with some clarity, and very likely, a new set of questions as well, as we work together to build a set of metrics that will give value to each partner in the supply chain.
A second outcome for the pilot participants themselves will be the ability to judge whether or not the SISC metrics support their own internal sustainability strategies. With a year of on the ground experience, participants will have gained information from which to judge if SISC metrics will provide them with useful tools to assess their own sustainability performance and improve efficiencies.
A third outcome we expect will be increased participation and attention to the SISC metrics development as we move beyond abstract concepts to concrete learning from the experience of putting the metrics to the test.
More detailed information was distributed in a memorandum to the Metric Review Committee on March 30, 2010 and is available for download.