(May 8, 2020) – With the COVID-19 pandemic dominating the news, Americans have likely heard about scientific modeling and trend projections more in the past couple months than any other time in their lives. One thing is clear: Designing models to predict the future is a tough business, especially when facing multiple variables.
In the almond business, the difficult task of predicting the future has long been the responsibility of the United States Department of Agriculture’s National Agricultural Statistics Service (USDA-NASS). Since 1997, the USDA-NASS Pacific Regional Office has conducted annual acreage surveys of California almonds, which serve as the starting point for predicting each year’s crop production in the form of the Subjective Forecast and Objective Report. For more than two decades, the industry has relied on these reports for California almond production data.
“USDA-NASS wants to get the numbers right and there is a natural variation in their estimates,” said Mel Machado, director of Member Relations at Blue Diamond Growers. “They are trying to apply a very precise science – statistics – to a very imprecise medium – plant growth.”
Recent forecasting estimates missed the mark
It’s no secret that the 2018 and 2019 USDA-NASS Objective Reports missed actual crop yields by uncharacteristically large amounts compared to previous USDA-NASS projections. The 2019 Objective Report predicted a 2.2 billion-pound crop – 12.3% lower than the more than 2.5 billion pounds harvested. In 2018, the Objective Report overestimated actual production by 7.46%.
“We want to avoid these kinds of wide discrepancies,” said Tim Birmingham, director of Quality Assurance and Industry Services at the Almond Board of California (ABC). “Still, the Objective Report has only been outside its confidence interval four times out of the past 13 years. It’s also important that people understand the Objective Report is a range, representing a potential high yield and potential low yield.”
Birmingham said the Objective Report is expected to capture the true size of California’s almond crop in a given range – or band – with 80% confidence. For example, the 2019 Objective Report of 2.2 billion pounds meant there was an 80% chance the crop would be between 2.05 and 2.35 billion pounds.
Still, industry leaders sought to make improvements in the forecasting process.
“After the miss two years ago and then again last year, we knew something had to change,” said Brian Ezell, vice president of the Almond Division at Wonderful Pistachios & Almonds. “Just a 100-million-pound change from the Objective Report will move the market. We seek the best information so that everyone is on the same playing field and so that there is less price volatility. With volatility, you get loss of demand.”
Industry-USDA task force convenes to identify solutions
In the fall of 2019, ABC convened an Almond Forecasting Task Force made up of industry members including Machado, Ezell, USDA-NASS officials and ABC staff to review almond forecasting procedures and identify potential changes to sampling methodologies to improve future crop estimates.
“This task force was in response to what happened last year, where the Objective Report number was quite a bit different than the Subjective Forecast number, with the final crop ending up closer to the Subjective Forecast,” said Birmingham, who participated in the task force on behalf of ABC. “To solve this, we thought we had to ramp up acreage surveys to be more commensurate with the size of the industry and that we needed to be able to capture new almond varieties entering production. Everything we’ve done on this task force has been focused on those areas.”
The foundation for each year’s Objective Report is the Almond Acreage Report, which consists of estimated total almond acreage, as well as bearing and non-bearing acreage. The report also includes detailed data on almond variety, year planted and county location, gathered through a voluntary survey mailed to almond growers each year.
“The acreage survey mailed to growers is so important to capture the size of the industry and varieties planted,” Birmingham said. “It’s important for growers to complete the survey accurately and in a timely fashion. The more accurate we are in the Acreage Report, the more accurate the following reports will be, such as the Objective Report.”
In the latest California Almond Acreage Report published by the California Department of Food and Agriculture, USDA-NASS acknowledges two variables that make predicting almond acreage less than perfect:
- “A voluntary survey of nearly 6,000 almond growers is unlikely to attain 100 percent completeness,” and
- “It is difficult for USDA-NASS to detect growers that are planting almonds for the first time.”
“Number one, they have to find you, and number two, you have to respond,” Machado said. “If growers don’t respond to the acreage report survey, USDA-NASS ends up creating a forecast with one eye closed.”
Ezell echoes the point: “The details in the report don’t match the actual acreage in a given year. As the acreage grows, the problem only gets worse. You can’t start with bad data to build a forecast.”
Recommendations aim to improve acreage, varietal information
To help USDA-NASS identify new growers and acreage, the task force recommended leveraging almond acreage information from Land IQ, a Sacramento-based agricultural and environmental scientific research and consulting firm, which developed and maintains a comprehensive, living map of California almonds. ABC first commissioned Land IQ to build its acreage map in 2018.
“Land IQ uses satellite imagery to map almond acreage across the state, so they see new acreage going in before that information likely gets to USDA-NASS,” Birmingham said. “We want to share that information with USDA-NASS so they can quickly identify new parcels of acreage to include in their annual survey and better determine which varieties are going in.”
As such, this year Land IQ acreage estimate data and information was shared with USDA-NASS as a source to fine-tune the official California Almond Acreage Report, and it will be used to help refine other forecasts, as well.
The next piece of the puzzle for the task force was to make recommendations to USDA-NASS for improving the acreage report by varietal, a key component in forecasting crop production yields.
“Every variety has a different size and sets a different amount of nuts in a normal year. If too much weight is placed on one variety versus another, the forecasting estimate is going to be off,” said Ezell.
The task force recommended that incoming production receipts be used in an effort to help USDA-NASS better account for changes in acreage varietals and quickly identify varietal gaps in its orchard sampling scheme. In addition, data shared by Land IQ will help USDA-NASS more quickly identify new almond acres.
Orchard sampling changes seek to improve accuracy of Objective Report
Also released by USDA-NASS, the main Objective Report number – how many pounds of almonds the industry is estimated to produce in the upcoming crop year – is based on physical in-field sampling of randomly selected almond orchards from across the state based on age, variety and location. With the changes recommended by the task force to better reflect the actual make-up of almond acreage, future samplings should also more closely reflect the entirety of the California almond industry.
“The task force also wanted to increase the number of orchard samples used in the Objective Report to keep up with the growth of the industry,” Birmingham said. “Anywhere from 700-850 orchards are sampled every year and that number hasn’t increased over the years. The task force recommended increasing the sample size to 1,000 orchards.”
In addition to sampling more orchards, the task force also recommended doubling the count units per tree sampled. The current method for determining tree set includes counting nuts along a random limb within a randomly selected tree. A sample of the counted nuts are then measured for weight, width, thickness and length. Starting in 2020 and into the future, two limbs will be counted per tree.
“There will be a lot more data gathered to make a better estimate of the crop,” said Birmingham, who noted that USDA-NASS has accepted all the recommended changes proposed by ABC’s task force.
With a recognition of the difficulty of crop forecasting, hopes remain high that future Objective Reports will be closer to actual crop yields.
“I’ve done nut counts and I know how difficult it can be,” Machado said. “I look at the average of how close USDA-NASS has been over the years and they’ve done a good job. Folks think you can predict the crop through an algorithm, but you simply cannot do it from a keyboard.”
“The proof will be in the pudding when we see a report later this year and the confidence will go up after several years of successful projections,” Ezell said. “It’s not the fault of USDA-NASS. They just don’t have the information. We hope that the work done by the task force will help them improve.”
