Forecasting the future

18 months ago, AUSD and the Board of Education received a demographic study for the district from Lapkoff and Gobalet. This study came under a lot of criticism during the Edison enrollment/lottery discussion and I thought it would be interesting to look at how it has faired two years out.

A quick sample of comments about the report from Mike McMahon’s fabulous website:

“I should also point out that I’m fairly convinced that some of your demographic data are wrong.”

And

“The study from Lapkoff and Gobalet Demographic Research of March 12, 2007 is a start; however, it’s not nearly comprehensive enough because a survey was not taken of local residents to determine the actual number of children who will be entering into the system over the next five years.”

The argument at the time (and it is one that is starting to rise again) is that the district was relying on bad data for it’s predictions of future enrollments and that the district should literally go door-to-door and talk to people about the possibility that they will be attending Edison at some future date.

Much to their credit, concerned Edison parents set out to do just that and released a study with the data that they collected. Their conclusion:

While the ESNN survey covered approximately 33% of the residences within the Edison School Area, the sample data does not represent a statistically random selection across all residences, and therefore cannot be reasonably used to project numbers for the entire district. But the ESNN survey results do show that the 2007 Lapkoff and Gobalet Demographic Report projections for Kindergarten enrollment in the East End are underestimating demand for Edison School. That report used a traditional Kindergarten to Birth rate (K/B) assumption to predict K enrollment, rather than looking at an adjusted “student yield” that would reflect the higher numbers of families with young children that have moved into the area.

The district’s study looked at the area “East of Park” and did not break their forecasts down by school. Their prediction was 152 kindergarten students in 2009. The actual number returning packets? 167. This year, Edison enrollment is 47% of the East of Park enrollment. The district’s forecasts would lead them to believe that enrollment in Edison kindergarten would be around 71 kids. As mentioned in yesterday’s post, it’s looking like the number of kindergarten students that will show up at Edison next fall will be between 68 and 72. The district’s forecast seems to have hit the nail squarely.

Looking at the forecast based on the ESNN study, the study interviewed 1/3 of the households in the Edison district and never claimed to be statistically significant, the study found 44 children headed for Edison this Fall. The report did not use this number to extrapolate a forecasted number (like 132 kids in fall ’09, which a direct translation would show). However it also assumed that the number in the report was meaningful enough as to call the data in the district’s own study as seen in their recommendation to the district:

We also recommend that the District and Board of Education reach out to all parents of young children in the Edison Area, and those of the other AUSD elementary schools, to obtain a more accurate picture of the numbers of expected Kindergarten enrollees in upcoming years.

I bring this up not to say anything about the ESNN study, or the people involved in it, but I also think it’s instructive to go back and look at the studies we use in the public sphere and see how they perform. The city has produced a number of traffic studies of late, and many of them can be verified with post-project results. The Alameda Theater is one of them, and it turns out that if anything, the traffic study overstated slightly the traffic impact of the theater/garage project.

It’s good to know that many (probably not all) of these studies from the past few years appear to be holding up, it can give us confidence in the work that is provided us in future discussions. I’m not big on quoting Ronald Reagan, but “trust, but verify” comes to mind. The point isn’t “don’t question the city’s studies” but perhaps “don’t assume that studies and forecasts are ridiculous and inaccurate.”