Forecasting the future

18 months ago, AUSD and the Board of Education received a demographic study for the district from Lapkoff and Gobalet. This study came under a lot of criticism during the Edison enrollment/lottery discussion and I thought it would be interesting to look at how it has faired two years out.

A quick sample of comments about the report from Mike McMahon’s fabulous website:

“I should also point out that I’m fairly convinced that some of your demographic data are wrong.”

And

“The study from Lapkoff and Gobalet Demographic Research of March 12, 2007 is a start; however, it’s not nearly comprehensive enough because a survey was not taken of local residents to determine the actual number of children who will be entering into the system over the next five years.”

The argument at the time (and it is one that is starting to rise again) is that the district was relying on bad data for it’s predictions of future enrollments and that the district should literally go door-to-door and talk to people about the possibility that they will be attending Edison at some future date.

Much to their credit, concerned Edison parents set out to do just that and released a study with the data that they collected. Their conclusion:

While the ESNN survey covered approximately 33% of the residences within the Edison School Area, the sample data does not represent a statistically random selection across all residences, and therefore cannot be reasonably used to project numbers for the entire district. But the ESNN survey results do show that the 2007 Lapkoff and Gobalet Demographic Report projections for Kindergarten enrollment in the East End are underestimating demand for Edison School. That report used a traditional Kindergarten to Birth rate (K/B) assumption to predict K enrollment, rather than looking at an adjusted “student yield” that would reflect the higher numbers of families with young children that have moved into the area.

The district’s study looked at the area “East of Park” and did not break their forecasts down by school. Their prediction was 152 kindergarten students in 2009. The actual number returning packets? 167. This year, Edison enrollment is 47% of the East of Park enrollment. The district’s forecasts would lead them to believe that enrollment in Edison kindergarten would be around 71 kids. As mentioned in yesterday’s post, it’s looking like the number of kindergarten students that will show up at Edison next fall will be between 68 and 72. The district’s forecast seems to have hit the nail squarely.

Looking at the forecast based on the ESNN study, the study interviewed 1/3 of the households in the Edison district and never claimed to be statistically significant, the study found 44 children headed for Edison this Fall. The report did not use this number to extrapolate a forecasted number (like 132 kids in fall ’09, which a direct translation would show). However it also assumed that the number in the report was meaningful enough as to call the data in the district’s own study as seen in their recommendation to the district:

We also recommend that the District and Board of Education reach out to all parents of young children in the Edison Area, and those of the other AUSD elementary schools, to obtain a more accurate picture of the numbers of expected Kindergarten enrollees in upcoming years.

I bring this up not to say anything about the ESNN study, or the people involved in it, but I also think it’s instructive to go back and look at the studies we use in the public sphere and see how they perform. The city has produced a number of traffic studies of late, and many of them can be verified with post-project results. The Alameda Theater is one of them, and it turns out that if anything, the traffic study overstated slightly the traffic impact of the theater/garage project.

It’s good to know that many (probably not all) of these studies from the past few years appear to be holding up, it can give us confidence in the work that is provided us in future discussions. I’m not big on quoting Ronald Reagan, but “trust, but verify” comes to mind. The point isn’t “don’t question the city’s studies” but perhaps “don’t assume that studies and forecasts are ridiculous and inaccurate.”

2 comments for “Forecasting the future

  1. Andy Currid
    February 13, 2009 at 12:23 pm

    It’s definitely of value to review previously forecasted data against what actually transpires. With regard to school enrollment, I think you also have to recognize some factors that the data doesn’t capture.

    The current demand East of Park is reported as 175, which is 15% greater than L&G’s predicted enrollment of 152. As you note, the number that will eventually be recorded as enrolled for 2009/10 will be lower, for a number of reasons – one of which is that there isn’t capacity for 175 students, there’s currently only capacity for 140, or 160, if Edison follows through with an additional class.

    Between now and September, some enrollees will voluntary look for alternatives, such as another school in the district, or private school. Some will be involuntarily diverted in September.

    The historical data contained in L&G’s demographic report does not capture demand, only actual enrollment. If you look at demand, its forcecast is off by 15%. If we look at actual enrollment recorded at the end of September, L&G will be much closer than 15%.

    The ESNN survey measured demand, not enrollment. The difference between those two is very directly affected by AUSD’s enrollment policies and the actions that it takes in dealing with excess demand. If you talk to incoming K parents about which number means more to them at this point, I think they’d say it’s demand.

  2. February 13, 2009 at 2:06 pm

    I won’t disagree, but I do think that there is a difference between possible demand and actual demand. Even during years in which enrollment was not at capacity, the number of enrollment packets was higher than those who actually showed up.

    I won’t downplay the impact of “not knowing” and in fact I think that I posted about it yesterday. But it also needs to be acknowledged that a number of people apply for kindergarten at Edison as a back up plan against other private educational opportunities not working out.

    In the end, the ESNN numbers were significantly less accurate as a planning tool than the District’s own numbers. The ESNN study included 33% of the households in the Edison district but identified 56% of the actual enrollment/demand. This says to me that the households that were spoken to were those most likely to have kids, a significant methodolgical problem. (i realize it’s volunteers, etc. I’m not knocking the effort, but I think that the findings have been used by some as a way of claiming the district was being willfully ignorant of where it’s heading.)

    I was surprised at how close the L&G numbers ended up to being. I had a expected a much larger undercounting. Planning studies aren’t expected to be completely accurate.

    All that said, I think that there’s a good question to be answered about why the district appears unprepared for this result given its own data let alone the ESNN data. Were I headed to the School Board to address the issue, I’d point out that the district had their own, board approved numbers that showed this result on the way, and given the board’s direction/opinion to “find the space” if there’s an issue, the community should get a good reason why it might not happen.

Leave a Reply

Your email address will not be published. Required fields are marked *