2003 Predictions -- Keeping Score 
By Zack Scott and Tom Tippett
January 31, 2004
In a few weeks, when we release our annual Projection Disk, we'll give our customers a chance to get a head start on the baseball season.
With projected statistics and ratings for over 1600 established big leaguers and top minor-league prospects, plus league schedules, park factors, team rosters, projected pitching rotations, bullpen assignments, lineups and depth charts, the Projection Disk includes everything you need to play out the coming season with Diamond Mind Baseball.
Our projection system uses a blend of major-league and minor-league stats from the past three years, adjusted for factors such as the level of competition, ballpark effects (including minor-league parks), league rules (DH vs non-DH), and age.
We've been creating Projection Disks since 1998. That's also the year we began running preseason simulations and presenting our projected standings for the coming season along with comments on the outlook for all 30 teams.
After the real season and postseason have been played out, we compare our results to previous years and to the predictions made by other leading baseball experts and publications.
Comparing predictions
In addition to projecting the order of finish, our simulations provide us with projected win-loss records, projected runs for and against, and the probability that each team will make the postseason by winning its division or grabbing the wild card.
For our own projections, we like to see how the current season stacked up against our previous efforts. One way to do that is to compare projected wins to actual wins. In 2003, the average difference between projected and actual wins was the lowest, by a good margin, in the six years we've been doing this.
We'd love to be able to evaluate other predictions that way, but most of them don't include projected win-loss records. Instead, they give the projected order of finish without indicating which races are expected to be hotly contested and which will be runaways.
So we do our best to assign a meaningful score to each prediction based solely on order of finish within each division. Our friend Pete Palmer, co-author of Total Baseball and The Hidden Game of Baseball, has been projecting team standings for more than 30 years, and he routinely collects predictions and ranks them at the end of the year.
Pete's rankings are based on a simple scoring system -- subtract each team's actual placement from their projected placement, square this difference, and add them up for all the teams. For example, if you predict a team will finish fourth and they finish second, that's a difference of two places. Square the result, and you get four points. Do this for every team and you get a total score. The lower the score, the more accurate your predictions.
By the way, we don't try to break ties. If, for example, two teams tie for first, we say that each team finished in 1.5th place for the purposes of figuring out how many places a prediction was off. Suppose a team was projected to finish third and they tied for first instead. That's a difference of 1.5 places. The square of 1.5 is 2.25, so that would be the point total for this team. That's why you'll see some fractional scores in the tables below.
Keeping things in perspective
That first year, we created a little database with our projected standings and those of fourteen national publications, and we were pleased to see that we ended the year with the best accuracy score among those fifteen forecasts. When we wrote up the results and posted them to our web site, however, we were very careful not to make any grand claims, saying:
"I'm not sure what to make of all this. It's just one year, and it's entirely possible that we were just lucky. Time will tell whether our approach to projecting seasons is consistently better than average."
Over time, we expanded our database to include the predictions of prominent baseball writers from major newspapers and other publications. Unfortunately, ESPN.com did not publish complete predictions (only division winners) by their staffers in 2003. This is the main reason why our pool of predictors is much smaller this year than it was for the 2002 season.
In the sections below, we'll show you how these prognosticators ranked in 2003 and over a period of years, with the period varying in length depending on when we added that person or publication to our database. We don't make any claims of completeness here -- there are lots of other predictions that are not in our database -- but we think you'll find that our sample is an interesting one.
For several reasons, we want to emphasize that it's important that nobody take these rankings too seriously.
First, this isn't the only scoring system one could use to rank these projections, of course. A fellow named Gerry Hamilton runs a predictions contest every year (see http://www.tidepool.com/~ggh1/index.html) and assigns a score based on how many games each team finished out of their predicted place in the standings. (By the way, we came 4th out of 160 predictions in their 2003 contest, and we're pleased to say that the winner, Atticus Ryan, is a Diamond Mind customer.)
Second, it's not entirely fair to put all of these predictions into the same group. Because of publishing deadlines, the predictions in the spring baseball magazines are made long before spring training started, others (including ours) are usually prepared in early-to-mid March, while some are published just before opening day. Obviously, the later you do them, the more information you have on player movement and injuries.
Third, many newspaper editors ask staff writers to make predictions so their readers have something to chew on for a couple of days. Some writers hate doing them but comply because their editors insist. Others may make off-the-wall picks just for grins. We don't have a reliable way to decide which are serious, so we include them all. But we do ask you to remember that some of these predictions may have been made in jest.
Finally, our projections are based on the average of many simulated seasons. That means that the normal ups and downs of a single season are smoothed out. For example, in 2003, we had St. Louis winning the NL Central with 89 wins, but their win total in any one simulated season could have been much higher or lower, and they didn't always finish first. In seven of the fifty seasons that we ran last March, Chicago won that division even though the overall averages put them in third place.
Rankings for 2003
It's interesting to see how everyone did this year:
Forecaster Score Los Angeles Times 18 Diamond Mind simulations 28 Baseball America 28 Baseball Digest 28 Las Vegas over-under line 30 Sports Illustrated 30 Bob Hohler, Boston Globe 32 Danny Sheridan, USA Today 32 Gordon Edes, Boston Globe 32 San Francisco Chronicle 32 USA Today 32 Jackie MacMullan, Boston Globe 34 Tony DeMarco, MSNBC.com 34 Athlon 36 ESPN the magazine 36 Street & Smith 36 Sports Weekly 38 Bob Ryan, Boston Globe 40 Lindy's 40 MLB Yearbook 40 2002 final standings 42 Mazeroski 44 The Sporting News (spring magazine) 44 Spring Training Yearbook 48 Steve Mann 48 Dan Shaughnessy, Boston Globe 56 Pete Palmer 56 Spring training results 70
The "Diamond Mind simulations" entry is the one representing the average result of simulating the season 50 times. These simulations were done about three weeks before the season started.
There are three entries in this list that don't represent the views of a writer or a publication:
- if you predicted that the 2003 standings would be the same as in 2002, your score would have been 42
- if you put together a set of standings based on the Las Vegas over-under line, you'd have scored 30
- if you predicted that the regular season standings would match the 2003 spring training standings, your score would have been 70. In other words, the spring training results were almost useless as a predictor of the real season, and that's been true for at least the past three years.
Reviewing the divisions
Much more interesting than the overall scores, in our opinion, are the details. Which teams were consistently under- or over-estimated? Which divisions contained the biggest surprises? Did anyone predict that certain teams would have a sudden change of fortune?
Leaving out the entries that don't represent writers or publications, here are some observations about how the others saw things last spring:
AL East. Six and counting. That's how many consecutive seasons have ended with the teams finishing in the same order. And the prognosticators are catching on. All but three nailed the division, making this division their most successful collective prediction. Steve Mann was the only one to pick Boston over New York. Athlon and Lindy's put Baltimore ahead of Toronto.
AL Central. Everyone correctly saw this as a two-team race between the Twins (18 of 26 first-place selections) and White Sox. Only one (MLB Yearbook) had KC in the correct slot (3rd) and five predictors had them finishing even worse than the historically-bad Tigers. Thanks to KC, MLB Yearbook was the lone forecaster to get the division right from top to bottom.
AL West. The 2002 world champion Angels did almost nothing to improve their team during the offseason, so it didn't come as a surprise that most people picked Oakland to repeat as division winner. That said, twenty-three predictors did think Anaheim was good enough to finish in the top two. The only two (Diamond Mind and Steve Mann) that correctly placed them third were also the only two to be exactly right on this division. For us, it was a very close call. Both Seattle and Anaheim averaged 91 wins in our 50 simulations, but we listed Seattle second because they edged the Angels by a fraction of a win.
NL East. Perhaps the Braves are the Yankees of the NL East, but that may be the only resemblance between the two eastern divisions. No one forecasted the NL East correctly from top to bottom. The main culprits were the world champion Marlins. Not one predictor had them finishing second and all but two had the Contraction Kids doing no better than fourth.
We were among the ten that incorrectly thought Philly would end Atlanta's reign atop the division. Fifteen gave Atlanta the division. The Boston Globe's Dan Shaughnessy killed his score by being the only one to pick the Mets as division winners. But Shaughnessy and Globe colleague Jackie MacMullan were the only two that correctly slotted Philly into third place. For the third straight season, many people overrated the Mets -- 85% of the predictions had NY finishing no worse than third and only one (Diamond Mind) had them in the basement.
NL Central. For the second straight year, the Cubs did a number on many a predictor. Jackie MacMullan was the sole forecaster to have the Cubs taking the division. Everyone else chose St. Louis (16 times) or Houston (9 times) with most (19) picking the Cubs for third. Pete Palmer's ranking plummeted because he had Chicago finishing fifth. Almost everyone pegged Cincinnati and Pittsburgh for fourth and fifth place, respectively, and only two (Baseball Digest and SF Chronicle) had them correctly reversed. Nobody got this division exactly right, but everyone had Milwaukee in last place. That was the only unanimous decision in any division.
NL West. For three years in a row, we were among the most optimistic about the Rockies' chances. Some have speculated that the high altitude of Coors Field affects a player's road performance in a way that isn't being captured in our projections. Then again, that may have nothing to do with it. Denny Neagle's injury, coupled with disappointing seasons from long-time Rockies Larry Walker and Jose Jimenez, go a long way toward explaining this year's losing record. We were one of two (Street and Smith was the other) to put the Rockies in third. Everyone else had them lower than that.
Much to our surprise, the Dodgers received as many first-place predictions as the defending NL champion Giants. We were among the seven forecasters who picked San Francisco to win it, while twelve saw Arizona taking the division. Of those who didn't pick the Diamondbacks to win it, most had them finishing second. Only three (LA Times, MLB Yearbook, and Shaughnessy) had them correctly slotted in third. LA was predicted to finish anywhere from first to fourth but most (14) had them third. This division was picked correctly from top to bottom only once, by the LA Times.
Summing up. In 2003, there was one division that was dead simple (23 of 26 predictions nailed the AL East), two that nobody got right, two with one correct prediction, and one with two. Based on the previous five years, that seems about par for the course. The baseball world looked a bit different back in March, and a lot can change in six months.
Six-year rankings
Here are the rankings for those who were included in our sample every year:
Forecaster 2003 2002 2001 2000 1999 1998 Total Diamond Mind 28.0 40.0 54.5 68.0 42.0 44.5 277.0 Sports Illustrated 30.0 48.0 56.5 40.0 56.0 54.0 284.5 Las Vegas over-under 30.0 46.0 65.5 51.5 48.0 52.0 293.0 Sports Weekly 38.0 42.0 46.5 58.0 51.5 60.0 296.0 Steve Mann 48.0 60.0 38.5 58.0 54.0 44.0 302.5 Sporting News 44.0 54.0 52.5 38.0 78.0 54.0 320.5 Athlon 36.0 38.0 67.5 42.0 72.0 72.0 327.5 Pete Palmer 56.0 50.0 70.5 54.0 40.0 58.0 328.5 Street & Smith 36.0 70.0 68.5 58.0 68.0 64.0 364.5 Previous season 42.0 48.0 64.5 56.0 70.0 100.0 380.5
Five-year rankings
In 1999, we added some writers from the Boston Globe, so the five-year totals include a few more names than did the previous table. We also added writers from ESPN.com that year, but they did not publish predictions beyond division winners in 2003, so we had to drop them from this list.
Forecaster 2003 2002 2001 2000 1999 Total Gordon Edes, Boston Globe 32.0 54.0 56.5 26.0 28.0 196.5 Sports Illustrated 30.0 48.0 56.5 40.0 56.0 230.5 Diamond Mind simulations 28.0 40.0 54.5 68.0 42.0 232.5 Baseball Weekly 38.0 42.0 46.5 58.0 51.5 236.0 Las Vegas over-under line 30.0 46.0 65.5 51.5 48.0 241.0 Baseball America 28.0 48.0 54.5 54.0 70.0 254.5 Athlon 36.0 38.0 67.5 42.0 72.0 255.5 Steve Mann 48.0 60.0 38.5 58.0 54.0 258.5 Sporting News 44.0 54.0 52.5 38.0 78.0 266.5 Pete Palmer 56.0 50.0 70.5 54.0 40.0 270.5 Bob Ryan, Boston Globe 40.0 58.0 84.5 58.0 40.0 280.5 Previous season standings 42.0 48.0 64.5 56.0 70.0 280.5 Dan Shaughnessy, Globe 56.0 70.0 44.5 54.0 58.0 282.5 Street & Smith 36.0 70.0 68.5 58.0 68.0 300.5
Four-year rankings
The Diamond Mind simulations missed the mark by quite a bit in 2000, so they rank lower here than in any of the other tables. We added a new concept to our projection system that year, but we were very unhappy with the results, and we took that out of the model before doing this again in 2001. The results have been much better since.
Forecaster 2003 2002 2001 2000 Total Gordon Edes, Boston Globe 32.0 54.0 56.5 26.0 168.5 Sports Illustrated 30.0 48.0 56.5 40.0 174.5 Athlon 36.0 38.0 67.5 42.0 183.5 Baseball America 28.0 48.0 54.5 54.0 184.5 Sports Weekly 38.0 42.0 46.5 58.0 184.5 Sporting News 44.0 54.0 52.5 38.0 188.5 Diamond Mind simulations 28.0 40.0 54.5 68.0 190.5 Las Vegas over-under line 30.0 46.0 65.5 51.5 193.0 Steve Mann 48.0 60.0 38.5 58.0 204.5 Previous season standings 42.0 48.0 64.5 56.0 210.5 Dan Shaughnessy, Globe 56.0 70.0 44.5 54.0 224.5 Pete Palmer 56.0 50.0 70.5 54.0 230.5 Street & Smith 36.0 70.0 68.5 58.0 232.5 Bob Ryan, Boston Globe 40.0 58.0 84.5 58.0 240.5
Three-year rankings
Lindy's and the San Francisco Chronicle were strong additions to the predictor pool in 2001.
Forecaster 2003 2002 2001 Total Lindy's 40.0 42.0 36.5 118.5 SF Chronicle 32.0 50.0 36.5 118.5 Diamond Mind simulations 28.0 40.0 54.5 122.5 Sports Weekly 38.0 42.0 46.5 126.5 Baseball America 28.0 48.0 54.5 130.5 Sports Illustrated 30.0 48.0 56.5 134.5 Los Angeles Times 18.0 44.0 73.5 135.5 Tony DeMarco, MSNBC.com 34.0 34.0 67.5 135.5 Athlon 36.0 38.0 67.5 141.5 Las Vegas over-under line 30.0 46.0 65.5 141.5 Gordon Edes, Boston Globe 32.0 54.0 56.5 142.5 Steve Mann 48.0 60.0 38.5 146.5 Sporting News 44.0 54.0 52.5 150.5 Previous season standings 42.0 48.0 64.5 154.5 Dan Shaughnessy, Globe 56.0 70.0 44.5 170.5 Street & Smith 36.0 70.0 68.5 174.5 Pete Palmer 56.0 50.0 70.5 176.5 Bob Ryan, Boston Globe 40.0 58.0 84.5 182.5
Two-year rankings
Finally, here's how things have looked in 2002-2003. As you can see, the LA Times has had one mediocre year followed by an excellent one. It will be interesting to see if they can remain top dog.
Forecaster 2003 2002 Total Los Angeles Times 18.0 44.0 62.0 Diamond Mind simulations 28.0 40.0 68.0 Tony DeMarco, MSNBC.com 34.0 34.0 68.0 Bob Hohler, Boston Globe 32.0 38.0 70.0 Athlon 36.0 38.0 74.0 Baseball America 28.0 48.0 76.0 Danny Sheridan, USA Today 32.0 44.0 76.0 Las Vegas over-under line 30.0 46.0 76.0 Sports Illustrated 30.0 48.0 78.0 Sports Weekly 38.0 42.0 80.0 Lindy's 40.0 42.0 82.0 SF Chronicle 32.0 50.0 82.0 Gordon Edes, Boston Globe 32.0 54.0 86.0 Previous season standings 42.0 48.0 90.0 USA Today 32.0 58.0 90.0 Bob Ryan, Boston Globe 40.0 58.0 98.0 Sporting News 44.0 54.0 98.0 Pete Palmer 56.0 50.0 106.0 Street & Smith 36.0 70.0 106.0 Steve Mann 48.0 60.0 108.0 Dan Shaughnessy, Globe 56.0 70.0 126.0
Conclusions
Except for the 2000 season, we've been pretty happy with our results.
If there's any one thing that stands out, it's the system's ability to identify over-rated teams. In 2003, for example, our simulations indicated that (a) the Mets were likely to finish at the bottom of their division again, (b) the Angels were very unlikely to repeat their 2002 success, and (c) the Dodgers wouldn't score enough runs to make a serious run at the NL West title.
On the other hand, we didn't anticipate the sudden emergence of the Marlins and Royals. In our simulations, both teams were in the 70s for wins, and while we believe that both teams were over-achievers who will be fortunate to do as well in 2004, nobody can take away what they accomplished last year.
All in all, this process -- projecting the season in March, watching the real thing for six months, and taking a look back after the season -- is highly educational for us.
So we'll be back with our projections of the 2004 team standings in late March. As always, we're sure we'll learn something. We're certain to be surprised by some of the results. And we'll undoubtedly come up with a bunch of things to watch for as the real season unfolds.
