2002 Predictions -- Keeping Score 
By Tom Tippett
November 25, 2002
In the spring of 1998, we released our first annual Projection Disk, enabling you to play the coming season using Diamond Mind Baseball and over 1500 established big leaguers and top minor-league prospects.
Players on our Projection Disks are rated to perform in accordance with stats that we create using our projection system. That system produces expected statistics for all batters and pitchers based on a blend of major-league and minor-league stats from the past three years, adjusted for factors such as the level of competition (majors, AAA, AA), ballpark effects (including minor-league parks), league rules (DH vs non-DH), and age (young players are projected to improve while older players are projected to fade a little each year).
After the projected stats have been computed and the player ratings assigned, we set up a manager profile for each team with the starting rotation, bullpen assignments, lineups versus left- and right-handed pitchers, and roles for bench players (such as platoons, spot starters, and defensive replacements) based on best assessment of how the players will be used in the coming season.
We then simulate the season many times, average the results to come up with our projected final standings for the season, and write an article describing the results and commenting on the outlook for every team.
That first year, we were curious to see how our projected standings would stack up against the pre-season predictions in leading magazines such as Sports Illustrated, The Sporting News, Street and Smith, and USA Today Baseball Weekly.
Assigning scores
To do those rankings, we needed a way to assign an accuracy score to each prediction. So we turned to our friend Pete Palmer, the co-author of Total Baseball and The Hidden Game of Baseball. Pete has been projecting team standings for more than 25 years, and he routinely collects predictions and ranks them at the end of the year.
Pete's rankings are based on a simple scoring system -- subtract each team's actual placement from their projected placement, square this difference, and add them up for all the teams. For example, if you predict a team will finish fourth and they finish second, that's a difference of two places. Square the result, and you get four points. Do this for every team and you get a total score. The lower the score, the more accurate your predictions.
We don't try to break ties and name one team as having finished ahead of the other. If, for example, two teams tie for first, we say that each team finished in 1.5th place for the purposes of figuring out how many places a prediction was off. If a team was projected to finish third and they tied for first instead, that's a difference of 1.5 places. The square of 1.5 is 2.25, so that would be the point total for this team. That's why you'll see some fractional scores in the tables below.
Keeping things in perspective
That first year, we created a little database with our projected standings and those of fourteen national publications, and we were pleased to see that we ended the year with the best accuracy score among those fifteen forecasts as measured by Pete's formula. When we wrote up the results and posted them to our web site, however, we were very careful not to make any grand claims, saying:
"I'm not sure what to make of all this. It's just one year, and it's entirely possible that we were just lucky. Time will tell whether our approach to projecting seasons is consistently better than average. But it sure is fun to make those predictions and then take a look at them later, so we'll keep doing it."
Over time, we expanded our database to include the predictions of prominent baseball writers from major newspapers and those of several ESPN.com staffers. In the sections below, we'll show you how these prognosticators ranked in 2002 and over a period of years, with the period varying in length depending on when we added that forecaster to our database. We don't make any claims of completeness here -- there are lots of other predictions that are not in our database -- but I think you'll find that our sample is an interesting one.
For several reasons, I want to emphasize that it's important that nobody take these rankings too seriously.
First, this isn't the only scoring system one could use to rank these projections, of course. The rankings at ESPN.com use the same approach but don't square the differences. A fellow named Gerry Hamilton runs a predictions contest every year (see http://www.tidepool.com/~ggh1/index.html) and assigns a score based on how many games each team finished out of their predicted place in the standings.
Second, it's not entirely fair to put all of these predictions into the same group. Because of publishing deadlines, the predictions in the spring baseball magazines are made long before spring training started, others (including ours) are usually prepared in early-to-mid March, while some are published just before opening day. Obviously, the later you do them, the more information you have on player movement and injuries.
Third, many newspaper editors ask staff writers to make predictions so their readers have something to chew on for a couple of days. Some writers hate doing them but comply because their editors insist. Others may make off-the-wall picks just for grins. We don't have a reliable way to decide which are serious, so we include them all. But we do ask you to remember that some of these predictions may have been made in jest.
Finally, our projections are based on the average of many simulated seasons. That means that the normal ups and downs of a single season are smoothed out. For example, in 2002, we had the A's winning the AL West with 96 wins, but their win total in any one simulated season could have been much higher or lower, and they didn't always finish first. In two of the fifty seasons that we ran last March, Anaheim won that division even though the overall averages put them in last place.
Rankings for 2002
It's interesting to see how everyone did this year, of course, and those rankings are shown in the first table below.
More interesting, at least to my mind, is the process of looking back at those predictions and identifying the patterns that emerge. Which teams were consistently under- or over-estimated? Which divisions contained the biggest surprises? Did anyone predict that certain teams would have a sudden change of fortune? So we'll follow the table with a brief analysis of the six division races.
Forecaster Score Tony DeMarco, MSNBC.com 34 Athlon 38 Bob Hohler, Boston Globe 38 Peter Gammons, ESPN 38 Diamond Mind simulations 40 Baseball Weekly 42 Brandon Funston, ESPN.com 42 Eric Karabell, ESPN.com 42 Jayson Stark, ESPN 42 Lindy's 42 Danny Sheridan, USA Today 44 Los Angeles Times 44 Zack Scott, Diamond Mind 44 Dallas Morning News 46 Las Vegas over-under line 46 2001 final standings 48 Andy Latack, ESPN.com 48 Baseball America 48 Chicago Tribune 48 David Schoenfield, ESPN.com 48 Rob Neyer, ESPN.com 48 Sean McAdam, ESPN.com 48 Sports Illustrated 48 Tim Kurkjian, ESPN.com 48 Pete Palmer 50 San Francisco Chronicle 50 Alan Schwarz, ESPN.com 54 The Sporting News (spring magazine) 54 Gordon Edes, Boston Globe 54 Matt Szefc, ESPN.com 54 Phil Rogers, ESPN.com 56 Spring Training Yearbook 56 Bob Ryan, Boston Globe 58 USA Today 58 Jim Caple, ESPN.com 60 Steve Mann 60 Bob Klapisch, ESPN.com 62 Joe Sheehan, Baseball Prospectus 62 David Lipman, ESPN.com 66 Dan Shaughnessy, Boston Globe 70 Street & Smith 70 John Sickels, ESPN.com 74 Michael Holley, Boston Globe 76 Rany Jazayerli, Baseball Prospectus 78 Spring training results 86
The "Diamond Mind simulations" entry is the one representing the average result of simulating the season 50 times. These simulations were done about three weeks before the season started. Two weeks later, Zack Scott of Diamond Mind made his own predictions. These were based largely on the simulation results but also took his own hunches into account.
There are three entries in this list that don't represent the views of a writer or a publication. If you predicted that the 2002 standings would be the same as in 2001, your score would have been 48. If you put together a set of standings based on the Las Vegas over-under line, you'd have scored 46. And if you predicted that the regular season standings would match the 2002 spring training standings, your score would have been 86. In other words, for the second year in a row, the spring training results were almost useless as a predictor of the real season.
Reviewing the divisions
Much more interesting than the overall scores, in my opinion, are the details. Leaving out the entries that don't represent writers or publications, here are some observations about how the others saw things last spring:
AL East. The teams have finished in the same order five years in a row. Michael Holley of the Boston Globe was the only one to pick Boston ahead of New York. Three people from ESPN.com (David Lipman, Eric Karabell, and John Sickels) put Toronto in second. Holley and Dan Shaughnessy of the Globe were alone in having Baltimore third, a pick that looked mighty good before the Orioles collapsed in the last six weeks. Everybody else had Baltimore and Tampa Bay finishing fourth and fifth, with the O's getting the nod for fourth a little more often than not.
AL Central. The White Sox were the consensus pick as division winner, but not by a lot. Twenty-four predictions put the pale hose in first, fourteen correctly picked the Twins, and four thought Cleveland would hold on for one more year. Two intrepid souls (Jim Caple and Steve Mann) were brave enough to pick the Tigers for third, but everybody else had KC and Detroit bringing up the rear, with Detroit picked fourth about twice as often as KC. Three forecasters (Bob Hohler, John Sickels, and The Dallas Morning News) got the division right from top to bottom.
AL West. It won't come as a shock when I say that most people gave the division to Seattle after the Mariners' record-setting 2001 season. But it was far from unanimous, and about a quarter of them gave the division to Oakland, while Michael Holley picked the Rangers to finish first. Only three (The Sporting News, Gordon Edes, and Jayson Stark) picked Anaheim as high as second, and all three of them had Seattle first and Oakland third. Twenty-seven predictions put Anaheim in the basement. Needless to say, nobody was exactly right on this division.
NL East. This division was the undoing of many a predictor this year. Eleven predictors thought the Mets would end Atlanta's run at the top. Twenty-two more put them second, five had them third, two (Diamond Mind and Steve Mann) ranked them fourth, and one (Joe Sheehan) had them in the basement. By picking Florida first, Steve Mann was the only one to have a team other than Atlanta or New York atop the division. Montreal was picked last by everyone except Joe Sheehan and Pete Palmer, and neither of them had the Expos higher than fourth. Another division that nobody got right.
NL Central. For those who finished near the bottom of the rankings, if the Mets didn't get them, the Cubs did. All but three submissions had either St. Louis (27 times) or Houston (12 times) on top, but three (Dan Shaughnessy, Bob Ryan, and John Sickels) picked the Cubs. Ten others put Chicago second, with only two (Tony DeMarco and David Schoenfield) putting them as low as fourth. It seems surprising now, but nine predictions saw Milwaukee matching its fourth-place finish from a year. Most, however, had Cincinnati rebounding to take that fourth spot. Pittsburgh was picked last on almost every list, but a few had them fifth, and the LA Times were the only ones to correctly pick them fourth. Cincinnati was picked last on the only four submissions that didn't have Pittsburgh or Milwaukee in that slot. Nobody picked this division correctly from top to bottom.
By the way, when we ran our first batch of simulations back in March, Cincinnati finished one game ahead of Chicago. We had a few more minor adjustments to make, and while we were working on those adjustments, I was very worried that we'd end up picking the Cubs fourth and then spend six months watching them win the division. Now we can see that those early results would have made us look very smart. At the time, however, I was very relieved to see the Cubs edge Cincinnati by one game when we ran our fifty seasons for real.
NL West. For the second year in a row, we were among the most optimistic about the Rockies chances. Seven predictions put the Rockies in third and everyone else had them lower than that. And Diamond Mind? In our simulations, they averaged 85 wins and edged Arizona by one game for second place. We were among the 25 forecasters who picked the Giants to win it, while only 15 saw the defending world champion Diamondbacks repeating atop the division. Two (Joe Sheehan and Rany Jazayerli) picked the Padres. Most predictions had the Dodgers finishing either third or fourth, but there were exceptions, as they were listed second three times (Bob Ryan, Sean McAdam, Phil Rogers) and in the basement twice (Rob Neyer and Rany Jazayerli). This division was picked correctly from top to bottom six times, by Tony DeMarco, Street and Smith, Athlon, Lindy's, Eric Karabell, and Steve Mann.
So, we had one division that was dead simple (24 predictions nailed the AL East), three that nobody got right, one with three correct predictions, and one with five. Based on the past five years, that seems about par for the course. The baseball world looked a bit different back in March, and a lot can change in six months.
Five-year rankings
Looking back over the past five years, here are the rankings for those who were included in our sample every year. Disappearing are Baseball Digest and Mazeroski magazines, neither of which were published in 2002. In what is most likely just a coincidence, those two were at the bottom of last year's four-year rankings:
Forecaster 2002 2001 2000 1999 1998 Total Diamond Mind simulations 40.0 54.5 68.0 42.0 44.5 249.0 Steve Mann 60.0 38.5 58.0 54.0 44.0 254.5 Sports Illustrated 48.0 56.5 40.0 56.0 54.0 254.5 Baseball Weekly 42.0 46.5 58.0 51.5 60.0 258.0 Las Vegas over-under line 46.0 65.5 51.5 48.0 52.0 263.0 Pete Palmer 50.0 70.5 54.0 40.0 58.0 272.5 Sporting News 54.0 52.5 38.0 78.0 54.0 276.5 Athlon 38.0 67.5 42.0 72.0 72.0 291.5 Street & Smith 70.0 68.5 58.0 68.0 64.0 328.5 Previous season standings 48.0 64.5 56.0 70.0 100.0 338.5
Four-year rankings
In 1999, we added some writers from the Boston Globe and ESPN.com, so the four-year totals include a few more names than did the previous table:
Forecaster 2002 2001 2000 1999 Total Gordon Edes, Boston Globe 54.0 56.5 26.0 28.0 164.5 Baseball Weekly 42.0 46.5 58.0 51.5 198.0 David Schoenfield, ESPN.com 48.0 56.5 56.0 40.0 200.5 Sports Illustrated 48.0 56.5 40.0 56.0 200.5 Diamond Mind simulations 40.0 54.5 68.0 42.0 204.5 Rob Neyer, ESPN.com 48.0 66.5 48.0 44.0 206.5 Peter Gammons, ESPN.com 38.0 56.5 48.0 66.0 208.5 Steve Mann 60.0 38.5 58.0 54.0 210.5 Las Vegas over-under line 46.0 65.5 51.5 48.0 211.0 Pete Palmer 50.0 70.5 54.0 40.0 214.5 Rany Jazayerli, BP 78.0 62.5 46.0 30.0 216.5 Athlon 38.0 67.5 42.0 72.0 219.5 Sporting News 54.0 52.5 38.0 78.0 222.5 Baseball America 48.0 54.5 54.0 70.0 226.5 Dan Shaughnessy, Globe 70.0 44.5 54.0 58.0 226.5 Previous year standings 48.0 64.5 56.0 70.0 238.5 Bob Ryan, Boston Globe 58.0 84.5 58.0 40.0 240.5 John Sickels, ESPN.com 74.0 68.5 58.0 58.0 258.5 Bob Klapisch, ESPN.com 62.0 57.5 78.0 62.0 259.5 Street & Smith 70.0 68.5 58.0 68.0 264.5
Three-year rankings
The Diamond Mind simulations missed the mark by quite a bit in 2000, so they rank lower here than in any of the other tables. We added a new concept to our projection system that year, but we were very unhappy with the results, and we took that out of the model before doing this again in 2001. The results have been much better since.
Forecaster 2002 2001 2000 Total Sean McAdam, ESPN.com 48.0 32.5 38.0 118.5 Gordon Edes, Boston Globe 54.0 56.5 26.0 136.5 Peter Gammons, ESPN.com 38.0 56.5 48.0 142.5 Sporting News 54.0 52.5 38.0 144.5 Sports Illustrated 48.0 56.5 40.0 144.5 Baseball Weekly 42.0 46.5 58.0 146.5 Athlon 38.0 67.5 42.0 147.5 Baseball America 48.0 54.5 54.0 156.5 Steve Mann 60.0 38.5 58.0 156.5 David Schoenfield, ESPN.com 48.0 56.5 56.0 160.5 Diamond Mind simulations 40.0 54.5 68.0 162.5 Rob Neyer, ESPN.com 48.0 66.5 48.0 162.5 Las Vegas over-under line 46.0 65.5 51.5 163.0 Dan Shaughnessy, Globe 70.0 44.5 54.0 168.5 Previous year standings 48.0 64.5 56.0 168.5 Pete Palmer 50.0 70.5 54.0 174.5 Phil Rogers, ESPN.com 56.0 62.5 56.0 174.5 Matt Szefc, ESPN.com 54.0 56.5 68.0 178.5 Rany Jazayerli, BP 78.0 62.5 46.0 186.5 Street & Smith 70.0 68.5 58.0 196.5 Bob Klapisch, ESPN.com 62.0 57.5 78.0 197.5 John Sickels, ESPN.com 74.0 68.5 58.0 200.0 Bob Ryan, Boston Globe 58.0 84.5 58.0 200.5
Two-year rankings
Finally, here's how things have looked in 2001-2002. As you can see, Lindy's has had two very good years in a row and have to be regarded as the top dog for the time being. Sean McAdam slipped a bit in 2002 after two extremely good years, but is still right at the top. This doesn't surprise me a bit; I have the good fortune of hearing Sean regularly on Boston's main sports radio station, and he really knows his stuff.
Forecaster 2002 2001 Total Lindy's 42.0 36.5 78.5 Sean McAdam, ESPN.com 48.0 32.5 80.5 SF Chronicle 50.0 36.5 86.5 Baseball Weekly 42.0 46.5 88.5 Jayson Stark, ESPN.com 42.0 46.5 88.5 Diamond Mind simulations 40.0 54.5 94.5 Peter Gammons, ESPN.com 38.0 56.5 94.5 Steve Mann 60.0 38.5 98.5 Tony DeMarco, MSNBC.com 34.0 67.5 101.5 Baseball America 48.0 54.5 102.5 Zack Scott, Diamond Mind 44.0 58.5 102.5 David Schoenfield, ESPN.com 48.0 56.5 104.5 Sports Illustrated 48.0 56.5 104.5 Athlon 38.0 67.5 105.5 Sporting News 54.0 52.5 106.5 Chicago Tribune 48.0 62.5 110.5 Gordon Edes, Boston Globe 54.0 56.5 110.5 Matt Szefc, ESPN.com 54.0 56.5 110.5 Las Vegas over-under line 46.0 65.5 111.5 Previous year standings 48.0 64.5 112.5 Rob Neyer, ESPN.com 48.0 66.5 114.5 Dan Shaughnessy, Globe 70.0 44.5 114.5 Los Angeles Times 44.0 73.5 117.5 Phil Rogers, ESPN.com 56.0 62.5 118.5 Bob Klapisch, ESPN.com 62.0 57.5 119.5 Pete Palmer 50.0 70.5 120.5 Alan Scwarz, ESPN.com 54.0 70.5 124.5 David Lipman, ESPN.com 66.0 64.5 130.5 Street & Smith 70.0 68.5 138.5 Rany Jazayerli, BP 78.0 62.5 140.5 John Sickels, ESPN.com 74.0 68.5 142.5 Bob Ryan, Boston Globe 58.0 84.5 142.5
Conclusions
Except for the 2000 season, our approach to developing projections seems to be providing good results.
If there's any one thing that stands out, it's the system's ability to identify over-rated teams. In 2002, for example, our simulations indicated that (a) the Mets would have real trouble scoring runs even with the addition of guys like Mo Vaughn and Roberto Alomar, (b) the Seattle offense would come back to earth after a terrific 2001 season, and (c) the Cubs were likely to be battling the Reds for third and fourth place, not challenging for the division title.
On the other hand, we didn't anticipate the sudden emergence of some of the game's best bullpens. (I'm not sure anyone else saw this coming, either.) In our simulations, the relievers on the Angels, Twins, and Braves were nowhere near as good as they were in the real 2002 season.
And if there's a theme from the past five years, it's that our simulations occasionally project a team with great hitting and barely acceptable pitching to do very well, and it seems as if their real-life counterparts often blow a bunch of leads early in the year and then fall apart. The Rangers were this year's example, though a very tough division and key injuries on both sides of the ball were a major factor as well.
I wish we were better at projecting which young teams will continue to get better, as the Twins did this year, and which will backslide unexpectedly. Before the season, I wouldn't have been surprised to see the Marlins outperform the Twins, but it didn't work out that way.
Still, it's a lot of fun and a highly educational process for us. When we run the simulations in the spring, we always end up learning something. We always end up being surprised by some of the results. And we always end up with a bunch of things to watch for as the real season unfolds. So we'll take another run at it next spring and we'll report back after the season.
