The Futility Infielder

Relief pitchers are the most underrepresented position in the Hall of Fame. Thus far, voters have deemed only two of them, Hoyt Wilhelm and Rollie Fingers, worthy of admission to Cooperstown. This year's ballot contained three reasonable candidates -- Rich Gossage, Lee Smith and Bruce Sutter -- none of whom came close to the 75% necessary for election.

There's no shortage of reasons for why these firemen are getting closed out:

• Unlike all of the other positions, we have a very tough time measuring the current candidates against those enshrined; a class of two doesn't exactly make a strong sample size or produce de facto standards for admission. Wilhelm is widely acknowledged as the greatest reliever ever, while Fingers simply had visibility and popularity -- in the form of several successful postseason appearances, a couple of big awards, and a distinctive moustache -- on his side.

• The tools which are readily at our disposal -- wins, losses, and especially saves -- do a less than ideal job for measuring the reliever's impact. This is especially true when comparing pitchers between different eras; Jeff Reardon has over 50% more saves than Hoyt Wilhelm, but anybody who wants to argue that Reardon was as valuable as Wilhelm has an uphill battle ahead.

• The tools which do a better job at helping us measure a reliever's impact, such as Baseball Prospectus' Adjusted Runs Prevented or Tangotiger's Leverage Index, are relatively recent developments based on play-by-play or situational data and thus unavailable for the larger chunk of the game's history.

• As Mike's Baseball Rants continues to explore, the role of the reliever has been in a nearly constant state of evolution across baseball history. Several pitchers are often identified as paradigm of the "modern" reliever based upon their pattern of usage, including Wilhelm, Sutter, and Dennis Eckersley.

• Finally, most baseball fans, whether knowledgeable statheads or simply men on barstools (not that the two are exclusive, and that'll be another round for me, thanks), intuitively grasp that while relief pitching is an important part of the game, the impact of an ace reliever isn't on par with that of an ace starter or All-Star position player. Several measures of player value -- both sabermetric and economic -- bear that out.

All of this combines to make a reliever's road to the Hall of Fame an uphill one. But that doesn't eliminate the question of where our three candidates fit in with respect to the two already in the Hall, to each other, and other good-to-great relievers who are or will be eligible for the Hall in the not-too-distant future. With Eckersley up for election next year, and a generation of save-happy closers on the horizon (Smith, the career leader, has already arrived), it's worth looking at some ways to compare them.

Last year Baseball Primer's Rich Rifkin introduced a measure designed to judge relief pitchers based on a combination of innings pitched and on ERA+ (which is park-adjusted ERA relative to the league). The reasoning behind this is simple: a pitcher's job is to prevent runs; a good pitcher prevents runs at a better than league-average rate; the more innings a pitcher throws at a better-than-average rate, the more valuable he is.

Based on the widely-agreed notion that Wilhem was the best ever, Rifkin called his measure the Hoyt Scale, and created a simple formula:

(ERA+) + 4*IP/75 = uH (unadjusted Hoyts)

Rifkin then produced a Hoyt Constant such that Wilhelm winds up with exactly 100 Hoyts, and all other relievers are calculated relative to the master. By Rich's calculations, the best relievers after the ol' knuckleballer were Kent Tekulve and Rich Gossage (84.2), John Franco (82.8), Dan Quisenberry (81.7), Lee Smith (80.4), Tom Henke (80.2), Sparky Lyle (81.0), and Rollie Fingers (79.3).

This was a quick-and-dirty attempt at getting a handle on the relative values of a select group of top-notch relievers, but it contained a few flaws. First off, Rifkin's numbers for calculating the Hoyt factor were off; for whatever reason he reported Wilhelm's total number of innings in relief as 1,890 when it's in fact 1,870 (still a major league record).

That's a minor problem, easily correctible. However, a much larger problem exists. Several pitchers in the group we're examining, including Wilhelm, have significant numbers of innings pitched as starters. Using a pitcher's total ERA+ and total number of innings favors anybody who racks up the mileage a starter gets; yet we're trying to measure relievers.

While breakdowns between innings pitched and runs allowed as starter/reliever are not always available, we do have a large amount of data for the thirty-five pitchers in my study. Using Retrosheet and a few instances where ALL of a pitchers appearances in a single season were starts, I was able to completely separate the stats as starter and reliever for eleven pitchers. Another eight obliged me by never starting a single game. That's over half of the pitchers for whom I was able to use Relief IP and Relief ERA+ to recalculate their Hoyts. We'll call all of these pitchers whose stats cooperate with our mission Group A. Wilhelm himself is included in this group because if we know his Relief IP, we know his Starting IP. I made an estimate of his Relief ERA+ which I'll explain shortly.

Here are the Group A pitchers, sorted by Relief Innings Pitched (RIP). GS(d) is the number of games started for which we have data for, and RERA+ is Relief ERA+. The rest should be familiar:

                G  GS GS(d)  W   L   Sv     IP     RIP    ERA+  RERA+

H. Wilhelm    1070  52   0   143 122  227  2254.3  1870.0  146   145

K. Tekulve    1050   0   0    94  90  184  1436.3  1436.3  132   132

S. Lyle        899   0   0    99  76  238  1390.3  1390.3  127   127

L. Smith      1022   6   6    71  92  478  1289.3  1252.3  132   143

T. Burgmeier   745   3   3    79  55  102  1258.7  1248.7  119   120

J. Orosco*    1187   4   4    85  78  142  1261.3  1243.0  130   132

B. Stanley     637  85  85   115  97  132  1707.0  1159.0  118   131

J. Franco*     998   0   0    88  76  422  1150.3  1150.3  143   143

J. Reardon     880   0   0    73  77  367  1132.7  1132.7  121   121

D. Jones       846   4   4    69  79  303  1128.3  1112.3  130   130

M. Jackson*    960   7   7    60  67  142  1141.7  1108.0  127   131

G. Lavelle     745   3   3    80  77  136  1085.0  1077.7  126   128

D. Quisenberry 674   0   0    56  46  244  1043.3  1043.3  146   146

B. Sutter      661   0   0    68  71  300  1042.3  1042.3  136   136

J. Montogomery 700   1   1    46  52  304   868.7   863.7  134   136

D. Eckersley  1071 361 361   197 171  390  3285.7   807.3  116   180

T. Henke       642   0   0    41  42  311   789.7   789.7  156   156

R. Aguilera    732  89  89    86  81  318  1291.3   740.3  117   131

J. Wetteland   618  17  17    48  45  330   765.0   683.0  148   165

T. Hoffman*    632   0   0    45  44  352   701.0   701.0  146   146

Group B consists of pitchers for whom we have incomplete data on their time as starters. For the group, we have data on 55% of their total starts. For Goose Gossage, we've got 32 out of his 37, for Rollie Fingers only 8 of 37. I went ahead and removed the known starter stats from their lines, such that we've got Relief IP and Relief ERA+ which still include some starter innings (which we'll adjust for down the road). Interestingly enough, every pitcher in either Group A or Group B had a better ERA+ as a reliever than as a starter, sometimes dramatically. Here are the Group B pitchers, sorted again by RIP (keep in mind that this RIP is not a complete total):

               G  GS GS(d)   W   L   Sv     IP     RIP    ERA+  RERA+

R. Gossage  1002  37  32    124 107  310  1890.3  1659.0   126   139

R. Fingers   944  37   8    114 118  341  1701.3  1656.7   119   122

G. Garber    931   9   1     96 113  218  1510.0  1505.7   117   117

T. McGraw    824  39  15     96  92  180  1514.7  1435.3   116   121

C. Carroll   731  28   9     96  73  143  1353.3  1299.0   120   123

M. Marshall  723  24  19     97 112  188  1386.7  1285.0   118   124

J. Hiller    545  43  35     87  76  125  1242.0  1012.0   134   135

Group C pitchers are the ones for whom we have no data on separating Relief IP and Relief ERA+. Five of the eight are older pitchers, contemporaries of Wilhelm who spent most of their careers as relievers. Perranoski and McMahon combined for only three starts; we could have ignored the lack of data and thrown them into Group A, but I chose to keep them here once I decided to add Nen and Hernandez to the study. Sorted by IP:

               G   GS  GS(d)   W   L   Sv     IP    RIP   ERA+  RERA+

L. McDaniel   987  74   0    141 119  172  2139.3   n/a    109   n/a

S. Miller     704  93   0    105 103  154  1694.0   n/a    115   n/a

E. Face       848  27   0    104  95  193  1375.0   n/a    109   n/a

D. McMahon    874   2   0     90  68  153  1310.7   n/a    119   n/a

R. Perranoski 737   1   0     79  74  179  1174.7   n/a    123   n/a

R. Myers      728  12   0     44  63  347   884.7   n/a    122   n/a

R. Hernandez  696   3   0     48  51  320   775.0   n/a    143   n/a

R. Nen        643   4   0     45  42  314   715.0   n/a    138   n/a

Here's a quick comparison of the three groups:

        G   GS   GS(d)   W    L    Sv     IP      RIP    ERA+  RERA+

A    16769  632  580   1643 1538  5422  26022.3  21851.3  130  136

B     5700  217  119    710  691  1505  10598.3   9852.7  121  126

C     6217  216    0    656  615  1832  10068.3    n/a    119  n/a 

In general the trend seems to be that the more data we have on these pitchers, the better that data reflects on them. Note the improved RERA+ for the A's and the B's.

Back to Wilhelm. Poring over his stats, I became concerned about the impact his one year as a regular starter (1959, 32 GP, 27 GS, 226 IP, 173 ERA+) had on his overall stats. So I decided to cobble together an estimate of his Relief ERA+. Knowing his total number of starts and innings as a starter, I calculated his number of innings pitched per start (7.39), and then resolved his pitching lines for each year he started games:

      G  GS   IP     SIP    RIP    ER    SER   RER  IP/GR

1958 39  10  131.0   73.9   57.1  34.0  19.2  14.8   1.97

1959 32  27  226.0  199.6   26.4  55.0  48.6   6.4   5.29

1960 41  11  147.0   81.3   65.7  54.0  29.9  24.1   2.19

1961 51   1  109.7    7.4  102.3  28.0   1.9  26.1   2.05

1963 55   3  136.3   22.2  114.2  40.0   6.5  33.5   2.20

                    384.3              106.0

Not a bad estimate; his ERA as a "starter" here is 2.48 compared to his career ERA of 2.52. But the one thing which troubled me about this was the last column, the estimated innings pitched per relief appearance. For 1958, this comes out to over 5 innnings pitched per appearance. I decided to rerun the numbers using a higher estimate for that season (8.0 IP/GS) and a lower estimate for all the others (6.75):

      G  GS   IP     SIP    RIP    ER     SER    RER   IP/GR

1958 39  10  131.0   67.3   63.7  34.0   17.5   16.5    2.20

1959 32  27  226.0  216.0   10.0  55.0   52.6    2.4    2.00

1960 41  11  147.0   74.0   73.0  54.0   27.2   26.8    2.43

1961 51   1  109.7    6.7  102.9  28.0    1.7   26.3    2.06

1963 55   3  136.3   20.2  116.1  40.0    5.9   34.1    2.23

             384.3                      104.9

My extra work eliminates only one more run, but it does get his innings pitched per appearance down to a more uniform range. I then removed the totals from his line and recalculated his ERA+ as a "reliever": 145, compared to his overall 146. Not a huge difference in the grand scheme of things, but enough to satisfy a few nagging doubts I had about the impact of that 1959 season.

Using Wilhelm's Relief ERA+ and Relief IP, we can now calculate a new Hoyt Constant so that the man winds up with an even 100. In Rifkin's original study it was .4051 (100/246.8), here it becomes .4086 (100/244.73).

One more hurdle remains: how to avoid overestimating the number of Hoyts for the Group B and Group C pitchers. I decided to dock them a small amount for each missing start as a percentage of their total appearances, settling on the following formula:

Hoyt = G - (1.5*(mGS)/G) * uH * Hc

G is Games, mGS is missing Games Started (the ones we DON'T have data for), uH is Unadjusted Hoyts, and Hc is the Hoyt Constant. I tested the factors of 1 to 3 in increments of 0.5, and 1.5 provided a good equilibrium; anything more and you penalize the old swingmen too much, anything less and you reward them too much for piling up the innings as a starter. For what it's worth, I also ran the calculations another way, using a very reasonable 6 IP/GS for the missing Games Started; the results are almost identical.

Anyway, and without further ado, here is new Hoyt list:

              IP     RIP    ERA+  RERA+  Hoyt

Wilhelm     2254.3  1870.0  146   145   100.0

Gossage     1809.3  1578.0  126   139    90.5

Smith       1289.3  1252.3  132   143    85.7

Tekulve     1436.3  1436.3  132   132    85.2

Franco      1150.3  1150.3  143   143    83.5

Quisenberry 1043.3  1043.3  146   146    82.4

Wetteland    765.0   683.0  148   165    82.3

Lyle        1390.3  1390.3  127   127    82.2

Fingers     1701.3  1656.7  119   122    82.0

Orosco      1261.3  1243.0  130   132    81.0

Henke        789.7   789.7  156   156    81.0

McDaniel    2139.3    n/a   109   n/a    80.9

Garber      1510.0  1505.7  117   117    79.7

Stanley     1707.0  1159.0  118   131    78.8

Sutter      1042.3  1042.3  136   136    78.3

Marshall    1386.7  1285.0  118   124    77.9

Jackson     1141.7  1108.0  127   131    77.7

Jones       1128.3  1112.3  130   130    77.4

McMahon     1310.7    n/a   119   n/a    76.9

McGraw      1514.7  1435.3  116   121    76.9

Burgmeier   1258.7  1248.7  119   120    76.2

Eckersley   3285.7   807.3  116   143    76.0

Lavelle     1085.0  1077.7  126   128    75.8

Perranoski  1174.7    n/a   123   n/a    75.7

Hiller      1242.0  1012.0  134   135    75.5

Carroll     1353.3  1299.0  120   123    75.5

Hoffman      701.0   701.0  146   146    74.9

Hernandez    775.0    n/a   143   n/a    74.8

Montogmery   868.7   863.7  134   136    74.4

Reardon     1132.7  1132.7  121   121    74.1

Nen          715.0    n/a   138   138    71.3

Face        1375.0    n/a   109   n/a    70.9

Aguilera    1291.3   740.3  117   131    69.7

Myers        884.7    n/a   122   n/a    67.4

Miller      1694.0    n/a   115   n/a    67.3

Even with the slight deduction for five missing starts, Gossage clearly leaps into second place in this study. Smith edges Tekulve for third place and Franco's alone in fifth. The next seven pitchers are separated by a mere 1.5 Hoyts. Rollie Fingers is right in the middle of that pack. In his original piece, Rifkin used Fingers' score to define the cutoff for Hall of Fame relievers. By this measure, Smith, Tekulve, Franco, Quiz, Wetteland, and Lyle should get the nod, while Orosco, Henke, McDaniel, Garber, Stanley, Sutter and a whole bunch of others fall by the wayside.

Intuitively, this isn't a bad conclusion, but it's worth remembering that Fingers' exact position might be considered somewhat fluid. We're missing 29 of his starts, and additional data (say, Retrosheet splits for 1970, when he started 19 games) could shift his position. If I'd used a different deduction factor, say 2.0 instead of 1.5 per missing start, it would have knocked him below Orosco and Henke at 80.7. A deduction factor of 1.0, on the other hand, would slide him past Quiz, Wetteland, and ol' Sparky at 83.3. Admittedly, one of the reasons I settled on 1.5 was because he fit into the middle of this grouping rather than significantly beyond or behind it.

It's just as well that we don't depend too much on Fingers' exact position, because as a barometer of what makes a Hall of Fame reliever, it's the definition of a slippery slope. But more importantly, the question is, is the Hoyt Scale alone enough to tell us who belongs in the Hall and who doesn't? I don't think so. It ignores postseason credentials, awards, and other factors. But it's of great help in pointing us in the right direction.

Let's remember what the Hoyt Scale does and doesn't do. The Hoyt is a measure of career value for relievers based entirely on runs and innings and the pitcher's performance relative to the league average. It doesn't take into account peak value. It dismisses any performance a pitcher had as a starter. It ignores the relatively trivial aspect of the reliever's W-L record, and somewhat helpfully shades us from being influenced by save totals. It's worth noting how the all-time save leaders rank:

            S   Hoyt  rank

Smith      478  85.7   3

Franco     422  83.5   5

Eck        390  76.0  22

Reardon    367  74.1  30

Hoffman    352  74.9  27

Myers      347  67.4  34

Fingers    341  82.0   9

Wetteland  330  78.1   7

Hernandez  320  74.8  28

Aguilera   318  69.7  33

Nen        314  71.3  31

Henke      311  81.0  11

Gossage    310  90.5   2

Montgomery 304  74.4  19

Jones      303  77.4  18

Sutter     300  78.3  15

Most of the more recent closers don't fare so well on this list, given their low number of innings pitched; Wetteland is the exception. On the contrary, the Hoyt rewards yeomen who racked up quality innings amid little fanfare. Tekulve, Orosco, McDaniel, Garber, Burgmeier, and Lavelle aren't exactly tip-of-the-ongue names when it comes to relief aces, but those guys were very good for a long time. Not Hall of Famers, perhaps, but no slouches either.

I should add somewhere in here that among the lower reaches of our "Top 35 (Guys Whose Hoyts I Bothered to Calculate)" there are probably pitchers I've omitted who would score just as well, especially among active players and players whose splits I don't have. Today's free-agent signing Steve Reed, a guy I suggested the Yanks find a spot for, rolls in at a respectable 71.5. Among older pitchers, Ron Reed, Dave Smith, and Steve Bedrosian are around 70 as well. If anyone finds a pitcher above 75 who's not active and who's missing from this list, let me know.

Among our three candidates for BBWAA election this year, Gossage has a clear edge on Smith, and Sutter's even further back. I still want to examine what Win Shares and the Leverage Index tell us about these relievers, but rather than dragging this out even longer, I'll hold that for another day.