The motivation that lead me to the pitcher evaluation method to be
described is simple: I wanted a single performance number that could
be used to rank all pitchers regardless of when and how long they
appear in a game. The increasing specialization of pitching (starter,
long relief, middle relief, closer, mop up) and a burgeoning of
statistics used to describe each specialty, makes comparing
individual pitchers increasingly difficult. Another problem with
specialized categories is that many pitchers don't fit into such neat
categories. However, to make comparisons between traditional pitching
evaluation categories and the new measures proposed, I will do
exactly this.
The method I will present assigns a fraction of
the game win or loss to each pitcher that appears in it. The fraction
is computed from the earned runs allowed and the number of innings
pitched by each pitcher. This apportioning of the game win and loss
gives the names for the measures: apportioned wins (APW) and
apportioned losses (APL).
If the method is to prove useful, I
believe it should satisfy a number of constraints. First, it should
have an interpretation that is close to existing, well known
measures. Pitching wins and losses are probably the best known and
most widely used of the current performance measures. Apportioned
wins and apportioned losses assigns one pitching win for each team
win and one pitching loss for each team loss. At any time during the
season, the sum of all pitching apportioned wins and losses will
equal the team win and loss totals as the do the traditional pitching
win and loss measures.
A second constraint on the new measure
is that all pitchers appearing in a game should share in the win or
loss. The present pitcher win-loss assignment system creates
considerable inequities: A starter gives up 3 runs in 5 innings but
his team is leading when he is relieved. The subsequent relievers
shut out the opposition. The win goes to the starter under the
present system. Another example: the starter provides 7 scoreless
innings. One reliever allows the lead to vanish. A closer surrenders
one more run but his team wins the game in the bottom of the ninth,
4-3. The closer gets the win. The relatively small number of wins
credited to each pitcher limits minimizes the beneficial effects of
averaging in reducing these types of inequities over the course of a
season.
The examples given in the proceeding paragraph
illustrate another problem with the traditional method. Credit given
a pitcher depends on when his team provides the winning runs. My
third constraint is to base credit for winning or losing solely on
pitching performance and not on when the team's offense provides the
runs.
There are two additional constraints I feel the measure
should incorporate. There should be no credit for a loss if no earned
runs were surrendered. The pitcher win or loss should not be held
hostage to fielding performance. Similarly, I don't believe a pitcher
should receive credit for a win unless he records at least one
out.
I have chosen innings pitched and earned runs allowed as
the basic quantities that will be combined to calculate the
apportioned wins, APW, and apportioned losses, APL, performance
measures. Each pitcher's contribution to a win is calculated from:
where IP is innings pitched and ER are earned runs allowed. This
ratio has the desired dependence on IP (increasing) and ER
(decreasing). The subscript k labels a particular pitcher.
In
general, summing the contributions of each pitcher will not equal the
required 1 for the game win. The first step in calculating APW is to
sum the contribution of each pitcher (N appeared in the game):
Dividing individual pitcher contributions by T gives the desired apportioned wins for each pitcher in the game:
As before, the subscript k labels which pitcher.
This
calculation is done for each game. Analogous to traditional pitching
wins, the APW for all the games a pitcher appears in are added
together to get his season total.
Some properties of this
formula are obvious. By construction, exactly one win is assigned for
each game and the sum of all N APWk is exactly 1. If there is only
one pitcher, he gets the entire win. In a game with no earned runs,
the win is distributed proportional to the innings pitched. If a
pitcher does not record an out, his APW for the game is 0.
Also
obvious is the close relation between the definition of the earned
run average and the quantity used in the APW calculation. The primary
difference is the addition of 1 to the ER terms in the denominator.
This is a technical requirement since division by zero would render
the results meaningless. Essentially, the term used in the APW
calculation is the reciprocal of the ERA. A smaller ERA for the game
leads to a larger fraction of the win.
Apportioned losses
(APL) are defined in a manner analogous to APW. If a loss is to be
credited, each pitchers contribution is:
As with APW, the calculation is made using outs rather than IP. Again, to avoid division by 0, 1 is added to each denominator. Here too, the sum of the individual contributions is not likely to be 1 so they are summed to get the normalizing quantity:
making each pitchers fraction of the loss:
It is entirely possible to have no earned runs in a loss. It
happens many times a season. The procedure I have followed to avoid
dividing by a 0 normalizing value is to apportion the loss based
solely on innings pitched if the losing staff did not surrender an
earned run.
The properties of this formula parallel the APW
formula. There is exactly 1 loss apportioned among the staff for each
loss. If there is only one pitcher, he receives the entire loss. By
design, if a pitcher does not give up an earned run, he does not get
a portion of the loss. (Excepting the case where the losing staff
surrendered no earned runs.)
With the exception of the 1 in
the denominator, each losing fraction is proportional to the pitcher
ERA for the game. A larger ERA for the game leads to a larger
fraction of the loss.
Two examples will help clarify this.
Pitcher A completes 7 innings giving up 2 earned runs. Reliever B
pitches a scoreless inning followed by C who gives 1 run while
recording the final outs. Assuming the winning run was scored before
he left the game, the official scoring would give a win to A and a
save to C and no credit to B.
The T(win) and APW lines indicate how the computation works.
Pitcher A's APW is computed 7/11.5 = .609 . The larger part of the
win goes to the starter. The run allowed by the closer significantly
has reduced his portion of the credit. If the same scoring occurs but
this time the pitching line describes a loss, the T(loss) and APL
lines indicate how the loss apportioning calculation is done. Here
the 1 run by the closer in a single inning earns a proportionally
larger share of the loss.
In the second example, the starter D
has a rocky outing, is followed by E and F who pitch shut out ball in
relief. The following table shows the APW and APL calculation
assuming the pitcher data corresponds to either a win or a loss.
If the pitching corresponds to a win, the middle reliever get the
largest share. The starter receives some credit in spite of his poor
showing. If the pitching data corresponds to a loss the starter
receives all of it since the relievers gave up no runs.
These
tables illustrate the basic calculations. More interesting is the
application of the technique to actual data. To do this, I have
obtained full season play by play accounts of the 1996 season from
the Baseball Workshop and the 1983 season from Retrosheet,
Inc. Analysis programs are also available but I have chosen to
write my own. The ability to apply a measure of my own design at the
game or even individual at bat level for an entire season is a
significant reward for the effort invested in writing such a
program.
The team name abbreviations I use are the BBWS and
Retrosheet, Inc. 3 letter
codes. When you understand that a final "n" stands for the
National League and that a final "a" for the American,
these codes are obvious.
In the data listings that follow
there are two additional quantities I calculate from the game data.
The first of these is ENT, the point in the game a pitcher enters.
The value of this quantity, which is 0 for starters, is the average
number of innings completed at time of game entry for the
pitcher.
The second quantity I have defined to help understand
the role of pitchers in the game data is INA, innings per appearance,
the total number of innings pitched divided by the number of
appearances for a pitcher. This quantity is listed in the tables that
follow.
To select groups of pitchers from the record I have
used the following selection criteria. First only pitchers with more
than 50 innings credited have been classified. Then using the average
time of entry into the game, ENT, four classes of pitchers are
defined:
Starters: ENT < 1 Long relievers: 1 <= ENT < 5 Middle relievers: 5 <= ENT < 7 Closers: 7 <= ENT
These definitions are admittedly arbitrary but are useful in the
context of this discussion. They are used to classify pitchers in the
data presentations that follow indicated using the underlined
letters.
Figure 1 shows a plot of APW vs. Wins for 1996
starters and also the regression line relating APW and wins. The
slope, 0.77, of the regression line combined with an intercept near
zero, 0.19, indicates that for starters about 3/4 of an APW is
credited for each official win. Using the ANOVA technique the
regression line explains 91% of the variation. Apparently for
starters, the fraction of wins lost by leaving a game early is more
important than official wins lost to the relievers. Figure 4 shows
APL as a function of losses for the same group of pitchers. The slope
of the regression line is 0.71 and the ANOVA calculation indicates
the regression line explains 78% of the variance.
Similarly
for starting pitchers, a regression analysis between APL and losses
also indicates a strong correlation. The slope of the regression line
is 0.71, the intercept is 0.43 and the ANOVA calculation indicates
the regression explains 78% of the variance between the two
quantities. As with APW, the number of APL credited is a little
smaller than the official loss numbers. The new measures are very
analogous to the traditional wins and losses for starters.
Figure
2 displays a comparison of APW vs. wins for the 1996 middle relievers
using the definitions given above. The poor correlation between APW
and Wins as shown by the slope of the curve, 0.41, a high intercept
of 3.6, and a small value for the amount of variation explained by
the variance, 0.15, from the regression line all indicate that the
traditional win measure does not adequately account for the
importance of the middle reliever.
Figure 3 shows the
correlation between APW and the traditional closer measure of
excellence, the save. There were 49 pitchers in 1996 that met my
closer definition. The regression line slope is 0.12, its intercept
is 6.0 and the ANOVA calculation indicates the regression line
represents 61% of the variance. The high intercept reflects the
earning of wins by the closers. I feel this shows good correlation
between saves and APW.
Tables 1 and 2 show the entire pitching
staffs of the 1996 New York Yankees and Atlanta Braves ranked by APW.
Column P is the pitching classification (S,L,M,C) defined above. The
importance of the middle reliever, Rivera, and closer, Wetteland, to
the Yankees is clearly shown in these rankings. The Atlanta pitching
is dominated by the 3 starters, Smoltz, Glavine and Maddux with the
closer Wohlers being fourth.
The following links are to tables
containing a listing of the top 150 pitchers ranked by APW.
Relievers, both middle and closers, appear in both these lists
suggesting that the principle intent, finding a single measure that
will rank all pitchers, has been achieved. Indeed, using the APW
measure, a closer and middle reliever lead the 1983 pitcher
standings. The traditional measure of pitching excellence, the 20
game winning season, can be achieved using APW but will not be quite
so common.
Following are links to tables containing the top
150 pitchers when ranked by the APW statistic.
I acknowledge the earlier work of Stephen Grant, "A New Pitching Evaluation Tool" that was given at the 1990 SABR Convention. My work is similar in spirit to his but differs greatly in detail.
Table 1: 1996 New York Yankee Pitchers Ranked by APW
P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB M Rivera, M 61 6.1 1.2 107.2 2.090 14.5 3.1 8 3 5 130 34 S Pettitte, A 35 0.1 6.0 221.0 3.869 13.6 7.2 21 8 0 162 72 C Wetteland, J 62 7.2 1.0 63.2 2.827 10.2 2.1 2 3 43 69 21 S Rogers, K 30 0.0 5.2 179.0 4.676 9.8 6.0 12 8 0 92 83 S Key, J 30 0.0 5.1 169.1 4.677 8.5 7.5 12 11 0 116 58 S Gooden, D 29 0.0 5.2 170.2 5.010 7.6 5.2 11 7 0 126 88 M Wickman, B 70 5.2 1.1 95.2 4.422 6.0 6.7 7 1 0 75 44 M Nelson, J 73 6.2 1.0 74.1 4.359 5.6 5.3 4 4 2 91 36 S Cone, D 11 0.0 6.1 72.0 2.875 4.6 1.8 7 2 0 71 34 S Mendoza, R 12 0.0 4.1 53.0 6.792 2.2 5.2 4 5 0 34 10 Boehringer, B 15 3.1 3.0 46.1 5.439 2.1 2.5 2 4 0 37 21 Polley, D 32 7.0 0.2 21.2 7.892 1.4 4.9 1 3 0 14 11 Howe, S 25 6.2 0.2 17.0 6.353 1.2 2.3 0 1 1 5 6 Mecir, J 26 6.0 1.1 40.1 5.132 1.2 1.8 1 1 0 38 23 Kamieniecki, S 7 1.0 3.0 22.2 11.12 1.2 1.4 1 2 0 15 19 Pavlas, D 16 6.1 1.1 23.0 2.348 1.0 0.3 0 0 1 18 7 Whitehurst, W 2 0.0 4.0 8.0 6.750 0.6 0.2 1 1 0 1 2 Gibson, P 4 5.1 1.0 4.1 6.231 0.3 0.1 0 0 0 3 0 Brewer, B 4 5.1 1.1 5.2 9.529 0.0 0.7 1 0 0 8 8 Aldrete, M 1 7.0 1.0 1.0 0.000 0.0 0.0 0 0 0 0 0
Table 2: 1996 Atlanta Braves Pitchers Ranked by APW P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB S Smoltz, J 35 0.0 7.0 253.2 2.945 19.2 6.5 24 8 0 276 54 S Glavine, T 36 0.0 6.1 235.1 2.983 13.1 8.9 15 10 0 181 84 S Maddux, G 35 0.0 7.0 245.0 2.718 12.3 9.0 15 11 0 172 28 C Wohlers, M 77 7.2 1.0 77.1 3.026 11.5 3.4 2 4 39 100 21 M Mcmichael, G 73 6.2 1.0 86.2 3.219 8.8 5.5 5 3 2 78 27 M Clontz, B 81 6.1 0.2 80.2 5.690 6.8 7.9 6 3 1 49 33 S Avery, S 24 0.0 5.1 131.0 4.466 5.2 6.1 7 10 0 86 40 M Bielecki, M 40 5.1 1.2 75.1 2.628 4.3 1.6 4 3 2 71 33 M Wade, T 44 5.2 1.1 69.2 2.971 3.9 2.4 5 0 1 79 47 S Schmidt, J 19 0.1 5.0 96.1 5.699 3.3 4.4 5 6 0 74 53 Borbon, P 43 7.1 0.2 36.0 2.750 3.1 1.8 3 0 1 31 7 Borowski, J 22 7.0 1.0 26.0 4.846 1.1 3.7 2 4 0 15 13 Woodall, B 8 3.2 2.1 19.2 7.322 1.0 1.3 2 2 0 20 4 Lomon, K 6 5.1 1.0 7.1 4.909 0.3 1.2 0 0 0 1 3 Schutz, C 3 6.1 1.0 3.1 2.700 0.0 0.1 0 0 0 5 2 Thobe, T 4 8.0 1.1 6.0 1.500 0.0 0.3 0 1 0 1 0
Revisions: Jan 1, 2009 Complete 1954 - 2008
Back to the J. F. Jarvis baseball page.
Copyright 1997-2009, John F. Jarvis