Wednesday, January 09, 2008

Fan Fielding Survey versus range measures

This is the seventh part of my series on fielding in 2007. The table of contents for the entire series is listed below:

Basic fielding stats
Converting Zone Rating to something useful
Revised Zone Rating
Probabilistic Model of Range
Fielding Bible
Ultimate Zone Rating
Fan Fielding Survey versus range measures
Outfield arms
Ranking the second basemen
Ranking the shortstops
Ranking the third basemen
Ranking the first baseman
Ranking the center fielders
Ranking the right fielders
Ranking the left fielders
What about catchers?

Today, I'm going to revisit Tom Tango's Fan Fielding Survey. Tango anually conducts a survey on defensive skills which many of you completed last year. For those of you who are not familiar with the survey, he asked fans to rate the fielding skills of players on their favorite teams just based on observation and instructed them not to use any stats at all. As you can see on the ballot, fans were asked to scout players on reaction/instincts, acceleration/first few steps, speed, hands, release/footwork, throwing strength and throwing accuracy.

He then tabulated the results which can be seen in detail on his site. Each player ends up with a score between 0 and 100 on each of the 7 skills. Tango explains that the league average rating for each of the 7 categories is 50 and that a player with a rating of 70 or better is in top 16% in the league.


Tango also came up with an idea on how to turn these ratings into runs scored above average. First, weighted averages are created for each player based on the importance of the skills at his position. The weights are shown on Table 1 below. I'm not exactly sure how he decided upon these weights but they make intuitive sense. For example, a third baseman's arm strength carries more weight than his speed. On the other hand, speed and acceleration are very important for outfielders.


Table 1: Weights for skills on fan fielding survey


Pos

Instincts

First Steps

Speed

Hands

Release

Arm Strength

Accuracy

C

1.3

0.3

0.3

1.3

1.3

1.3

1.3

1B

1.6

1.6

0.4

1.6

0.8

0.4

0.4

2B

1.6

1.6

0.8

1.6

0.8

0.4

0.4

SS

1.5

1.5

0.7

0.7

1.5

0.7

0.4

3B

0.9

0.9

0.5

0.9

0.9

1.9

0.9

LF

1.0

2.0

2.0

1.0

0.5

0.3

0.3

CF

1.0

1.9

1.9

1.0

0.5

0.5

0.2

RF

0.9

1.9

1.9

0.9

0.5

0.5

0.5


Once the weighted average is computed for a player, you then subtract from it the average score for all players at his position. I did this for all players and compared the distribution of scores to the distribution of runs scored per 150 games (RSAA/150) for PMR and RZR. It turned out that the distributions were very similar with standard deviations of 14 for PMR, 15 for RZR and 15 for fan fielding. Thus, I used these scores as RSAA/150 estimates for fan fielding.


Tango recommended multiplying the result by .7 to get run estimates because he was comparing to the UZR distribution which had a smaller standard Deviation (at least in 2003). I don't have all the UZR scores so I compared to distributions which I did have.


The results for the Tigers are in Table 2. If you are skeptical that you can turn fan observations into run estimates, then just think of the results as above or below average skill for the position.Table 3 compares the range measures discussed previously with the fan fielding estimate. The Fan Fielding results for the Tiger players correlate reasonably well with the range measures. Players that did well on the range measures - Curtis Granderson , Brandon Inge and Placido Polanco - were also given high scores by the fans. The fans also agreed that Carlos Guillen was a poor fielder last year. Magglio Ordonez and Sean Casey who went up or down according to the measure, were ranked below average by the fans.


One player who was ranked radically lower by the fans was Jacque Jones. There are a couple possible reasons for this. First, the Cubs had only 25 voters and there was a low agreement among them in (.55) in rating Jones. This makes me think that a small number of fans may have brought down his average. Another reason was that the range measures do not consider an outfielder's ability to prevent base runners from advancing with his arm and, according to the fans, Jones has poor throwing skills. I'll talk more about throwing arms shortly.



Table 2: Fan Fielding Scores for Tigers - 2007


POS

Player

n

instinct

first step

speed

hands

release

arm strength

accuracy

RSAA/150

1B

Casey

44

57

20

11

71

43

37

45

-4

2B

Polanco

52

81

67

54

89

86

56

89

17

3B

Inge

53

83

83

68

73

75

91

62

17

3B

Cabrera

9

55

29

28

42

51

73

33

-12

CF

Granderson

52

82

84

81

81

63

47

45

9

CF

Jones

25

47

62

66

53

32

25

6

-15

LF

Monroe

38

37

46

47

38

49

58

48

3

RF

Ordonez

50

52

39

36

52

45

48

51

-11

SS

Renteria

22

64

52

52

67

70

61

72

-1

SS

Guillen

52

52

41

42

29

38

47

31

-22


Table 3: Comparison of runs of RSAA/150 across methods


PLAYER

RZR

ZR

PMR

+/-

UZR

FFS

Granderson

34

10

19

18

18

9

Jones

24

9

13

42

N/A

-15

Inge

13

12

16

18

12

17

Polanco

12

4

9

8

N/A

17

Monroe

9

7

-12

N/A

N/A

3

Renteria

5

-9

-4

N/A

N/A

-1

Ordonez

-5

10

-3

N/A

14

-10

Casey

-8

8

-15

N/A

8

-4

Cabrera

-13

-16

-26

-20

-28

-12

Guillen

-18

-7

-18

N/A

-24

-22

2 comments:

  1. I am not a fan of the fan fielding survey.

    It does exactly the opposite of what these complex methods of analysis are supposed to accomplish. The complex methods aim to make things more objective by avoiding subjective measures of ability. Instead, this method uses the most subjective measure one can find, human opinion.

    ReplyDelete
  2. I have mixed feelings about the value of the survey. Like you said, it is very subjective and I wouldn't use it as a stand alone tool. However, I think it has some value. One good thing is that it gets fans interested in the evaluation of fielding.

    I think it's also interesting to see how fan perceptions correlate with actual performance. The correlation is actually not too bad. Of course, most of the fans that participated (at least the Tigers fans) are very knowledgable fans.

    Another thing is that there is still some disagreement among the objective stats for some players. The survey gives us another piece of evidence for those players. It might help to give us clues about why the objective measures vary for certain players.

    Lee

    ReplyDelete

Twitter

Blog Archive

Subscribe

My Sabermetrics Book

My Sabermetrics Book
One of Baseball America's top ten books of 2010

Other Sabermetrics Books

Stat Counter