Comparing bottom-up and top-down ratings for individual soccer players

Correctly assessing the contributions of an individual player in a team sport is challenging. However, an ability to better evaluate each player can translate into improved team performance, through better recruitment or team selection decisions. Two main ideas have emerged for using data to evaluate players: Top-down ratings observe the performance of the team as a whole and then distribute credit for this performance onto the players involved. Bottom-up ratings assign a value to each action performed, and then evaluate a player based on the sum of values for actions performed by that player. This paper compares a variant of plus-minus ratings, which is a top-down rating, and a bottom-up rating based on valuing actions by estimating probabilities. The reliability of ratings is measured by whether similar ratings are produced when using different data sets, while the validity of ratings is evaluated through the quality of match outcome forecasts generated when the ratings are used as predictor variables. The results indicate that the plus-minus ratings perform better than the bottom-up ratings with respect to the reliability and validity measures chosen and that plus-minus ratings have certain advantages that may be difficult to replicate in bottom-up ratings.