In nflverse/nflseedR: Functions to Efficiently Simulate and Evaluate NFL Seasons

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
library(nflseedR)
options(digits = 3)
options(nflreadr.verbose = FALSE)
options(keep.source = TRUE)
if (future::supportsMulticore()){
  future::plan("multicore")
} else {
  future::plan("multisession")
}

nflseedR 2.0 introduced a new approach to NFL season simulations with the new function nfl_simulations(). It is a new approach with a different api and a significant performance boost compared to simulate_nfl().

The general idea is that the user only passes a table with NFL schedule data to the function. Then all missing results are simulated. It is possible to limit the simulations to the regular season or to simulate playoffs afterwards. You can also switch on the determination of the draft order if required.

What are the advantages of this approach?

You have full control over which games are simulated.
It is even easier to answer questions like "what if team x wins the next two games?"
You can put together your own playoffs.
Waiving postseason or draft order improves run time.

It is still possible to pass your own function for determining the weekly results and all the additional data required for this to the simulator. The user has full control and nflseedR takes care of standings, tiebreakers and everything else with maximum efficiency.

Function API

The detailed documentation of the function parameters can be found directly in the function help page (click nfl_simulations()). This section explains the big picture.

At its core, nflseedR only requires 2 things to run NFL simulations.

The games of a season for which missing results are to be simulated.
A function to compute_results of those games.

Simulations are performed on a weekly basis. The reason for this is that Elo-based approaches have proven successful for NFL simulations. This requires updating Elo values based on one week's results and using the new values for next week's simulation. nflseedR provides a default function for compute_results which is named nflseedR_compute_results so that the simulator is functional right from the start.

If not provided by the user (we'll discuss how to do this later), the function picks a random Elo for every team, and uses those as the starting Elo ratings for all teams. It will adjust Elo ratings within each simulation as each week is simulated. (The Elo model used is loosely based off of that of FiveThirtyEight.)

Quick Start

Let's do a quick run-through and look at the output. In the first step we use nflseedR's compute_result function and let this function compute random Elo values at the beginning of the simulation. In this case, all we need is a list of games. There is also an example of this for demonstration purposes. We will now invoke it.

all_games <- nflseedR::sims_games_example |> dplyr::filter(game_type == "REG")
DT::datatable(all_games)

The example is the entire 2022 NFL regular season in which random results were removed. Now we use this list of games and let nflseedR simulate the missing results. Once the regular season is over, it determines the playoff participants and then simulates the playoffs. For this example, we choose 4 simulations in 2 chunks. Of course, 4 simulations is not a realistic number. Commonly 10k simulations are performed, but the author strongly recommends that more than 10k simulations are carried out. At least 25k, better 50k. Due to the efficiency of the new simulator, this is no longer a problem.

sims <- nflseedR::nfl_simulations(
  games = all_games,
  simulations = 4,
  chunks = 2
)

Before we take a closer look at the contents of a simulation, we will first use this example in the next section to get an overview of the output of an nflseedR simulation.

Simulation Output

The output of nfl_simulations() is a list of class "nflseedR_simulation" that holds five data frames with simulation results as well as a list of parameters used in the simulation. Here are the contents of each:

standings

One row per team per simulation. Variable names and their descriptions are provided in the table below.

DT::datatable(
  nflseedR::dictionary_standings,
  filter = "top",
  options = list(scrollX = TRUE),
  rownames = FALSE
)

games

One row per game per simulation. Variable names and their descriptions are provided in the table below.

DT::datatable(
  nflseedR::dictionary_games,
  filter = "top",
  options = list(scrollX = TRUE),
  rownames = FALSE
)

overall

One row per team, aggregated across all simulations. Sorted by conf then division then team. This is a good reference for looking at futures bets, since this should represent your model's chances of various things happening for each team. Variable names and their descriptions are provided in the table below.

DT::datatable(
  nflseedR::dictionary_overall,
  filter = "top",
  options = list(scrollX = TRUE),
  rownames = FALSE
)

team_wins

Each team has a row for 0 wins, 0.5 wins, 1 win, 1.5 wins, ... , 15.5 wins, 16 wins. These are aggregated across all simulations and are intending to represent your model's probability for teams being below or above a certain win total, making them an excellent reference for using your model to make win total bets. Variable names and their descriptions are provided in the table below.

DT::datatable(
  nflseedR::dictionary_team_wins,
  filter = "top",
  options = list(scrollX = TRUE),
  rownames = FALSE
)

game_summary

One row per game, aggregated across all simulations. Sorted by game_type then week then away_team then home_team. Variable names and their descriptions are provided in the table below.

DT::datatable(
  nflseedR::dictionary_game_summary,
  filter = "top",
  options = list(scrollX = TRUE),
  rownames = FALSE
)

sim_params

a list of parameters used in the simulation. The list contains the following parameters which are described in nfl_simulations()

nfl_season
playoff_seeds
simulations
chunks
byes_per_conf
tiebreaker_depth
sim_include
verbosity
sims_per_round
nflseedR_version
finished_at

Realistic Example

In the quick start example above, we have simulated a few games of the 2022 season based on randomly generated Elo values. Now let's run through a slightly more realistic example. This time we simulate the complete 2024 season and use market implied team ratings.

First of all, we need the games. We load the 2024 season and remove the postseason, because we want to simulate the postseason as well. We also remove all results because nflseedR only simulates missing results.

games <- nflreadr::load_schedules(2024) |> 
  dplyr::filter(game_type == "REG") |> 
  dplyr::mutate(
    result = NA_integer_,
    away_score = NA_integer_,
    home_score = NA_integer_
  )

Now we need the team ratings. The following table shows market implied team spreads. These are from August 2024 and are based on all look-ahead point spreads from various sports books at that time just before the 2024 season started. The number represents the points the corresponding team would be favored by against an average team on a neutral field.

team_ratings <- tibble::tribble(
  ~team, ~spread,
  "ARI", -2.4,
  "ATL",  0.0,
  "BAL",  3.9,
  "BUF",  2.1,
  "CAR", -5.0,
  "CHI", -0.2,
  "CIN",  2.9,
  "CLE",  1.1,
  "DAL",  2.1,
  "DEN", -4.0,
  "DET",  2.6,
  "GB",   1.8,
  "HOU",  1.8,
  "IND", -0.6,
  "JAX",  0.1,
  "KC",   4.5,
  "LA",   0.9,
  "LAC", -0.7,
  "LV",  -2.7,
  "MIA",  1.6,
  "MIN", -2.1,
  "NE",  -4.5,
  "NO",  -2.2,
  "NYG", -3.5,
  "NYJ",  2.1,
  "PHI",  2.0,
  "PIT", -0.6,
  "SEA", -1.5,
  "SF",   5.2,
  "TB",  -1.8,
  "TEN", -3.2,
  "WAS", -3.2
)
setNames(team_ratings$spread, team_ratings$team)

We use a very simple approach to convert the spreads into Elo ratings.

team_ratings$elo <- 1500 + 25 * team_ratings$spread

The default compute_results function expects team ratings as named vector.

team_ratings <- setNames(team_ratings$elo, team_ratings$team)
print(team_ratings)

We have the games and Elo ratings going into the season. We can now use them to simulate, choose 50k simulations and turn off user messaging with the verbosity argument for maximum speed. Don't be afraid of 50k simulations. At the time of writing, this code chunk takes less than 90 seconds to run on a 2022 MacBook Air with M2 chip.

# We set a seed for reproducible results
# Please see section "Reproducible Random Number Generation (RNG)" in the 
# help page of nfl_simulations for more details on the seed type "L'Ecuyer-CMRG"
set.seed(5, "L'Ecuyer-CMRG")

sims <- nflseedR::nfl_simulations(
  games = games,
  elo = team_ratings,
  simulations = 50000,
  chunks = 20,
  verbosity = "NONE"
)

Please pay attention to how we pass the team ratings to the simulation. The function nfl_simulations() provides dots ... as function argument. You can pass any R object as argument to the dots as long as the argument is named. In your compute_results function you then only have to search the dots for the objects you need. The default function does exactly that. It searches for the object named elo, and if this object exists, then these values are used as initial Elo values and then updated from week to week. More details can be found in the source code of the function nflseedR_compute_results (see next chunk).

cli::cat_line("This is the source code of `nflseedR_compute_results`. 
Note that it is written in data.table for maximum performance and 
consistency with the rest of the simulation code. 
You are not required to write your own function in data.table.")
cli::cli_code(deparse(nflseedR::nflseedR_compute_results, control = "all"))

The output contains a lot of pre-aggregated information, as well as the individual results from each game of each simulation. For example, let's look at the overall results of the Chargers:

sims$overall |> dplyr::filter(team == "LAC") |> knitr::kable()

We can see the Chargers got r round(sims$overall$wins[sims$overall$team == "LAC"], 1) wins on average. They made the playoffs r scales::label_percent(1)(sims$overall$playoff[sims$overall$team == "LAC"]) of simulations, won the division in r scales::label_percent(1)(sims$overall$div1[sims$overall$team == "LAC"]), won the Super Bowl in r scales::label_percent(1)(sims$overall$won_sb[sims$overall$team == "LAC"]), and in r scales::label_percent(1)(sims$overall$draft5[sims$overall$team == "LAC"]) did they receive a top five draft pick. The teams section of the output will show how a team did in each simulated season.

Now let nflseedR summarize the simulation for you by using summary() with the nflseedR simulation object. This will print a gt table.

summary(sims)

Use Your Own Model

User-defined `compute_results`

But of course the real value of nflseedR is putting in your own model into the simulator. To accomplish this, you can write your own compute_results function which will determine the outcome of games instead.

Now it gets important: nflseedR will use your function during the simulations without any checks. The reason for this is that we want maximum performance and such tests would slow us down. In addition, repeated checks are not necessary if we have checked once in advance that the function works as nflseedR expects it to. And this is exactly what simulations_verify_fct() is for. So if you want to use your own compute_results function, all you need to do is check how it works in advance with simulations_verify_fct(). If there are no errors, then your function works exactly as nflseedR needs it to simulate. Please see the documentation of simulations_verify_fct() for a detailed description of expected compute_results behavior.

As an example for a custom compute_results function, here's a very stupid model that makes the team earlier alphabetically win by 3 points 90% of the time, and lose by 3 points the other 10% of the time.

stupid_games_model <- function(teams, games, week_num, ...) {
  # make the earlier alphabetical team win 90% of the time
  games <- games |> 
    dplyr::mutate(
      result = dplyr::case_when(
        !is.na(result) | week != week_num ~ result,
        away_team < home_team ~
          sample(c(-3, 3), dplyr::n(), prob = c(0.9, 0.1), replace = TRUE),
        away_team > home_team ~
          sample(c(-3, 3), dplyr::n(), prob = c(0.1, 0.9), replace = TRUE),
        TRUE ~ 0
      )
    )

  # return values
  list(teams = teams, games = games)
}

When you create this function, the first two inputs are data on the teams (one row per team per sim), and data on the games (one row per game per sim). The third argument is the week number currently being simulated, as only one week is processed at a time.

Your function's job - by whatever means you choose - is to update the result column for that week's games in each of the sims with the number of points the home team won by (or lost by if negative, or 0 if the game ended in a tie).

It returns both the teams and the games data. It does both because this way you can store information in new columns by team or by game to use in the next call. For example, the default function updates a team's Elo after the game, and stores it in the teams data. When the simulator processes the next week, it uses the updated Elo rating to inform the team's next game.

We can verify that the function works as required

nflseedR::simulations_verify_fct(stupid_games_model)

Let's run a simulation with stupid_games_model and see what happens:

sims2 <- nflseedR::nfl_simulations(
  games = games,
  compute_results = stupid_games_model,
  simulations = 500,
  chunks = 1,
  verbosity = "NONE"
)

sims2$overall |> 
  dplyr::arrange(team) |> 
  gt::gt_preview(top_n = 5, bottom_n = 5)

As you might expect, the earliest alphabetical teams win a lot. The Cardinals won the Super Bowl in r scales::label_percent(1)(sims2$overall$won_sb[sims2$overall$team == "ARI"]) of seasons! Meanwhile, the teams at the bottom alphabetically are virtually certain to be at the top of the draft order with the Commanders picking 1st overall in r scales::label_percent(1)(sims2$overall$draft1[sims2$overall$team == "WAS"]) of seasons.

Adding In Your Own Data

This is all well and good, you might be thinking, but your model works off of other data not in the simulator! How can that work? This is where we utilize R's ability to have generic arguments.

The ... in the function definition means that the function can be called with any number of additional arguments. You can name these whatever you want, as long as they're not already the name of other defined arguments.

When you call the nfl_simulations() function, it too uses the ... syntax, which allows you to pass in any number of additional arguments to the function. The simulator will in turn pass these on to your function that processes games.

For example, let's slightly modify our last example:

library(rlang)
biased_games_model <- function(teams, games, week_num, ...) {

  # collect all arguments in dots
  args <- list(...)

  # if "best" or "worst" are in dots, assign them to best of worst
  # if not, then args$best and args$worst will be NULL and we replace 
  # them with empty strings
  best <- args$best %||% ""
  worst <- args$worst %||% ""

  # make the best team always win and the worst team always lose
  # otherwise, make the earlier alphabetical team win 90% of the time
  games <- games |>
    dplyr::mutate(
      result = dplyr::case_when(
        !is.na(result) | week != week_num ~ result,
        away_team == best | home_team == worst ~ -3,
        away_team == worst | home_team == best ~ 3,
        away_team < home_team ~
          sample(c(-3, 3), dplyr::n(), prob = c(0.9, 0.1), replace = TRUE),
        away_team > home_team ~
          sample(c(-3, 3), dplyr::n(), prob = c(0.1, 0.9), replace = TRUE),
        TRUE ~ 0
      )
    )

  # return values
  list(teams = teams, games = games)
}

This allows us to define best and worst, and use that information to determine a result (in this case, have the best team always win and the worst team always lose). While best and worst are in this example single-length character vectors, they can be data frames or any other R data type.

Let's simulate using this:

sims3 <- nflseedR::nfl_simulations(
  games = games,
  compute_results = biased_games_model, 
  simulations = 500,
  chunks = 1,
  best = "CHI", 
  worst = "GB",
  verbosity = "NONE"
)
sims3$overall |> 
  dplyr::arrange(-wins) |> 
  gt::gt_preview(top_n = 5, bottom_n = 5)

And this shows exactly what we expect. By defining the Bears as the best team, they always go 17-0, win the division, and win the Super Bowl. Similarly, the Packers always go 0-17, and never make the playoffs.

Passing Data in from One Week to the Next

Sometimes though you want your data to keep updating as the simulation progresses. For example, the Elo-based model that nflseedR uses by default updates each team's Elo after each game. You can pass in the starting Elo values per team, and as games are simulated, update the Elo values for each team and store them in the teams data. This column will be part of the teams data passed into your function when the following week is simulated and your function is called.

Read the code and comments of the source of nflseedR_compute_results for specific tips on doing this but here are good ones:

You can add columns to teams and/or games if you want.
When doing joins to do the above, do left joins to make sure no rows are removed.
Remove any "helper" columns you generate along the way you don't actually need before returning.
Make sure any column doesn't get blindly joined on so it has .x and .y versions in week 2 and R throws an error because an expected column name doesn't exist.