In irwinsnet/FIRST_api_R: Tools to Download and Convert FIRST API Data to R Data Frames

The Getting Started vignette covered how to use the GetSession() function to create a Session object and how to use that Session object in other firstapiR functions. This vignette provides an overview of the remaining firstapiR functions. For detailed information on each function, type help(function_name) at the console or in the RStudio help view search box.

Version 2.0.1

This guide is for version 2.0.1 of the firstapiR package. Guidance for version 1.0.0 of firstapiR is located here .

Simple Functions for Retrieving FIRST Data

There are several functions that take no arguments other than the Session argument.

GetServerStatus(session) ~ Retrieves information about the FIRST API server.

GetSeason(session) ~ Gets information about the season specified in session$season.

GetDistricts(session) ~ Provides a list of FIRST districts.

GetAwardsList(session) ~ Obtains a list and description of awards available to FRC teams.

Data Retrieval Functions with Additional Arguments

All other firstapiR functions take at least a few additional arguments. Most of the additional arguments are optional and, usually, only one or two of the additional arguments are used at any one time.

GetEvents(session, event, team, district, exclude_district) ~ Provide information about FRC competition events.

GetTeams(session, team, event, district, state, page) ~ Gets information about FIRST FRC teams.

GetSchedule(session, event, level, team, start, end) ~ Downloads the match schedule for a specific event.

GetHybridSchedule(session, event, level, start, end) ~ Downloads the match schedule, along with results for completed matches.

GetMatchResults(session, event, level, team, match, start, end) ~ Gets the scores for the selected matches.

GetScores(session, event, level, team, match, start, end) ~ Downloads the scoring sheet for each match.

GetAlliances(session, event) ~ Obtains the playoff alliances.

GetRankings(session, event, team, top) ~ Gets the team rankings.

GetAwards(session, event, team) ~ Lists the awards and recipients for an FRC competition.

Function Arguments

Many of the same arguments are used in several different firstapiR functions.

team

The four digit FRC team number.

districts

A code representing a FIRST district. Use GetDistricts() to obtain the district codes that can be used in other firstapiR functions:

``` {r districtCodes} library(firstapiR) sn <- GetSession("username", "key") districts <- GetDistricts(sn)

### event

An event code, such as _WAAMV_ (disctrict competition held at Auburn MountainView
High School in Washington State) or _PNCMP_ (Pacific Northwest District
Championships). Use the GetEvents() function to get the event codes:

``` {r eventCodes}
sn <- GetSession("username", "key")
PNW_events <- GetEvents(sn, district = "PNW")
PNW_events[, c("code", "name")]
PNW_events[, c("code", "dateStart")]

level

Either "playoff" or "qual". Defaults to "qual".

start, end, match

Integers specifying the range of matches that will be included in the returned data.

Pages Argument

There is one firstapiR function, GetTeams(), that accepts the page argument. When requesting data on FRC teams, the FIRST API will split the results into multiple pages if there are more than 65 teams in the response, requiring a separate HTTP request for each page. Users that are requesting the data frame format can ignore the page argument because GetTeams will automatically detect if there are multiple pages in the response, conduct a separate HTTP request for each page, and merge the results into a single data frame. This feature is not available when XML or JSON formats are sepecifed in the Session$format parameter. Users requesting XML or JSON formatted data will have to call GetTeams() for each page of data, incrementing the page argument for each request.

Modified-Since Arguments

There are two additional arguments that can be passed to almost all firstapiR functions (only GetSession() and GetServerStatus() do not accept them). The two arguments, which are always optional, are mod_since and only_mod_since. These two arguments help to reduce the load on the FIRST API server by allowing the server to skip the database query and send only a short HTTP response when the FIRST API server data has not changed since the user last queried the server.

Both of these arguments accept a character vector containing an HTTP formatted date and time string. If the user includes the mod_since argument, and no data has changed on the FIRST API server since the date and time specified in the argument, the FIRST API server will provide an HTTP 304 response with no content. When this happens, irstapiR functions will return a logical character vector with the value NA. Here is an example:

# Create an HTTP date-time string set to midnight GMT yesterday
mod_date <- httr::http_date(as.POSIXct(Sys.Date() - 1))

# Request recently changed data from the server
match_results <- firstapiR::GetMatchResults(sn, event = "PNCMP",
                                            mod_since = mod_date)

# Assuming there have been no updates to the data since yesterday, this
#   returns TRUE
is.na(match_results)

# The value passed to mod_since returned as an attribute, even when the
#   result is NA
print(attr(match_results, "mod_since"))

If the user includes the only_mod_since argument, the FIRST API server will return only the data that has changed since the date and time specified in the only_mod_since argument. In no changes have been made, the FIRST API server provides a 304 response and firstapiR functions return a logical vector set to NA.

# Create an HTTP date-time string set to midnight GMT yesterday
mod_date <- httr::http_date(as.POSIXct(Sys.Date() - 1))

# Request recently changed data from the server
match_results <- firstapiR::GetMatchResults(sn, event = "PNCMP",
                                            only_mod_since = mod_date)

# Assuming there have been no updates to the data since yesterday, this
#   returns TRUE
is.na(match_results)

# The value passed to mod_since returned as an attribute, even when the
#   result is NA
print(attr(match_results, "only_mod_since"))

If the mod_since or only_mod_since arguments are not specified, the corresponding mod_since or only_mod_sinceattribute attached to the return value will be set toNULL`.

Users can keep track of the date and time provided in the last_modified attribute, which corresponds to the most recent date and time that the information changed on the FIRST API server, and use this date and time to request only new data from the server. If a user repeats a firstapiR function call with the same arguments and sets the mod_since or only_mod_since argument to the last_modified attribute value from the intial firstapiR function call, the FIRST API server will return all requested data -- the same data that was provided in the first function call. Users who would rather receive a 304 response in this situation should either add at least a second to the last_modified value, or should instead use the time_downloaded attribute value to set the mod_since or only_mod_since arguments.

Data Frame Shaping Functions

Team Shape

The functions GetSchedule(), GetHybridSchedule(), and GetMatchResults() return data frames with one row per team. For example, let's take a look at the first two qualification matches from the 2016 Pacific Northwest District Championships:

sn <- firstapiR::GetSession("username", "key")
match_results <- firstapiR::GetMatchResults(sn, "PNCMP", level = "qual")
match_results[1:12, c("match", "alliance", "station", "team", "scoreAuto",
                      "scoreFinal")]

There is one team listed in each row, and it takes six rows to provide all of the information from a single match. This is called team shape. This format is useful for calculating summary statistics for individual teams. For example, to see average scores for the ten teams with the highest scores:

agg_scores <- aggregate(match_results[c("scoreFinal", "scoreAuto",
                                        "scoreFoul")],
                        list("team" = match_results$team), mean)
agg_scores[order(agg_scores$scoreFinal, decreasing = TRUE), ][1:10, ]

Data frames in team shape will have their shape attribute set to "team".

attr(match_results, "shape")

Alliance Shape

For some calculations, such as offensive power rating (OPR), the alliance shape, with three teams per row and two rows per match (one row each for the red and blue alliances), is more suitable. The function ToAllianceShape() will reshape Schedule, HybridSchedule, or MatchResults data frames from team to alliance shape.

mresults_alliance <- firstapiR::ToAllianceShape(match_results)
mresults_alliance[1:4, c("match", "alliance", "team.1", "team.2", "team.3",
                         "scoreAuto", "scoreFinal")]

Data frames in alliance shape will have their shape attribute set to "alliance".

attr(mresults_alliance, "shape")

Match Shape

Finally, Schedule, HybridSchedule, and MatchResults data frames may be converted from team to match shape.

mresults_match <- firstapiR::ToMatchShape(match_results)
mresults_match[1:2, c("match", "team.Red1", "team.Red2", "team.Red3",
                      "team.Blue1", "team.Blue2", "team.Blue3",
                      "scoreFinal.Blue", "scoreFinal.Red")]

Converting Back to Team Shape

The ToTeamShape() function will convert match and alliance data frames back to team shape. This function depends on the ReshapeWide attribute, which is added to the match and alliance data frames by R's reshape() function. ToTeamShape() will not work if the ReshapeWide attribute is deleted.

Converting Between Match and Alliance Shape

To convert a data frame from match to alliance shape or vice-versa, first convert the data frame back to team shape.

Saving and Loading Data

Use firstapiR's SaveData() function to open a save file dialog box and save data to an RDS data file. The ReadData() function opens an open file dialog and returns the value saved in the RDS file. Refer to R's documentation on the saveRDS() and `readRDS()' functions for more information on RDS data files.

SaveData(firstapiR::GetAlliances(sn, event = "WAAMV"))
evt_alliances <- firstapiR::ReadData()

Getting all Event Data

Sometimes it's handy to get all the available information for an FRC competition with single command. The GetAll() function retrieves all data for a specific event and stores the data in a list. I recommend not using this function during competitions to avoid overloading the server. Also, save the event data to a file so it only needs to be downloaded once.

# Get all data for the 2016 district competition at Auburn Mountainview HS
sn <- firstapiR::GetSession("username", "key")
all_data <- firstapiR::GetAll(sn, event = "WAAMV")

Preloaded Data

Version 2.0.1 of firstapiR includes all competition data for the 2016 FRC championships in St. Louis. Data for each subdivision and the finals are available in R lists named archimedes2016, carson2016, carver2016, curie2016, galileo2016, hopper2016, newton2016, tesla2016, and einstein2016.

# Show elements in curie2016 list:
names(firstapiR::curie2016)

# Show alliances for Curie Subdivision (first 8 columns)
firstapiR::curie2016$alliances[1:8]

Merging Team and Score Data

The GetScores() function retrieves detailed performance data for for both the red and blue alliances for each match. However the Scores data frame does not list the teams that were assigned to each alliance. The function MergeResults() merges a HybridResults data frame with a Scores data frame to create a Results data frame that contains both the detailed match performance data and the teams that were assigned to each match.

# Merge MatchResults and Scores data frames
curie_results <- firstapiR::MergeResults(firstapiR::curie2016$hybrid.qual,
                                         firstapiR::curie2016$scores.qual)
# Show structure of resulting data frame
str(curie_results)

Actually, using MergeResults() is not necessary for preloaded data (such as curie2016 or for any data structures created with the GetAll() function. These lists already contain merged data frames -- see curie2016$results.qual and curie2016$results.playoff for examples. I used the preloaded data to demonstrate the MergeResults() function because that's an easy way to do it.

Merging Qualification and Playoff Data Frames

There are several functions provide data for either qualification or playoff matches, but not both. Use R's rbind() function to merge this data into a single data frame.

# Number qualification matches
nrow(firstapiR::curie2016$matches.qual)

# Number playoff matches
nrow(firstapiR::curie2016$matches.playoff)

# Merge qualification and playoff data into a single table
all_matches <- rbind(firstapiR::curie2016$matches.qual,
                     firstapiR::curie2016$matches.playoff)

# Total Matches
nrow(all_matches)

Ask Questions

Post an issue on the firstapiR github repository if you have questions, find bugs or have recommendations for the next version.

Links

Getting Started Guide

Irwinsnet Home Page

irwinsnet/FIRST_api_R documentation built on Dec. 22, 2020, 5:12 p.m.