github_api_iterate_pages: GitHub Page Iterator

In sailuh/kaiaulu: Kaiaulu

GitHub Page Iterator

Description

GitHub API endpoints return data in pages, each containing by default 100 entries. This iterator can be used to iterate over the next page in order to download all the project's data available from the endpoint (up to a user-defined maximum or the remaining available requests in the used user's token). This function can differentiate between data downloaded from search endpoint or not, in which the issues are differently nested. It also differentiates these endpoints from commit data, which also uses a different level of nesting. This is important in order to extract the minimum and maximum value of time created for each page for the naming convention, which is owner_repo_(min time)_(max time).json.

Usage

github_api_iterate_pages(
  token,
  gh_response,
  save_folder_path,
  prefix = NA,
  max_pages = NA,
  verbose = TRUE
)

Arguments

token	Your GitHub API token
gh_response	A response returned by any GitHub endpoint which is paginated (e.g. github_api_project_commits).
save_folder_path	A folder path to save the downloaded json pages "as-is".
prefix	Prefix to be added to every json file name
max_pages	The maximum number of pages to download. MAX = Available token requests left
verbose	Boolean value that prints operating messages when set to TRUE, does not print when false. Operating messages may be details about certain parts of the code correctly executing or printing names of files created, etc.

References

For details see https://docs.github.com/en/free-pro-team@latest/rest/guides/traversing-with-pagination.

sailuh/kaiaulu documentation built on Dec. 10, 2024, 3:14 a.m.