In vjcitn/bbsBuildArtifacts: analyze build artifacts for Bioconductor

Introduction

We'd like to be able to get a quick overview of status for a subset of Bioconductor packages. "Status" is relative to Bioconductor version, package version, build platform, and condition of the platform.

We want to be able to work with the current artifacts provided at, e.g., https://bioconductor.org/checkResults/3.17/bioc-LATEST/report.tgz. Such gzipped tar resources are prepared for different types of resource.

suppressMessages({
 suppressPackageStartupMessages({
  library(bbsBuildArtifacts)
  library(dplyr)
  library(DT)
  library(ggplot2)
  })
})
bbsBuildArtifacts:::valid_types()

We'll focus on type bioc for now, which associates with Software packages. We don't yet know if report.gz has the same structure for all types, but we hope so.

Our objective is to learn the status and processing times for various phases of the build process for all packages, and to analyze error, warning and note events programatically.

Artifact set manager

We define an S4 class ArtifSet to manage key information about builds. ArtifSet instances are produced using setup_artifacts.

We've produced a thinned version of the BBS report.tgz accessible in the package at demo_path().

od = getwd()
td = tempdir()
setwd(td)
untar(demo_path())
af = setup_artifacts(type="bioc", version="3.17", extracted=".")
af

Information of immediate interest can be derived and tabulated in a data.frame instance.

setwd(td)
d = as.data.frame(af)
dim(d)
head(d)

We use this to get statistics about timings for package installation, building and checking.

sapply(split(d$elapsed_time, d$host), sum, na.rm=TRUE)/3600 # hours
sapply(split(d$elapsed_time, d$phase), sum, na.rm=TRUE)/3600 # hours
ggplot(mutate(d, elapsed_time_sec=elapsed_time), aes(y=elapsed_time_sec, x=host)) + 
    geom_boxplot() + facet_grid(.~phase) + scale_y_log10()

Information recorded about a package

Because the build/check processes are error prone, some packages may lack information described below.

`raw_info`

This is a selection from content of an info.dcf produced for each package.

setwd(td) # need to be there
str(bbsBuildArtifacts:::make_raw_info(af, "zinbwave"))

Detailed information about events

Here's how we can obtain the installation log for SummarizedExperiment on nebbiolo2.

setwd(td)
pd1 <- make_BBS_package_data(af, "zinbwave")
pd1
names(slot(pd1, "host_data"))
hd = slot(pd1, "host_data")
cat(slot(hd$nebbiolo1, "install"), sep="\n")