pixsets.md

In JingningShi/GifRepo: A Shiny Application for Automatic Measurements of Tree-Ring Widths on Digital Images

title: "Pixsets: representing pixel sets in imager" author: "Simon Barthelmé" date: "2019-03-02" output: html_document: toc: true number_sections: true vignette: > %\VignetteIndexEntry{pixsets} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8}

If you've already gone through the "Getting started" vignette, you know about image objects ("cimg" class). This vignette introduces pixsets, which is the other important kind of objects. Pixsets represent sets of pixels, or equivalently binary images, AKA "masks".

From images to pixsets and back

A pixset is what you get if you run a test on an image:

im <- load.example('parrots') %>% grayscale

Error in load.example("parrots") %>% grayscale: could not find function "%>%"

px <- im > .6 #Select pixels with high luminance

Error in eval(expr, envir, enclos): object 'im' not found

px

Error in eval(expr, envir, enclos): object 'px' not found

plot(px)

Error in plot(px): object 'px' not found

Internally, a pixel set is just an array of logical (boolean) values:

str(px)

Error in str(px): object 'px' not found

The "TRUE" values correspond to pixels in the set, and "FALSE" to pixels not in the set. The dimensions of the pixset are the same as that of the original image:

all(dim(px) == dim(im))

Error in eval(expr, envir, enclos): object 'px' not found

To count the number of pixels in the set, use sum:

sum(px) #Number of pixels in set

Error in eval(expr, envir, enclos): object 'px' not found

mean(px) #Proportion

Error in mean(px): object 'px' not found

Converting a pixset to an image results in an image of the same size with zeroes and ones:

as.cimg(px)

Error in as.cimg(px): could not find function "as.cimg"

##same thing: automatic conversion to a numeric type
px + 0

Error in eval(expr, envir, enclos): object 'px' not found

Indexing using pixsets

You can use pixsets the same way you'd normally use an array of logicals, e.g. for indexing:

mean(im[px])

Error in mean(im[px]): object 'im' not found

mean(im[!px])

Error in mean(im[!px]): object 'im' not found

which(px) %>% head

Error in which(px) %>% head: could not find function "%>%"

Plotting and visualising pixsets

The "highlight" function is a good way of visualising pixel sets:

plot(im)

Error in plot(im): object 'im' not found

px <- (isoblur(im,4)  > .5 )

Error in isoblur(im, 4): could not find function "isoblur"

highlight(px)

Error in highlight(px): could not find function "highlight"

highlight extracts the contours of the pixset (see ?contours) and plots them.

colorise is also useful:

colorise(im,px,"red",alpha=.5) %>% plot

Error in colorise(im, px, "red", alpha = 0.5) %>% plot: could not find function "%>%"

You can also use plain old "plot":

plot(px)

Error in plot(px): object 'px' not found

It converts "px" to an image and uses plot.cimg.

Coordinates for pixels in pixsets

The where function returns coordinates for pixels in the set:

where(px) %>% head

Error in where(px) %>% head: could not find function "%>%"

where returns a data.frame. That format is especially convenient if you want to compute some statistics on the coordinates, e.g., the center of mass of a region defined by a pixset:

where(px) %>% dplyr::summarise(mx=mean(x),my=mean(y))

Error in where(px) %>% dplyr::summarise(mx = mean(x), my = mean(y)): could not find function "%>%"

Selecting contiguous regions, splitting into contiguous regions

In segmentation problems one usually wants contiguous regions: px.flood uses the flood fill algorithm (AKA the bucket tool in image editors) to select pixels based on similarity.

plot(im)

Error in plot(im): object 'im' not found

#Start the fill at location (180,274). sigma sets the tolerance
px.flood(im,180,274,sigma=.21) %>% highlight

Error in px.flood(im, 180, 274, sigma = 0.21) %>% highlight: could not find function "%>%"

It's also common to want to split a pixset into contiguous regions: use split_connected.

sp <- split_connected(px) #returns an imlist 

Error in split_connected(px): could not find function "split_connected"

plot(sp[1:4])

Error in plot(sp[1:4]): object 'sp' not found

sp

Error in eval(expr, envir, enclos): object 'sp' not found

Each element in the list is a connected pixset. You can use split_connected to check connectedness (there are faster ways, of course, but this is simple):

is.connected <- function(px) length(split_connected(px)) == 1
sapply(sp,is.connected)

Error in lapply(X = X, FUN = FUN, ...): object 'sp' not found

is.connected(px)

Error in split_connected(px): could not find function "split_connected"

Use the "high_connectivity" argument to extend to diagonal neighbours as well. See ?label for more.

Boundaries

The boundary function computes the boundaries of the set:

boundary(px) %>% plot

Error in boundary(px) %>% plot: could not find function "%>%"

##Make your own highlight function:
plot(im)

Error in plot(im): object 'im' not found

boundary(px) %>% where %$% { points(x,y,cex=.1,col="red") }

Error in boundary(px) %>% where %$% {: could not find function "%$%"

Growing, shrinking, morphological operations

The grow and shrink operators let you grow and shrink pixsets using morphological operators (dilation and erosion, resp.). Have a look at the article on (morphology)[https://dahtah.github.io/imager/morphology.html] for more:

plot(im)

Error in plot(im): object 'im' not found

highlight(px)

Error in highlight(px): could not find function "highlight"

#Grow by 5 pixels
grow(px,5) %>% highlight(col="green")

Error in grow(px, 5) %>% highlight(col = "green"): could not find function "%>%"

#Shrink by 5 pixels
shrink(px,5) %>% highlight(col="blue")

Error in shrink(px, 5) %>% highlight(col = "blue"): could not find function "%>%"

#Compute bounding box
bbox(px) %>% highlight(col="yellow")

Error in bbox(px) %>% highlight(col = "yellow"): could not find function "%>%"

Common pixsets

There's a few convenience functions defining convenience pixsets:

px.none(im) #No pixels

Error in px.none(im): could not find function "px.none"

px.all(im) #All of them

Error in px.all(im): could not find function "px.all"

plot(im)

Error in plot(im): object 'im' not found

#Image borders at depth 10
px.borders(im,10) %>% highlight

Error in px.borders(im, 10) %>% highlight: could not find function "%>%"

#Left-hand border (5 pixels), see also px.top, px.bottom, etc.
px.left(im,5) %>% highlight(col="green")

Error in px.left(im, 5) %>% highlight(col = "green"): could not find function "%>%"

Splitting and concatenating pixsets

imsplit and imappend both work on pixsets.

#Split pixset in two along x
imsplit(px,"x",2) %>% plot(layout="row")

Error in imsplit(px, "x", 2) %>% plot(layout = "row"): could not find function "%>%"

#Splitting pixsets results into a list of pixsets
imsplit(px,"x",2) %>% str

Error in imsplit(px, "x", 2) %>% str: could not find function "%>%"

#Cut along y, append along x
imsplit(px,"y",2) %>% imappend("x") %>% plot()

Error in imsplit(px, "y", 2) %>% imappend("x") %>% plot(): could not find function "%>%"

You can use reductions the same way you'd use them on images, which is especially useful when working with colour images.

Working with colour images

Be careful: each colour channel is treated as having its own set of pixels, so that a colour pixset has the same dimension as the image it originated from, e.g.:

px <- boats > .8

Error in eval(expr, envir, enclos): object 'boats' not found

px

Error in eval(expr, envir, enclos): object 'px' not found

where(px) %>% head

Error in where(px) %>% head: could not find function "%>%"

Here "px" tells us the location in all locations and across channels of pixels with values higher than .8. If you plot it you'll see the following:

plot(px)

Error in plot(px): object 'px' not found

Red dots correspond to pixels in the red channel, green in the green channel, etc. You can also view the set by splitting:

imsplit(px,"c") %>% plot

Error in imsplit(px, "c") %>% plot: could not find function "%>%"

If you need to find the pixel locations that have a value of .9 in all channels, use a reduction:

#parall stands for "parallel-all", and works similarly to parmax, parmin, etc.
imsplit(px,"c") %>% parall %>% where %>% head

Error in imsplit(px, "c") %>% parall %>% where %>% head: could not find function "%>%"

#at each location, test if any channel is in px
imsplit(px,"c") %>% parany %>% where %>% head

Error in imsplit(px, "c") %>% parany %>% where %>% head: could not find function "%>%"

#highlight the set (unsurprisingly, it's mostly white pixels)
plot(boats)

Error in plot(boats): object 'boats' not found

imsplit(px,"c") %>% parany %>% highlight

Error in imsplit(px, "c") %>% parany %>% highlight: could not find function "%>%"

An example: segmentation with pixsets

The following example is derived from the documentation for scikit-image. The objective is to segment the coins from the background.

im <- load.example("coins")

Error in load.example("coins"): could not find function "load.example"

plot(im)

Error in plot(im): object 'im' not found

A simple thresholding doesn't work because the illumination varies:

threshold(im) %>% plot

Error in threshold(im) %>% plot: could not find function "%>%"

It's possible to correct the illumination using a linear model:

library(dplyr)
d <- as.data.frame(im)

Error in as.data.frame(im): object 'im' not found

##Subsamble, fit a linear model
m <- sample_n(d,1e4) %>% lm(value ~ x*y,data=.) 

Error in sample_n(d, 10000): object 'd' not found

##Correct by removing the trend
im.c <- im-predict(m,d)

Error in eval(expr, envir, enclos): object 'im' not found

out <- threshold(im.c)

Error in threshold(im.c): could not find function "threshold"

plot(out)

Error in plot(out): object 'out' not found

Although that's much better we need to clean this up a bit:

out <- clean(out,3) %>% imager::fill(7)

Error in clean(out, 3): could not find function "clean"

plot(im)

Error in plot(im): object 'im' not found

highlight(out)

Error in highlight(out): could not find function "highlight"

Another approach is to use a watershed. We start from seeds regions representing known foreground and known background pixels:

bg <- (!threshold(im.c,"10%"))

Error in threshold(im.c, "10%"): could not find function "threshold"

fg <- (threshold(im.c,"90%"))

Error in threshold(im.c, "90%"): could not find function "threshold"

imlist(fg,bg) %>% plot(layout="row")

Error in imlist(fg, bg): could not find function "imlist"

#Build a seed image where fg pixels have value 2, bg 1, and the rest are 0
seed <- bg+2*fg

Error in eval(expr, envir, enclos): object 'bg' not found

plot(seed)

Error in plot(seed): object 'seed' not found

The watershed transform will propagate background and foreground labels to neighbouring pixels, according to a priority map (the lower the priority, the slower the propagation). Using the priority map it's possible to prevent label propagation across image edges:

edges <- imgradient(im,"xy") %>% enorm

Error in imgradient(im, "xy"): could not find function "imgradient"

p <- 1/(1+edges)

Error in eval(expr, envir, enclos): object 'edges' not found

plot(p)

Error in plot(p): object 'p' not found

We run the watershed transform:

ws <- (watershed(seed,p)==1)

Error in watershed(seed, p): could not find function "watershed"

plot(ws)

Error in plot(ws): object 'ws' not found

We still need to fill in some holes and remove a spurious area. To fill in holes, we use a bucket fill on the background starting from the top-left corner:

ws <- bucketfill(ws,1,1,color=2) %>% {!( . == 2) }

Error in bucketfill(ws, 1, 1, color = 2): could not find function "bucketfill"

plot(ws)

Error in plot(ws): object 'ws' not found

To remove the spurious area one possibility is to use "clean":

clean(ws,5) %>% plot

Error in clean(ws, 5): could not find function "clean"

Another is to split the pixset into connected components, and remove ones with small areas:

split_connected(ws) %>% purrr::discard(~ sum(.) < 100) %>%
    parany %>% plot

Error in split_connected(ws): could not find function "split_connected"

Here's a comparison of the segmentations obtained using the two methods:

layout(t(1:2))
plot(im,main="Thresholding")

Error in plot(im, main = "Thresholding"): object 'im' not found

highlight(out)

Error in highlight(out): could not find function "highlight"

plot(im,main="Watershed")

Error in plot(im, main = "Watershed"): object 'im' not found

out2 <- clean(ws,5)

Error in clean(ws, 5): could not find function "clean"

highlight(out2,col="green")

Error in highlight(out2, col = "green"): could not find function "highlight"

JingningShi/GifRepo documentation built on May 14, 2019, 10:59 p.m.