NEWS.md

NEWS

0.2.1

0.2.0.xxx

Work on documentation

Work on ExtractImages_to/ExtractMasks_to functions (breaking)

Work on image extraction at cpp level

Work on Gating Strategy

Work on graphs

Miscellaneous

0.2.0

0.1.9.xxx

0.1.9

0.1.8.xxx

work on plot:

work on FCS:

reading

add internal functions readFCSdataset and split it into smaller dedicated ones readFCSheader, readFCSdelimiter, readFCStext, readFCSdata

this could have lead to undesired error "mismatch between found vs expected number of parameters"

file parsing was performed assuming required keywords were upper case. now this assumption is removed and parsing is performed whatever the case.

formerly, other values could lead to undesired dataset removal in returned value. readFCSdataset is now used within readFCS to avoid that.

changes for inputs:

before if user wanted to modify options, he had to provide options with the exact options names. now part of options can be provided and missing part are reconstructed, in addition extra names does not trigger errors anymore.

changes for outputs

writing

this could have lead to undesired removal of keywords during FCS export

this means that to follow FCS3.0 specifications keywords should be 0x20-0x7E only.

before they were replaced by underscore "_".

when any of them is >= 1e8, then BOTH should be 0 and $BEGINDATA and $ENDDATA in TEXT segment should report offsets. Before only the one(s) >= 1e8 was/were set to 0 in HEADER (though, TEXT segment was correct)

0.1.8

0.1.7.xxx

work on TIFF (mainly dedicated to handle large files)

fix bug with offsets being bounded or converted to NA_integer. Offsets were stored as uint32_t in hpp/cpp and placed in Rcpp::IntegerVector and then converted with as.integer in R which makes them undesirably bounded to [-(2^31-1) : -2^31-1], i.e. max 2GB files):

improvement in offset extraction:

speed improvement during file export (ExportToXIF, subsetXIF, mergeXIF, XIFtoTIFF). Though it still remains long.

work on progress bar

improve logic (better mapping between value and steps).

using value < min in IFC:::setPB now allows for circular progress (i.e. pb restart at 0 and cycle once again and so on, until function stop). This way user can see that something is currently happening and not freezing.

automatically detect shiny for progress bar.

drop dependency on RcppProgress

work on seed (make IFC as transparent as possible to RNG)

this is achieved thanks to new functions R/seed.R

1/ fetch_seed (ensures backward compatibility while allowing new behaviour).

2/ with_seed (encapsulates expr to allow its evaluation with seed while permitting RNG state to be reset once done)

For mask/features/regions/pops definition spliting or other internal stuffs alternative names needs to be generated. These alt names are created using random generation by gen_altnames that don't need to be statistically relevant. Consequently, they should not impact current R seed. IFC package has now been restructured to allow this. Internally, when gen_altnames is used, we use of "Mersenne-Twister", "Inversion", "Rounding" by default as it was the one used in old R version (at least since RNGVersion('1.7.0')).

To allow random and reproducible sampling based on a string, a new internal pseudo_seed() has been created to compute a seed integer from string. It is used with:

fix potential bug with writeGatingStrategy where random ids are generated to enable boolean population decomposition and export to GatingML (ids may have been duplicated ?).

encapsulate requireNamespace("shiny", quietly = TRUE) within with_seed (calling requireNamespace("shiny", quietly = TRUE) alters R seed).

make use of rng = false attribute for cpp functions export when possible.

work on stats

fix stats computation with infinite value.

fix stats in 2D (subset by dimension and not using both dimensions).

simplify internal StatsReport.

handle several edge cases:

allow to display progress bar.

return more statistics in extractStats.

work on graphs

add backend argument to plotGraph, ExportToReport, DisplayReport, BatchReport to allow to choose which backend will be used for drawing. Default backend = "lattice", will behave as usual, while backend = "base" and backend = "raster" now allow user to use base graphics and further backend = "raster" to produce plots with huge amount of points very fast. backend = "raster" can also be used to generate reports of smaller size (which will open and render shortly).

plotGraph returned value plot element does not exist anymore and par.setting is now part of returned input element.

patch buildGraph to handle histogram plot extracted from rif files. In rif, population from graphRegion are used in order which is not compatible with what IFC needs, so order is modified if provided to remove these population names.

fix population order in legend (plot_base and plot_raster, not exposed before backend introduction)

adjustGraph (internal) be less permissive, ensure that features, transformations and types are respected (before only checks for existing pops and regions was performed). In addition, title should be recomputed when BasePop has changed. Finally, use it in applyGatingStrategy to ensure that plot can be produced when strategy is applied.

work on pops

allow propagation of attributes (notably, "reserved" used by IFCshiny).

add checks in popsWithin for recursive definition and for non duplicated names.

fix popsRename so as to handle graphs title.

allow 'base' modification in data_modify_pops + add needed graphs adjustments.

bugfix

in autoplot, when drawing a region requires the creation of a population.

non-finite values when computing combined features should be NaN.

stats computation with infinite feature values and bad subsetting in 2D graphs (subset by dimension and not using both dimensions).

in *popsRename fix edge cases of renaming pops (e.g. swapping pop1 <-> pop2), fix GraphRegion (if any) which were not correctly computed, and fix to handle graphs title.

in buildBatch, exported files names computation when input files are not .rif.

fix bug that prevented file from being released when an error did raise during reading at hpp/cpp level.

allow for reading large files at hpp/cpp level by using std::size_t instead of uint32_t for offset positions (in scan.hpp and decomp.hpp, in addition to tiff.hpp).

potential bug with writeGatingStrategy where random ids are generated to enable boolean population decomposition and export to GatingML (ids may have been duplicated ?).

Misc

data_add_pop_sample (internal), don't set new_name automatically if missing and force user to provide one.

redefine_obj, list only once duplicated names, use new_feat_def for mapping "to" and obj for "initial".

add (internal) data_modify_regions and data_modify_pops functions.

remove hpp/cpp_fast_cbind_functions which are not used anymore.

data_to_DAF now makes use of toBIN_features and toBIN_images internal functions.

order of images files export has changed. Previously, extra content was written before IFD. Now, IFD is written first and then content that does not fit within the 4 bytes slot for value is written just after the offset pointing to next IFD (see work on TIFF).

typo, missing param in toBIN_.* functions and UNDIFINED in getIFD.

This leads to the following visible changes for the user

0.1.7

0.1.6.xxx

improve and document logic behind switch_channel (used to allow changing all images/masks/features definition of one channel to another one)

create swap_channel function (which use switch_channel) to operate channel swap (basically, switch to one channel, switch to other channel, and merge)

Internally, the framework used for parsing definitions (features/pops/masks) has been consolidated to enhance accuracy and security. This is mostly based on splitn (for pops and features) which split definition into chunks according to allowed names and operators and ensure 1-to-1 replacement of them with unique random names created thanks to gen_altnames and random_name

create dedicated gen_altnames function to map allowed names to unique id

adapt splitn to handle features definition, simplify logic, make use of gen_altnames ancillary function, and avoid unnecessary unique id creation thanks to new alt_names argument and add scalar and dsplit arguments

add dedicated split_feat (based on splitn and get_altnames) for features to avoid repeated splitting loop

create featureIFC for static features types/names (used to validate definition and also determine name of a feature)

create gseq/cpp_seqmatch to match string sequence, used by feature_namer

modify feature_namer (used to determine name of a feature)

"combined" features resulted in wrong values when extracted (dedicated internal get_feat_value and getFeaturesValues are now here to solve this)

data_rm_features caused the removal of more features than it should (due to wrong pattern used for features names matching)

fix potential bug with NAs / NaNs values, e.g. when graphically defined on features containing NAs / NaNs, the resulted event should be 'FALSE' and not 'NA' to flag that it does NOT belong to the population

limit the scope of evaluation for features and populations computation

adding/exporting "tagged" population containing NAs / NaNs value is not allowed

warning/error messages are now more informative during population add/compute/export

improve and dry code for stats extraction / computation

group internal buildStats and statsCompute functions into buildStats

add internal functions to extract and generate Statistics Report

fix bug leading to error (pops reported count was NAs) when features contained NAs

fix bug in plotGraph, ExportToReport, and BatchReport in feature summary computation for graphs containing overlay of populations (1D and 2D) for values of the region(s) drawn

catch errors in density computation for histogram and send warning instead but allow graph to be created as far as possible (already catched in plot_lattice but not with plot_base nor plot_raster)

swap labels and symbols when using plot_lattice to gain in homogeneity with plot_base and plot_raster

"histogram" legend were ok for plot_lattice but not for plot_base and plot_raster which were displaying pch instead of lty

'NA' is now an allowed value for adjust_graph argument in data_rm functions to force graph(s) removal

implement and export new data_add_graphs data_rm_graphs functions

adjustGraphs now internally uses graph as intput rather than indice(s) of graph(s) from IFC_data object (which is more versatile and allows for testing any graphs in the context of 'obj')

catch error in adjustGraphs

fix bugs when trying to adjust graph in several places

1 - when testing if drawable which was not working since about https://github.com/gitdemont/IFC/commit/030df2bfb25680e3159aa4a15bf0d550543d4188

2 - when applied GraphRegion(s) result(s) in not found pop(s)

3 - when trying to applyGatingStrategy which did not actually test if graph(s) could be produced

fix bug in plotGraph, ExportToReport, and BatchReport in feature summary computation for graphs containing overlay of populations (1D and 2D) for values of the region(s) drawn

fix the fix https://github.com/gitdemont/IFC/commit/34f33de1f916673855c2bc69c0c1e686bd608c58 which did not allow to correctly pass extra parameters to dots ...

absence of predefined report in the analysis file induced error when building the batch

fix bug when no compensation was provided

replace first_only argument in readFCS by dataset to allow user to choose which dataset to extract rather than only '1st' or 'All' (default behavior remains unchanged, however)

allow import/export of keywords

fix potential bug with extra keywords due to wrong regex identification

gray -> rgb color conversion improvement

use Rcpp::no_init when possible

modify internal fastCbind (the underlying hpp/cpp functions induced more overhead than R cbind)

for progressbar, check if session missing and class rather than if length is 0

remove gc from fcs functions

allow to pass NULL and NA in texttomatrix / addText (before an error was thrown)

implement internal function data_to_AST to allow the creation of .ast file from IFC_data object

This leads to the following visible changes for the user

See the bugs fix above

first_only argument is replaced by dataset for FCS reading

IFC_data object returned by ExtracFromFCS has gained an new Keywords member

new data_add_graphs and data_rm_graphs functions are now exported

0.1.6

0.1.5.xxx

fix ExportToDAF that exports xml files containing NULL values

encode files names to native before passing them to underlying hpp/cpp files for reading

fix error in objectExtract when info argument is provided generated duplicated arguments

fix cpp_check_range that could lead to error when input can be summarize to a single value (this could happen when image object has no mask)

fix cpp_resize that did not take into account resizing information for masks

add add_tracking argument to mergeXIF, subsetXIF, XIFtoTIFF, and ExportToXIF functions to control whether exported file should contain tracking information

improve tracking of object_ID

improve XIF I/O speed (avoid conversion when building IFD and use raw instead thanks to new internal collapse_raw function)

improve numeric to string conversion when using objects ids as names/dimnames (e.g. avoid 1e+05, use 100000 instead)

create popsRename function to simplify population renaming in IFC_data object

fix bug with popsCompute for type "C" (combined) population computation when population definition contains repeated occurrence of same population name

This leads to the following visible changes for the user

mergeXIF, subsetXIF and ExportToXIF functions gain a new add_tracking argument

a new popsRename function is now exported

0.1.5

fix trailing slash NOTE in last CRAN submission

fix cpp_normalize which did not take into account force_range parameter

0.1.4

0.1.2.101 to 0.1.3.104

preparing for CRAN release

use doi instead of url

modify some links that have moved

improve drawing speed due to internal do.call that causes subtitute(x) to cast all x values to character

speed gain thanks to dissociation between IFC_plot object generation from plot and stats production

create internal plot_stats() dedicated function for stats extraction

create internal plot_lattice() dedicated function for plotting (also rename convert_to_baseplot to plot_base)

create internal plot_raster() and rasterplot() + cpp/hpp underlying functions to allow plot rasterization for very fast 2D graph drawing

better legend positioning

additional use of " " to prevent heightDetails potential error

fix bug with plotGraph which was opening a dev to retrieve lattice par settings

fix bug with grouping and precision = "light" when number of displayed population was <= 1

transformations (axes + density) should now be provided as character and are matched against allowed names

add text_only parameter

speed gain thanks creation of internal cpp/hpp functions after code profiling

better handle files with extra keywords segment

fix fcs import for multiple files merging when no spillover can be retrieved + when they have exactly same features names

fix large fcs file export

add support to bad-formatted fcs files (with empty data offsets in header)

warn user on missing additional datasets (i.e. non empty $NEXTDATA)

fix progress bar bug when extracting images/masks offsets

fix bug with population stats computation when IFC_data object contains tagged populations whose obj are not logical (i.e. numeric or integer)

fix bug when applying gating strategy with no graphs

better control/coercion on files read/write notably for graphs

internally improve file scan by using raw vector target rather than string

fix internal whoami() which was erroring on namespaced calls (i.e. with :: or :::)

improve ExportToReport selection argument to allow graphs to be passed as a matrix that defines report layout

export new BatchReport function

This leads to the following visible changes for the user

transformations should now be provided as character and will be matched against allowed names

a new BatchReport function is exported

readFCS gains a new 'text_only' entry in 'options' argument to allow user to only extract TXT segment

0.1.2.100

speed gain and better accuracy notably for files stored with DATATYPE "I"

can handle files with non R standard bits length (i.e. not 1,2,4,8)

fix bug with large PnR e.g. "4294967296" which resulted in NA

This leads to the following visible changes for the user

buildGraph gains a new 'densitylevel' entry in BasePop controlling levelplot when 'type' is "density"

when fcs is imported @IFC_date, @IFC_file, @IFC_fileName, @IFC_dataset, @IFC_version, and @IFC_FCSversion are automatically filled. $FIL is not filled if found. $CYT is not filled if found except if user requires it.

readFCS gains a new 'force_header' entry in 'options' argument to allow user to control if data offset(s) should be taken from header only

0.1.2

This leads to the following visible changes for the user

data_to_DAF now allows to create daf file with statistics table

data_to_DAF now allows to create daf file from an IFC_data object extracted from a rif file

ExtractFromXIF does not extract channels that were not acquired anymore

getInfo returned object gains an additional evmode value

buildGraph gains a new maxpoints parameter

data_rm_ functions gain a new adjust_graph parameter

subsetOffsets and getOffsets returned object gain an additional test attribute

getInfo now returns XIF_test value in addition to former elements

objectParam gains a new spatial_correction parameter which when TRUE adds a 2 new columns, namely spatial_X and spatial_Y, to the returned data.frame in 'channels'

paletteIFC now maps Control to Gray81

axes display have been slightly changed to show labels at pretty positions when LinLog transformation is used

0.1.1

0.1.0

0.0.9



gitdemont/IFC documentation built on Feb. 20, 2025, 12:55 a.m.