NEWS.md

In Paradigm4/SciDBR: An R Interface to SciDB

Version 4.0.4

• Date: 2023-11-07
• Speed up large fetches in Arrow format over SciDB HTTP API

Version 4.0.3

• Date: 2023-09-28
• Changes to CI configuration

Version 4.0.2

• Date: 2023-07-24
• Fix spurious warnings during garbage collection

Version 4.0.1

• Date: 2023-07-19
• Dependency fixes: R6 added to Imports; bit64 moved from Depends to Imports; glue::glue() usage replaced with paste()/paste0()

Version 4.0.0

• Date: 2023-05-05
• SciDBR can now connect directly to SciDB versions 23.2 and higher using the new SciDB HTTP API, without requiring Shim. To use it, connect using scidbconnect() as normal, but specify the HTTP API port number (usually 7000 + the SciDB port number; most often 8239). SciDBR will automatically determine the protocol and API to use, using the SciDB HTTP API for direct connections or the Shim API for shim connections.

Version 3.1.6

• Date: 2023-05-03
• Add tools for converting R filter expressions into corresponding AFL (https://github.com/Paradigm4/SciDBR/pull/284)

Version 3.1.5

• Date: 2023-04-10
• Fix bug -- passing curl options as a list was not working in R>=4.2 (fix #279)

Version 3.1.4

• Date: 2023-02-06
• Fix bug -- when using sort but no AIO, SciDBR may produce an error when using sort() (fix #263)

Version 3.1.3

• Date: 2023-01-04
• Consolidate curl handle creation code and add ability to set this via package config (fix #253)

Version 3.1.2

• Date: 2022-09-07
• Changes how as.scidb() and store() handle 'gc' when not specified (https://github.com/Paradigm4/SciDBR/pull/252)
• Allows 'gc' to clean up named arrays, and not just 'Rarray_...' arrays

Version 3.1.1

• Date: 2022-07-06
• Avoid protocol issue with libcurl and HTTP/2 (https://github.com/Paradigm4/SciDBR/pull/251)

Version 3.1.0

• Date: 2022-03-18
• Initial support for Arrow format (https://github.com/Paradigm4/SciDBR/pull/243)

Version 3.0.10

• Date: 2021-11-17
• Fix for datetimes returned in situations where the client and server are not in the same time zone

Version 3.0.9

• Date: 2021-10-18
• Fix for #236 (scidbr objects are not printing to console)

Version 3.0.8

• Date: 2021-09-29
• Fixed issue of munged queries (https://github.com/Paradigm4/SciDBR/issues/234) arising from potential sprintf bug

Version 3.0.7

• Date: 2021-09-06
• In as.scidb, added a new optional parameter - max_byte_size (default value: 500MB) for uploading R vectors in a multi-part fashion into SciDB. Example:

scidb::as.scidb(db, x = c(1:10), gc = TRUE)
scidb::as.scidb(db, x = c(1:10)/10, max_byte_size = 8, gc = TRUE)

Version 3.0.6

• Date: 2021-08-13
• In as.R, SciDB data frames are downloaded without the artificial dimensions

Version 3.0.5

• Date: 2021-07-20
• Fix for uploading long vectors via as.scidb() https://github.com/Paradigm4/SciDBR/issues/220

Version 3.0.4

• Date: 2021-05-26
• Fix for uploading character vectors containing NA values

Version 3.0.3

• Date: 2021-05-13
• Support for uploading vectors, dense and sparse matrices as temp arrays via as.scidb
• Partial fix for CRAN reviewer comments (https://github.com/Paradigm4/SciDBR/issues/213)

Version 2.0.0

• Big change: all SciDB arrays are presented as data frames
• Greatly simplified the package, improved data transfer performance
• Support for SciDB 16.9 and older versions at least back to 15.7

Version 1.1-3

• Support for upcoming SciDB 14.12, untested backwards support for older releases

• Improved type parsing support, new binary transfer option for data frame objecs. This is much faster than UTF-8 transfer from SciDB.

• Support for streaming data from SciDB.

• Option to disable interrupt handling for faster large data transfers.

• Added functions include order, more flexible dist, peek.

Version 1.1-2

SCIDB SUPPORT

• Support for SciDB 14.3, untested support for older SciDB releases

• Dropping arrays from other R sessions requires a non-default option facilitating multiple-user settings.

• Queries can be canceled with R user interrupts now (for example with CTRL + C or ESC).

IMPROVEMENTS

• Many improvements to merge and aggregate functions

• N-d arrays with multiple attributes can now index with $

• Improved R save/load of R data files containing SciDB objects

NEW FEATURES

• Support for Paradigm4 glm and a simple glm model matrix builder

• Support for Paradigm4 truncated SVD routine

• Added hist, quantile, all.equal, antijoin, a c (SciDB concat-like) function and many others

Version 1.1-1

WINDOWED AGGREGATES

• The aggregate function now supports moving and fixed window aggregates.

TLS/SSL ENCRYPTION AND AUTHENTICATION

• The package now supports TLS/SSL encrypted communication with the shim network service and authentication. The only function affected by this is scidbconnect -- simply supply an SSL port number and username and password arguments.

LAZY EVALUATION

• Most scidb and scidbdf object can now represent array promises, un-evaluated SciDB queries equipped with a result schema and a context environment.

NEW COMPOSABLE OPERATORS

• Many new functions were introduced that closely follow underlying SciDB AFL operators. All of the new functions are composable with lazy evaluation of the underlying query using the new scidbexpr class. Results are only computed and stored by SciDB when required or explicitly requested.

R SPARSE MATRIX SUPPORT

• The package now maps sparse SciDB arrays to sparse R matrices. Only matrices (2-d arrays) with double-precision attributes are supported. Sparse array arithmetic uses the new P4 sparse matrix multiply operator when available.

USING RCurl NOW

• We introduced a dependency on RCurl in order to support SSL and authentication with the SciDB shim service.

NO MORE SciDB NID

• The package no longer tries to support SciDB arrays with NID dimensions, which never really worked anyway. Instead, many functions now take advantage of the new SciDB uniq and index_lookup operators if available (>=SciDB 13.6). Future package versions will take this further and introduce array dimension labeling using the new operators.

MANY NEW DATABASE-LIKE FEATURES

• Aggregate was improved, merge sort, unique, index_lookup, and other new functions were added. See the vignette for more information.

Version 1.1-0

SIGNIFICANT BUG FIX:

• Materializing subsetting operations could return inconsistently ordered data when results spanned SciDB array chunks across multiple SciDB instances. Data are now returned correctly in such cases.

OTHER BUG FIXES:

• Fixed a bug in the processing of the start argument in the as.scidb function.

• Fixed several bugs in the image function.

NEW FEATURES:

• The iquery function now accepts n=Inf to efficiently download all output from query at once. The iterative=TRUE option should still be used with smaller n values to iterate over large results.

• The crossprod and tcrossprod functions are now available for SciDB arrays and mixed SciDB/R objects.

• A diag function is now available for SciDB matrices, returning result as a new SciDB 1-D array (vector).

• Element-wise exponentiation was implemented for scidb array objects.

• Implemented a sum function for scidb array objects.

Paradigm4/SciDBR documentation built on Nov. 9, 2023, 4:58 a.m.