Internals

In Rtinycc: Builds the 'TinyCC' Command-Line Interface and Library for 'C' Scripting in 'R'

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
tcc_bind <- Rtinycc::tcc_bind
tcc_callback_close <- Rtinycc::tcc_callback_close
tcc_compile <- Rtinycc::tcc_compile
tcc_cstring <- Rtinycc::tcc_cstring
tcc_data_ptr <- Rtinycc::tcc_data_ptr
tcc_ffi <- Rtinycc::tcc_ffi
tcc_get_symbol <- Rtinycc::tcc_get_symbol
tcc_link <- Rtinycc::tcc_link
tcc_malloc <- Rtinycc::tcc_malloc
tcc_relocate <- Rtinycc::tcc_relocate
tcc_set_options <- Rtinycc::tcc_set_options
tcc_source <- Rtinycc::tcc_source

This article is about how Rtinycc works internally. It is not the stable user-facing API contract. The pieces described here are current implementation choices and may change as the package evolves.

At a high level, the package is built as a pipeline:

1. an R-side recipe is accumulated in a tcc_ffi object
2. that recipe is turned into a generated C translation unit
3. TinyCC compiles and relocates the generated code in memory
4. wrapper pointers are recovered and exposed back to R as closures

The tcc_ffi Object Is a Recipe

tcc_ffi() does not compile anything by itself. It creates a plain R object that accumulates:

• bound symbols
• user headers and user C code
• library and include paths
• extra compiler options
• helper declarations such as structs, unions, enums, globals, and callback use

That state lives in the tcc_ffi list object built by tcc_ffi_object(). The important point is that tcc_compile() works from this declarative recipe, not from an already-live TCC process.

ffi <- tcc_ffi() |>
  tcc_source("int add(int a, int b) { return a + b; }") |>
  tcc_bind(add = list(args = list("i32", "i32"), returns = "i32"))

names(ffi)

Code Generation Is Central

tcc_compile() calls the internal generate_ffi_code() helper to assemble one large C source string. That generated source is the real boundary layer between R and the target C functions.

Internally, the generated translation unit is assembled in this order:

• a TinyCC workaround for _Complex
• R.h and Rinternals.h
• callback trampoline declarations when needed
• user headers
• external declarations for tcc_link()
• user C code
• generated helpers for structs, unions, enums, globals, and raw access
• generated SEXP wrappers for each bound symbol

For a small binding:

code <- Rtinycc:::generate_ffi_code(
  symbols = ffi$symbols,
  headers = ffi$headers,
  c_code = ffi$c_code,
  is_external = FALSE,
  structs = ffi$structs,
  unions = ffi$unions,
  enums = ffi$enums,
  globals = ffi$globals,
  container_of = ffi$container_of,
  field_addr = ffi$field_addr,
  struct_raw_access = ffi$struct_raw_access,
  introspect = ffi$introspect
)

grepl("SEXP R_wrap_add", code, fixed = TRUE)

The wrapper is where input coercion, range checks, callback trampoline setup, actual C invocation, and return boxing happen.

How Values Move Between R, The Wrapper, And C

The important internal boundary is not "R calls user C directly". The flow is:

1. an R closure created by make_callable() calls .Call with the compiled wrapper's native symbol external pointer
2. that wrapper receives SEXP arguments
3. wrapper code uses the R C API to decode or borrow data from those SEXPs
4. the wrapper calls the target C symbol using ordinary C arguments
5. the wrapper converts the C result back into a SEXP
6. .Call returns that SEXP to the R interpreter

So the generated wrapper is the translator between:

• R evaluation and SEXP objects on one side
• the target function's plain C signature on the other side

This is why Rtinycc includes R.h and Rinternals.h in every generated translation unit and why the wrapper code uses constructors and accessors such as:

• asInteger() / asReal()
• RAW(), INTEGER(), REAL(), LOGICAL()
• STRING_ELT() and Rf_translateCharUTF8()
• ScalarInteger(), ScalarReal(), ScalarLogical()
• mkString() and R_MakeExternalPtr()

At the R level, make_callable() builds a small closure around the compiled wrapper pointer. That closure does argument-count validation, checks that the pointer is still valid, and then hands control to .Call.

The wrapper itself is where the actual C API interaction happens.

Copying Versus Borrowing Happens In The Wrapper

The copy model is mostly determined by the generated conversion code.

Scalar inputs are copied or coerced into local C values:

• integers and booleans go through asInteger()
• doubles go through asReal()
• range checks happen before the target function is called

These are not zero-copy paths.

Vector inputs are split into two groups:

• raw, integer_array, numeric_array, and logical_array borrow the underlying R vector storage directly
• cstring_array allocates a temporary pointer array with R_alloc() and fills it from translated R strings

String and pointer inputs need more care:

• cstring uses STRING_ELT() plus Rf_translateCharUTF8() for the duration of the call
• ptr reads the raw address from an external pointer with R_ExternalPtrAddr()
• sexp passes the original SEXP through unchanged

Returns have their own copy model:

• scalar returns are boxed into fresh R objects
• cstring returns are copied into R-managed string memory with mkString()
• ptr returns stay as external pointers to raw addresses
• array returns allocate a fresh R vector and memcpy() the C buffer into it

So the internal design is intentionally mixed:

• borrow when R already has contiguous vector storage that matches the C view
• copy when returning data into R-managed memory
• keep raw pointers raw when the package cannot safely invent ownership

That is the main semantic reason the generated wrapper layer exists.

Why lambda.r Is Used

The large rule file R/aaa_ffi_codegen_rules.R uses lambda.r as a small dispatch DSL. The package imports %as% and UseFunction, and defines rules like:

• ffi_input_rule(...)
• ffi_return_rule(...)
• array_return_alloc_line_rule(...)
• c_default_return_rule(...)
• ffi_c_type_map_rule(...)

Those rules are not user-facing metaprogramming. They are an internal way to register many small code-generation cases without turning R/ffi_codegen.R into one enormous nest of if and switch statements.

In practice, generate_c_input() and generate_c_return() delegate into that rule table:

Rtinycc:::generate_c_input("x", "arg1_", "i32")
Rtinycc:::generate_c_return("res", "f64")

The main tradeoff is simple:

• lambda.r keeps the dispatch table explicit and composable
• the rule file becomes long because many integer, floating-point, and helper cases are still written out individually

So lambda.r here is being used for internal rule dispatch and code-template selection, not because the public API depends on functional programming style.

Wrapper Builders Work at the SEXP Boundary

Rtinycc is not using a libffi ABI layer. The generated wrappers are normal C functions with SEXP signatures so that R can call them through .Call.

The key internal steps are:

• generate_wrappers() decides which wrapper variants are needed
• generate_c_wrapper() builds the normal synchronous wrapper body
• generate_async_exec_wrapper() builds the async execution path for callback_async: arguments
• generate_callback_trampolines() emits trampoline functions for callback arguments

For non-variadic bindings, the generated wrapper is named R_wrap_<symbol>. Variadic bindings generate several wrapper variants and dispatch is chosen later from R based on tail arity or inferred tail types.

This design keeps platform-specific calling conventions inside compiled C rather than trying to reproduce them from R.

Protection And Lifetime Rules Matter

Because wrappers use the R C API directly, protection and object lifetime are part of the internal design.

When wrapper code allocates a fresh R object, it protects that object until the result is fully built and returned. Typical cases include:

• array returns that allocate out
• cstring returns that construct an R string
• callback trampolines that build an argument list before calling back into R

Borrowed pointers have a different constraint: they are only sound as long as the underlying owner stays alive and the wrapper does not invalidate the assumption by introducing unexpected allocation patterns.

This is especially important for:

• zero-copy vector inputs
• borrowed field-address helpers for structs and unions
• callback token pointers that must remain tied to a live callback registry

The package also uses external pointer metadata and protected slots to encode lifetime relationships. For example, borrowed field pointers can keep their owner object alive by storing that owner in the external pointer's protected field.

Ownership And Lifetime Semantics In The Main Cases

The main internal cases are easier to reason about if you separate them by who owns the underlying storage and how long the view is valid.

Call-scoped borrows from R objects

These values are borrowed from existing R objects and are only intended to be used during the wrapper call:

• raw, integer_array, numeric_array, and logical_array inputs borrow the backing R vector storage
• cstring input borrows the translated string pointer for the duration of the call
• sexp input borrows the original R object directly

The wrapper does not transfer ownership of these objects to C. If target C code stores the pointer and uses it after the call returns, that is outside the safe contract.

Owned native allocations

These are heap allocations owned through explicit external-pointer semantics:

• tcc_malloc() returns rtinycc_owned memory with a finalizer
• tcc_cstring() returns a malloc-backed UTF-8 C string with the same owned tag
• generated struct and union constructors allocate native storage and attach type-specific finalizers

These objects have a stable native lifetime until:

• they are explicitly freed
• their owner-specific free helper is called
• or their finalizer runs during normal R lifetime

Borrowed native views

These are external pointers that point into someone else's storage:

• tcc_data_ptr() returns a borrowed pointer
• field-address helpers for structs and unions return borrowed pointers
• many plain ptr returns are just raw addresses wrapped as external pointers

Borrowed pointers do not imply ownership and must not be freed as if they were rtinycc_owned. Their validity depends entirely on the lifetime of the underlying storage.

Returned R objects

When the wrapper returns a scalar, string, or copied array to R, the result is an ordinary R-managed object:

• scalar returns are fresh boxed R values
• cstring returns become fresh R strings
• array returns become fresh R vectors after copying

Once returned, these objects follow the normal R GC lifetime and are no longer tied to the lifetime of the original C storage.

Callback registry lifetime

Callbacks have a separate ownership model:

• the callback registry preserves the underlying R function
• callback tokens are external pointers referencing registry entries
• tcc_callback_close() releases the preserved function deterministically
• if not closed manually, finalizers and package unload eventually release it

This means the callback object is not just a function pointer. It is a managed pairing of:

• preserved R function state
• callback metadata
• one or more external-pointer handles to the token

Compiled object lifetime

A tcc_compiled object owns a live TCC state and the wrapper pointers recovered from that state.

When that state dies, the wrapper pointers are dead as machine-code references even though the R closures still exist. That is why the package stores a recipe and recompiles instead of pretending those pointers survive serialization.

Host Symbol Injection Happens Before Relocation

After the generated code is compiled, tcc_ffi_compile_state() calls the C entry point RC_libtcc_add_host_symbols() before tcc_relocate().

That host-injection step registers package-side C helpers with the live TCC state. This matters most on macOS, where the package cannot rely on the dynamic linker to expose every host symbol the same way TinyCC expects.

The injected symbols include:

• RC_free_finalizer
• callback invocation helpers
• async callback scheduling helpers
• async drain helpers
• the RC_callback_async_exec_c() helper used by generated async wrappers

The important semantic point is that some generated C code depends on package runtime helpers, not just on user code and the R API.

Callback Round-Trips Cross The Boundary Twice

Callbacks are the clearest example of value exchange between plain C and the R interpreter.

For synchronous callbacks:

1. generated C trampoline code receives plain C arguments
2. the trampoline boxes them into a VECSXP argument list
3. it calls RC_invoke_callback_id()
4. the runtime builds and evaluates the R call with R_tryEvalSilent()
5. the result is converted back into the declared C return type
6. the trampoline returns that C value to the original compiled code

So a callback call is:

• C values -> boxed into R objects
• evaluated in R
• converted back from R objects -> C values

Async callbacks add one more layer: arguments are first marshaled into a cross-thread task representation, then rebuilt as fresh R objects on the main thread before the callback is evaluated.

State Creation Is Separate from Compilation

The TCC state is created first, then populated and compiled.

Internally:

• tcc_ffi_create_state() creates the state with bundled TinyCC include/lib paths, user include/lib paths, and R headers/runtime library paths
• user compiler options are applied with tcc_set_options()
• tcc_ffi_compile_state() adds requested libraries, always links R, compiles the generated C string, injects host symbols, then relocates

This split is useful because both tcc_compile() and tcc_link() follow the same broad pattern even though one starts from user C source and the other starts from external-library declarations.

The Compiled Object Is an Environment of Closures

After relocation, tcc_compiled_object() recovers wrapper symbols with tcc_get_symbol() and turns them into R callables with make_callable().

That compiled object is an environment, not an S4 class or external pointer wrapper. The environment stores:

• callable closures for user symbols
• callable closures for generated helpers
• the live TCC state
• metadata such as symbol specs and helper specs

For non-variadic functions, make_callable() creates a closure that:

• checks arity
• checks that the wrapper pointer is still valid
• calls the wrapper through .Call

For variadic bindings, the closure selects the matching precompiled wrapper first, then calls that wrapper pointer.

Serialization Works by Recompiling the Recipe

Compiled wrapper pointers do not survive serialization as usable machine code. Rtinycc handles this by storing the original recipe:

• tcc_compile() stores .ffi on the compiled object
• tcc_link() stores .link_args
• $.tcc_compiled checks whether the state pointer is still valid
• if not, recompile_into() rebuilds a fresh compiled object and copies the bindings back into the target environment

So serialization support is not pointer persistence. It is recipe persistence plus transparent recompilation.

Where To Read Next

If you want to inspect the implementation directly, the main files are:

• R/ffi.R: high-level FFI object, compilation flow, compiled-object assembly
• R/ffi_codegen.R: generated wrapper and translation-unit builders
• R/aaa_ffi_codegen_rules.R: rule tables for conversions and mapping
• R/callbacks.R: callback parsing and trampoline generation helpers
• src/RC_libtcc.c: TCC/R bridge, host symbol injection, callback runtime

Rtinycc documentation built on April 28, 2026, 1:07 a.m.