nlp_embeddings_finisher: Spark NLP EmbeddingsFinisher

In r-spark/sparknlp: R Interface to John Snow Labs Spark NLP

Spark NLP EmbeddingsFinisher

Description

Spark ML transformer that is designed to deal with embedding annotators: WordEmbeddings, BertEmbeddings, SentenceEmbeddingd, and ChunkEmbeddings. By using EmbeddingsFinisher you can easily transform your embeddings into array of floats or Vectors which are compatible with Spark ML functions such as LDA, K-mean, Random Forest classifier or any other functions that require featureCol. See https://nlp.johnsnowlabs.com/docs/en/transformers#embeddingsfinisher

Usage

nlp_embeddings_finisher(
  x,
  input_cols,
  output_cols,
  clean_annotations = NULL,
  output_as_vector = NULL,
  uid = random_string("embeddings_finisher_")
)

Arguments

x	A spark_connection, ml_pipeline, or a tbl_spark.
input_cols	Input columns. String array.
output_cols	Output columns. String array.
clean_annotations	Whether to remove and cleanup the rest of the annotators (columns)
output_as_vector	if enabled, it will output the embeddings as Vectors instead of arrays
uid	A character string used to uniquely identify the ML estimator.

Value

The object returned depends on the class of x.

• spark_connection: When x is a spark_connection, the function returns an instance of a ml_estimator object. The object contains a pointer to a Spark Estimator object and can be used to compose Pipeline objects.

• ml_pipeline: When x is a ml_pipeline, the function returns a ml_pipeline with the NLP estimator appended to the pipeline.

• tbl_spark: When x is a tbl_spark, an estimator is constructed then immediately fit with the input tbl_spark, returning an NLP model.

r-spark/sparknlp documentation built on Oct. 15, 2022, 10:50 a.m.