union: Return a new SparkDataFrame containing the union of rows

Description Usage Arguments Details Value Note See Also Examples

Description

Return a new SparkDataFrame containing the union of rows in this SparkDataFrame and another SparkDataFrame. This is equivalent to UNION ALL in SQL. Input SparkDataFrames can have different schemas (names and data types).

Usage

1
2
3
4
union(x, y)

## S4 method for signature 'SparkDataFrame,SparkDataFrame'
union(x, y)

Arguments

x

A SparkDataFrame

y

A SparkDataFrame

Details

Note: This does not remove duplicate rows across the two SparkDataFrames. Also as standard in SQL, this function resolves columns by position (not by name).

Value

A SparkDataFrame containing the result of the union.

Note

union since 2.0.0

See Also

rbind unionByName

Other SparkDataFrame functions: SparkDataFrame-class, agg(), alias(), arrange(), as.data.frame(), attach,SparkDataFrame-method, broadcast(), cache(), checkpoint(), coalesce(), collect(), colnames(), coltypes(), createOrReplaceTempView(), crossJoin(), cube(), dapplyCollect(), dapply(), describe(), dim(), distinct(), dropDuplicates(), dropna(), drop(), dtypes(), exceptAll(), except(), explain(), filter(), first(), gapplyCollect(), gapply(), getNumPartitions(), group_by(), head(), hint(), histogram(), insertInto(), intersectAll(), intersect(), isLocal(), isStreaming(), join(), limit(), localCheckpoint(), merge(), mutate(), ncol(), nrow(), persist(), printSchema(), randomSplit(), rbind(), rename(), repartitionByRange(), repartition(), rollup(), sample(), saveAsTable(), schema(), selectExpr(), select(), showDF(), show(), storageLevel(), str(), subset(), summary(), take(), toJSON(), unionAll(), unionByName(), unpersist(), withColumn(), withWatermark(), with(), write.df(), write.jdbc(), write.json(), write.orc(), write.parquet(), write.stream(), write.text()

Examples

1
2
3
4
5
6
7
8
## Not run: 
sparkR.session()
df1 <- read.json(path)
df2 <- read.json(path2)
unioned <- union(df, df2)
unions <- rbind(df, df2, df3, df4)

## End(Not run)

SparkR documentation built on June 3, 2021, 5:05 p.m.