pyspark.sql.tvf.TableValuedFunction.inline#

TableValuedFunction.inline(input)[source]#

Explodes an array of structs into a table.

This function takes an input column containing an array of structs and returns a new column where each struct in the array is exploded into a separate row.

New in version 4.0.0.

Parameters

inputColumn: Input column of values to explode.

Returns

DataFrame

See also

pyspark.sql.functions.inline()

Examples

Example 1: Using inline with a single struct array

>>> import pyspark.sql.functions as sf
>>> spark.tvf.inline(sf.array(
...     sf.named_struct(sf.lit("a"), sf.lit(1), sf.lit("b"), sf.lit(2)),
...     sf.named_struct(sf.lit("a"), sf.lit(3), sf.lit("b"), sf.lit(4))
... )).show()
+---+---+
|  a|  b|
+---+---+
|  1|  2|
|  3|  4|
+---+---+

Example 2: Using inline with an empty struct array column

>>> import pyspark.sql.functions as sf
>>> spark.tvf.inline(sf.array().astype("array<struct<a:int,b:int>>")).show()
+---+---+
|  a|  b|
+---+---+
+---+---+

Example 3: Using inline with a struct array column containing null values

>>> import pyspark.sql.functions as sf
>>> spark.tvf.inline(sf.array(
...     sf.named_struct(sf.lit("a"), sf.lit(1), sf.lit("b"), sf.lit(2)),
...     sf.lit(None),
...     sf.named_struct(sf.lit("a"), sf.lit(3), sf.lit("b"), sf.lit(4))
... )).show()
+----+----+
|   a|   b|
+----+----+
|   1|   2|
|NULL|NULL|
|   3|   4|
+----+----+