pyspark.sql.functions.to_timestamp_ntz#

pyspark.sql.functions.to_timestamp_ntz(timestamp, format=None)[source]#

Parses the timestamp with the format to a timestamp without time zone. Returns null with invalid input.

New in version 3.5.0.

Parameters

timestampColumn or column name: Input column or strings.
formatColumn or column name, optional: format to use to convert type TimestampNTZType timestamp values.

See also

pyspark.sql.functions.to_date()
pyspark.sql.functions.to_timestamp()
pyspark.sql.functions.to_timestamp_ltz()
pyspark.sql.functions.to_utc_timestamp()
pyspark.sql.functions.to_unix_timestamp()
pyspark.sql.functions.date_format()

Examples

Example 1: Using default format to parse the timestamp string.

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('2015-04-08 12:12:12',)], ['ts'])
>>> df.select('*', sf.to_timestamp_ntz('ts')).show()
+-------------------+--------------------+
|                 ts|to_timestamp_ntz(ts)|
+-------------------+--------------------+
|2015-04-08 12:12:12| 2015-04-08 12:12:12|
+-------------------+--------------------+

Example 2: Using user-specified format ‘yyyy-MM-dd’ to parse the date string.

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame([('2016-12-31',)], ['dt'])
>>> df.select('*', sf.to_timestamp_ntz(df.dt, sf.lit('yyyy-MM-dd'))).show()
+----------+--------------------------------+
|        dt|to_timestamp_ntz(dt, yyyy-MM-dd)|
+----------+--------------------------------+
|2016-12-31|             2016-12-31 00:00:00|
+----------+--------------------------------+

Example 3: Using a format column to represent different formats.

>>> import pyspark.sql.functions as sf
>>> df = spark.createDataFrame(
...     [('2015-04-08', 'yyyy-MM-dd'), ('2025+01+09', 'yyyy+MM+dd')], ['dt', 'fmt'])
>>> df.select('*', sf.to_timestamp_ntz('dt', 'fmt')).show()
+----------+----------+-------------------------+
|        dt|       fmt|to_timestamp_ntz(dt, fmt)|
+----------+----------+-------------------------+
|2015-04-08|yyyy-MM-dd|      2015-04-08 00:00:00|
|2025+01+09|yyyy+MM+dd|      2025-01-09 00:00:00|
+----------+----------+-------------------------+