Date

Mutate Elastic Compatible

Synopsis

Parses dates from a field in various formats into a standardized timestamp.

Schema

date:
  field: <ident>
  formats: <enum[]>
  description: <text>
  if: <script>
  ignore_failure: <boolean>
  locale: <string>
  on_failure: <processor[]>
  on_success: <processor[]>
  output_format: <string>
  tag: <string>
  target_field: <ident>
  timezone: <string>

Configuration

Field	Required	Default	Description
field	Y	-	Field containing the date to parse
formats	N	-	Array of expected date formats. Supports: standard Go time layouts, ISO8601, UNIX, UNIX_MS
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if parsing fails
locale	N	-	Locale for date parsing (e.g., "en_US")
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
output_format	N	2006-01-02T15:04:05.000Z07:00	Format pattern for the output timestamp
tag	N	-	Identifier
target_field	N	@timestamp	Field to store the parsed date
timezone	N	UTC	Timezone for parsing (e.g., "Europe/Amsterdam")

Details

The processor features an advanced timestamp parser that can automatically detect and parse a wide variety of date/time formats without requiring explicit format specification. It supports nearly all timestamp formats, including:

Years	YYYY, YY, yyyy, yy
Months	MMMM (full name), MMM (abbreviated), MM (zero-padded), M
Days	DD, D, dd, d
Hours	HH (24-hour), hh (12-hour), h
Minutes / Seconds	mm, m, ss, s
Milliseconds	S, SSS, SSSSSS, SSSSSSSSS
Timezones	Z, ZZ, ZZZ
AM / PM	A, a

Human-readable and machine timestamps, as well as Java time patterns and Go time layouts are supported. Numeric timestamps (i.e. Unix epoch in seconds/milliseconds) are handled.

Output formats and timezones are customizable. Also, templates can be used for dynamic timezone and locale selection.

note

By default, the parsed date is stored in the @timestamp field, but this can be customized using target_field.

warning

Date parsing will fail if none of the specified formats match the input. Set ignore_failure to true if errors should be tolerated.

Examples

Standard

Parsing a date using the standard format...	{ "initial_date": "25/04/2016 12:02:01" } date: field: initial_date target_field: timestamp formats: ["02/01/2006 15:04:05"] timezone: "Europe/Amsterdam"
converts it to the ISO format with timezone:	{ "initial_date": "25/04/2016 12:02:01", "timestamp": "2016-04-25T12:02:01.000+02:00" }

Detection

Parsing various timestamps without explicit specs...	{ "date1": "2023-12-29_15:04:05", "date2": "29/Dec/2023:15:04:05 +0000", "date3": "Fri Dec 29 15:04:05 2023", "date4": "1461578521", "date5": "1461578521000" } processors: - date: field: date1 target_field: time1 - date: field: date2 target_field: time2 - date: field: date3 target_field: time3 - date: field: date4 target_field: time4 - date: field: date5 target_field: time5
detects and normalizes all dates automatically:	{ "time1": "2023-12-29T15:04:05.000Z", "time2": "2023-12-29T15:04:05.000Z", "time3": "2023-12-29T15:04:05.000Z", "time4": "2016-04-25T12:02:01.000Z", "time5": "2016-04-25T12:02:01.000Z" }

Nested Fields with ISO8601

Processing dates in nested objects...	{ "user": { "login_date": "2016-04-25T12:02:01.789Z" } } date: field: user.login_date target_field: timestamp formats: ["2006-01-02T15:04:05.000Z07:00"]
maintains the correct timestamp:	{ "user": { "login_date": "2016-04-25T12:02:01.789Z" }, "timestamp": "2016-04-25T12:02:01.789Z" }

Output Formats

Specifying a custom format...	{ "initial_date": "25/04/2016 12:02:01" } date: field: initial_date target_field: timestamp formats: ["02/01/2006 15:04:05"] output_format: "2006-01-02 15:04:05"
result:	{ "initial_date": "25/04/2016 12:02:01", "timestamp": "2016-04-25 12:02:01" }

Dynamic Timezone and Locale

Template fields for timezone and locale...	{ "event_date": "2016-04-25 12:02:01", "tz": "Europe/London", "lang": "en_GB" } date: field: event_date target_field: timestamp formats: ["2006-01-02 15:04:05"] timezone: "{{{tz}}}" locale: "{{{lang}}}"
can be used for parsing:	{ "event_date": "2016-04-25 12:02:01", "tz": "Europe/London", "lang": "en_GB", "timestamp": "2016-04-25T12:02:01.000+01:00" }