Processor

Properties

append object: Appends one or more values to an existing array if the field already exists and it is an array. Converts a scalar to an array and appends one or more values to it if the field exists and it is a scalar. Creates an array containing the provided values if the field doesn’t exist. Accepts a single value or an array of values.
attachment object: The attachment processor lets Elasticsearch extract file attachments in common formats (such as PPT, XLS, and PDF) by using the Apache text extraction library Tika.
bytes object: Converts a human readable byte value (for example 1kb) to its value in bytes (for example 1024). If the field is an array of strings, all members of the array will be converted. Supported human readable units are "b", "kb", "mb", "gb", "tb", "pb" case insensitive. An error will occur if the field is not a supported format or resultant value exceeds 2^63.
cef object: Converts a CEF message into a structured format.
circle object: Converts circle definitions of shapes to regular polygons which approximate them.
community_id object: Computes the Community ID for network flow data as defined in the Community ID Specification. You can use a community ID to correlate network events related to a single flow.
convert object: Converts a field in the currently ingested document to a different type, such as converting a string to an integer. If the field value is an array, all members will be converted.
csv object: Extracts fields from CSV line out of a single text field within a document. Any empty field in CSV will be skipped.
date object: Parses dates from fields, and then uses the date or timestamp as the timestamp for the document.
date_index_name object: The purpose of this processor is to point documents to the right time based index based on a date or timestamp field in a document by using the date math index name support.
dissect object: Extracts structured fields out of a single text field by matching the text field against a delimiter-based pattern.
dot_expander object: Expands a field with dots into an object field. This processor allows fields with dots in the name to be accessible by other processors in the pipeline. Otherwise these fields can’t be accessed by any processor.
drop object: Drops the document without raising any errors. This is useful to prevent the document from getting indexed based on some condition.
enrich object: The enrich processor can enrich documents with data from another index.
fail object: Raises an exception. This is useful for when you expect a pipeline to fail and want to relay a specific message to the requester.
fingerprint object: Computes a hash of the document’s content. You can use this hash for content fingerprinting.
foreach object: Runs an ingest processor on each element of an array or object.
ip_location object: Currently an undocumented alias for GeoIP Processor.
geo_grid object: Converts geo-grid definitions of grid tiles or cells to regular bounding boxes or polygons which describe their shape. This is useful if there is a need to interact with the tile shapes as spatially indexable fields.
geoip object: The geoip processor adds information about the geographical location of an IPv4 or IPv6 address.
grok object: Extracts structured fields out of a single text field within a document. You choose which field to extract matched fields from, as well as the grok pattern you expect will match. A grok pattern is like a regular expression that supports aliased expressions that can be reused.
gsub object: Converts a string field by applying a regular expression and a replacement. If the field is an array of string, all members of the array will be converted. If any non-string values are encountered, the processor will throw an exception.
html_strip object: Removes HTML tags from the field. If the field is an array of strings, HTML tags will be removed from all members of the array.
inference object: Uses a pre-trained data frame analytics model or a model deployed for natural language processing tasks to infer against the data that is being ingested in the pipeline.
join object: Joins each element of an array into a single string using a separator character between each element. Throws an error when the field is not an array.
json object: Parses a string containing JSON data into a structured object, string, or other value.
kv object: This processor helps automatically parse messages (or specific event fields) which are of the foo=bar variety.
lowercase object: Converts a string to its lowercase equivalent. If the field is an array of strings, all members of the array will be converted.
network_direction object: Calculates the network direction given a source IP address, destination IP address, and a list of internal networks.
normalize_for_stream object: Detects whether a document is OpenTelemetry-compliant and, if not, normalizes it so that it can be queried by clients that expect either ECS or OpenTelemetry Semantic Conventions formats.
pipeline object: Executes another pipeline.
recover_failure_document object: Recovers documents that have been stored in a data stream's failure store by restoring them to their original format.
redact object: The Redact processor uses the Grok rules engine to obscure text in the input document matching the given Grok patterns. The processor can be used to obscure Personal Identifying Information (PII) by configuring it to detect known patterns such as email or IP addresses. Text that matches a Grok pattern is replaced with a configurable string such as <EMAIL> where an email address is matched or simply replace all matches with the text <REDACTED> if preferred.
registered_domain object: Extracts the registered domain (also known as the effective top-level domain or eTLD), sub-domain, and top-level domain from a fully qualified domain name (FQDN). Uses the registered domains defined in the Mozilla Public Suffix List.
remove object: Removes existing fields. If one field doesn’t exist, an exception will be thrown.
rename object: Renames an existing field. If the field doesn’t exist or the new name is already used, an exception will be thrown.
reroute object: Routes a document to another target index or data stream. When setting the destination option, the target is explicitly specified and the dataset and namespace options can’t be set. When the destination option is not set, this processor is in a data stream mode. Note that in this mode, the reroute processor can only be used on data streams that follow the data stream naming scheme.
script object: Runs an inline or stored script on incoming documents. The script runs in the ingest context.
set object: Adds a field with the specified value. If the field already exists, its value will be replaced with the provided one.
set_security_user object: Sets user-related details (such as username, roles, email, full_name, metadata, api_key, realm and authentication_type) from the current authenticated user to the current document by pre-processing the ingest.
sort object: Sorts the elements of an array ascending or descending. Homogeneous arrays of numbers will be sorted numerically, while arrays of strings or heterogeneous arrays of strings + numbers will be sorted lexicographically. Throws an error when the field is not an array.
split object: Splits a field into an array using a separator character. Only works on string fields.
terminate object: Terminates the current ingest pipeline, causing no further processors to be run. This will normally be executed conditionally, using the if option.
trim object: Trims whitespace from a field. If the field is an array of strings, all members of the array will be trimmed. This only works on leading and trailing whitespace.
uppercase object: Converts a string to its uppercase equivalent. If the field is an array of strings, all members of the array will be converted.
urldecode object: URL-decodes a string. If the field is an array of strings, all members of the array will be decoded.
uri_parts object: Parses a Uniform Resource Identifier (URI) string and extracts its components as an object. This URI object includes properties for the URI’s domain, path, fragment, port, query, scheme, user info, username, and password.
user_agent object: The user_agent processor extracts details from the user agent string a browser sends with its web requests. This processor adds this information by default under the user_agent field.