Advanced Query Builder Syntax

The Advanced Query Builder is being introduced as an enhancement to improve filtering capabilities within the Logs Page. The existing traditional query builder lacks support for logical operators like AND and OR, limiting users to basic attribute-value filtering.

Users can switch to Advanced Query Mode for more complex filtering.
Supports multiple operators to refine log searches effectively.
Affects the Logs Explorer Page, Log Alerts, Metric Definitions, Live Tail, and Log Tiles in Dashboards.

Label search helps users filter logs by specific label values using different operators. Users define a label, choose an operator, and assign a value to create queries that refine log searches.

The following are details of various operators and query functions.

Basic Syntax

The standard format for label search queries follows:

Supported Operators

Equals (=)

Example: source = "agent"

Matches logs where the source is exactly "agent".

Not Equals (!=)

Example: source != "agent"

Matches logs where the source is not "agent".

Regex Match (=~)

Example: source =~ "agent"

Matches logs where the source matches the regex pattern "agent".

Not Regex Match (!~)

Example: source !~ "agent"

Matches logs where the source does not match the regex pattern "agent".

Combining Multiple Filters

Logical operators allow users to create more refined queries:

AND (| or AND)

Example: source = “agent” | level != "debug"

source = “agent” AND level = "error"

Matches logs where both conditions are true.

OR (OR)

Example: source = “agent” OR source = "kubernetes"

Returns logs where the source is either “agent” or "kubernetes".

Grouping with Parentheses

Example: source = "agent" AND (level = "error" OR level = "warn")

Returns logs where the source is "agent" and the level is either "error" or "warn".

Using the IN and NOT IN Operators

These operators allow checking against multiple values, supporting both plain strings and regex patterns:

Match Exact Values

Example: source IN ("aws", "azure", "agent")

Matches logs where the source is "aws", "azure", or "agent".

Exclude Specific Values

Example: source NOT IN ("aws", "azure", "agent")

Excludes logs where the source is "aws", "azure", or "agent".

Regex Patterns

Example: source IN ("a.*")

Matches any source starting with "a " (e.g., "aws", "agent").

Exclude Using Regex

Example: source NOT IN ("a.*", "syslog")

Excludes logs where the source starts with "a " or is exactly "syslog ".

Note

Values inside IN and NOT IN can be either plain strings or regular expressions.
The IN operator is supported in Log Explorer, Alerts page, Metric page, and the Sear Logs API.

Using groupBy in the Advanced Query Builder

The groupBy function enables users to group logs by one or more fields, primarily for count-based aggregations. It is only supported in:

Log Tile (on the dashboard)
Log Metrics

If used elsewhere, the function will be ignored.

Example Usage

groupBy(source, level)

Groups logs by source and level, counting entries for each unique combination.

More Query Examples

container_name IN ("logs-* ", "traces-*") AND container_name != "logs-query" AND level IN ("error" "fatal")

Matches logs where:
- container_name matches "logs-" or "traces-".
- container_name is not "logs-query".
- level is "error" or "fatal".

source = "kubernetes" AND level != "debug"

Matches logs from Kubernetes where the level is not "debug".

source IN ("agent", "syslog") AND message =~ "timeout"

Matches logs where the source is either "agent" or "syslog", and the message contains "timeout" (regex).

env = "prod" AND (level = "error" OR level = "warn")

Returns production logs where the level is either "error" or "warn".

container_name =~ "api-.*" AND level IN ("info", "error") AND message !~ "health"

Matches logs where:
- container_name starts with "api-".
- level is "info" or "error".
- message does not contain "health".

service != "auth-service" OR status_code = "500"

Matches logs where:
- The service is not "auth-service", or
- The status_code is "500".

Additional Functionalities

The additional functions for Advanced Query Builder include the following three functions.

splitString – Splits a string into multiple parts based on a delimiter.
SplitRegex – Uses regular expressions to split strings dynamically.
parseJson – Extracts structured data from JSON strings for better analysis.

splitString Function

The splitString function is used to divide a string field into multiple parts based on a specified delimiter (e.g., :, -, _). It then extracts a specific part using a 1-based index. This is particularly useful when dealing with composite fields like container image names (gcr.io/my-image:latest) or log strings that follow a predictable format.

Syntax

splitString(field=<field_name>, by='<delimiter>', index=<number>) [AS alias_name]

Parameters

Parameter	Description	Required/Optional
field	The field that contains the string you want to split.	Required
by	The delimiter used to split the string.	Required
index	Position to extract	Required
AS alias_name	Optional alias for the result of the function.	Optional

Default Behaviors

If index is not specified, it defaults to 0.
If alias is not specified, the default alias is <field_name>[<index>].

Examples

Extract the first part of the image (default index=1)

splitString(field=kubernetes_container_image, by=':')

  Alias: kubernetes_container_image[1]

Extract the second part with a custom alias

splitString(field=kubernetes_container_image, by=':', index=1) AS image_tag

Compare extracted value to a specific version

splitString(field=kubernetes_container_image, by=':', index=2) = '19.2' AS image_tag

  Meaning: only match logs where image tag is exactly '19.2'

Extract image tag and check if it’s greater than version ‘19.2’
```
splitString(field=kubernetes_container_image, by=':', index=2) > '19.2' AS image_tag
```
Meaning: include entries with version > ‘19.2’ (string comparison)

SplitRegex Function

The splitRegex function works like splitString, but uses a regular expression pattern for splitting. This is ideal when the delimiter is not fixed or is more complex.

Syntax

splitRegex(field=<field_name>, by='<delimiter>', index=<number>) [AS alias_name]

Parameters

Parameter	Description	Required/Optional
field	The field to split.	Required
regex	Regular expression pattern to split on.	Required
index	Position to extract	Required
AS alias_name	Optional alias for the result of the function.	Optional

Default Behaviors

If index is not specified, it defaults to 0.
If alias is not provided, the default alias is <field_name>[<index>].

Examples

Default index and alias (splits on whitespace)
Custom index with custom alias

splitRegex(field=log, by='\\s+')

    Creates alias: log[0]

Custom index with default alias

splitRegex(field=log, by='\\s+', index=2)

    Creates alias: log[2]

Custom index with custom alias

splitRegex(field=message, by='[-_:]', index=3) AS segment

parseJson Function

The parseJson function is designed to extract values from fields containing JSON-formatted strings. It uses dot notation to navigate nested JSON structures.

Syntax

parseJson(field=<field_name>, key=<json_path>) [AS alias_name]

Parameters

Parameter	Description	Required/Optional
field	Field containing the JSON string.	Required
key	Dot-separated path to the desired key.	Required
AS alias_name	Optional alias name for the extracted value.	Optional

Default Behaviors

If no alias is specified, the alias defaults to the JSON key path (e.g. kubernetes.container_name).

Behavior Notes

If the key path is not available, the result is null.
Supports nested keys using dot notation.

Examples

Extract a container name from nested JSON

parseJson(field=log, key=kubernetes.container_name)

     alias: kubernetes.container_name

Use custom alias

parseJson(field=log, key=kubernetes.container_name) AS container

    alias: container