Apex Logs provides a powerful query language supporting both plain-text and structured logs. This section explains the advantage that structured logging can provide over plain-text logs, as well as the query language reference.
To make the most out of structured logging it’s recommended to provide actionable, low cardinality message values to represent actions, while providing additional context information in fields rather than “baking” it right into the message.
For example using the message File Upload instead of tj@example.com uploaded some.png, as the former contains no dynamic information it’s easy to filter on and perform analytical queries against. Here’s what an example log Event might look like:
{
"level": "info",
"message": "Upload",
"fields": {
"file": "sloth.png",
"type": "image/png",
"duration": 2502,
"size": 43008,
"user": {
"name": "Tobi",
"email": "tobi@example.com"
},
"source": {
"host": "api-01"
}
}
}
Structured logs enable precise queries such as the following:
message = "Upload"
message = "Upload" and user.email = "tobi@example.com" and type = "image/*"
message = "Upload" and duration > 2s
message = "Upload" and duration between 1s and 1m
message = "Upload" and size >= 100kb
Simple search terms can also work in simple cases, as they search the event’s message and fields. For example instead of message = "Upload" you could search:
Upload
And instead of message = "Upload" and user.email = "tobi@example.com" and type = "image/png" a similar, but less precise search query would be:
Upload tobi@example.com image/png
Plain-text logs are essentially arbitrary lines of text, lacking any real structure. Here are a few examples:
user tobi@example.com signed in
user tobi@example.com uploaded sloth.png (took 1500ms) (103kb)
user tobi@example.com upload failed: Operation not permitted
A plain-text Event essentially bakes all of the information into the message, however, you may optionally fields to provide additional context such as the source host:
{
"level": "info",
"message": "user tobi@example.com uploaded sloth.png (image/png) (took 2.5s) (43kb)",
"fields": {
"source": {
"host": "api-01"
}
}
}
To search against these logs you could use queries similar to:
tobi@example.com ("signed in" or "signed out")
tobi@example.com uploaded
"Operation not permitted"
With plain-text logs it’s not possible to perform complex queries against the the duration (took 1500ms) or size (103kb) values, as they’re part of the message string.
With Apex Logs there’s no need to choose between the two techniques, you can mix and match the two as necessary.
Searches are case sensitive, with terms broken up by whitespace. A term is a single word, such as user or login as shown below, which will match the text “user” or “login” anywhere in the event.
user login
This query would match both of the following, as the two terms are joined with the AND operator:
user tj@apex.sh login
user login with email tj@apex.sh
If the term contains whitespace you can use single or double qoutes for an exact match:
"user login"
Only the following line is now matched:
user login with email tj@apex.sh
Terms search the entire event, so it also works well with structured log events, for example:
login tj@apex.sh
Will match the event:
{
"level": "info",
"message": "login",
"fields": {
"name": "Tobi",
"email": "tj@apex.sh"
}
}
In this case better performance can be achieved with a more precise query:
message = "login" email = "tj@apex.sh"
An unquoted search term can contain any character except the following: (, ), <, >, =, ,, !, ", ', |. For example the following are valid unquoted search terms:
~/path/to/image.png
tobi@example.com
64812AD3-46FD-4991-AE52-4D5514D8E655
A log event has an associated severity level, indicating if the log is purely informational, or represents a problem with your application. The following severity levels are supported:
Severity levels are treated as integers internally, ranging from 0 to 7, allowing for use with relational operators:
level >= info
level = error
When used alone, the severity level keywords behave as an >= expression. For example these expressions are identical, allowing you quickly show all categories of error:
error
level >= error
If you want to search for the literal text “info” or “error” anywhere in your event you can quote them to treat it as a term:
"info"
"error"
This section covers the literals available.
Booleans use the true and false keywords:
user.is_admin = true
user.is_admin = false
Integers use a sequence of digits.
cart.total > 15
You may use _ as a thousands separator, for example:
1_000_000
Floats use a sequence of digits followed by a ., another sequence of digits,
and optional exponent.
cart.total > 15.99
You may use _ as a thousands separator, for example:
1_000_000.99
The numeric duration units use a millisecond resolution, using the suffixes ms, s, m, h, and d.
message = "uploaded file" and duration > 1500
message = "uploaded file" and duration > 1500ms
message = "uploaded file" and duration > 1s
message = "uploaded file" and duration > 1.5s
message = "uploaded file" and duration > 5m
message = "uploaded file" and duration > 3h
message = "uploaded file" and duration > 1d
The numeric byte size units use the suffixes b, kb, mb, gb, and tb for querying multiples of 1024.
message = "uploaded file" and file.size > 1024
message = "uploaded file" and file.size > 1024b
message = "uploaded file" and file.size > 1kb
message = "uploaded file" and file.size > 2.5mb
message = "uploaded file" and file.size > 1gb
The backtick is used to denote a regular expression. By default the event’s message and fields are matched against the pattern, for example:
"Cart Checkout" `(tobi|jane)@example.com`
Matches the following event:
{
"level": "info",
"message": "Cart Checkout",
"fields": {
"user": {
"name": "Jane",
"email": "jane@example.com"
}
}
}
Logical operators evaluate to a boolean, allowing you to define more complex filter operations.
The and operator evalutes to true if both conditions are truthy:
error and message = "Upload Failed"
The and operator is implied by default:
message = "Upload Failed" host = "api-01"
Is equivalent to:
message = "Upload Failed" and host = "api-01"
The or operator evalutes to true if either condition is truthy:
message = "Uploaded File" and (file.type = "image/png" or file.name = "*.png")
The ! operator negates an expression:
!(url.path = "/admin" and user.is_admin = true)
The exists operator evalutes to true if the field is defined.
user.cart exists
Equality operators perform comparisons between fields and values.
The = operator evalutes to true if both operands are the same.
region = "us-west2"
The wildcard character * can be used to match zero or more characters:
user.email = "*@example.com"
user.email = "tobi@*"
user.email = "*@example.*"
The right-hand operand can be a regular expression with backticks:
user.email = `^(tobi|jane)@example\.com$`
This operator is aliased as ==.
The != operator evalutes to true if both operands are not the same.
region != "us-west2"
The wildcard character * can be used to match zero or more characters:
host != "api-*"
The right-hand operand can be a regular expression with backticks:
user.email != `^(tobi|jane)@example\.com$`
The contains operator evalutes to true if the left operand contains the string on the right.
product.name contains "Logitec"
product.name not contains "Logitec"
This is an alias of:
product.name = "*Logitec*"
product.name != "*Logitec*"
The starts with operator evalutes to true if the left operand starts with the string on the right.
region starts with "us-"
region not starts with "us-"
This is an alias of:
region = "us-*"
region != "us-*"
The ends with operator evalutes to true if the left operand ends with the string on the right.
region ends with "-west2"
region not ends with "-west2"
This is an alias of:
region ends with "*-west2"
region not ends with "*-west2"
The between operator evalutes to true if the left operand is between the range (inclusive).
response.status between 200 and 299
response.status not between 200 and 299
The > operator evalutes to true if the left operand is greater than the right.
cart.total > 15
The >= operator evalutes to true if the left operand is greater than or equal to the right.
cart.total >= 15
The < operator evalutes to true if the left operand is less than the right.
cart.total < 15
The <= operator evalutes to true if the left operand is less than or equal to the right.
cart.total <= 15
Event fields are accessed using the . operator. For example the following:
user.name = "Tobi"
user.email = "*@example.com"
cart.total > 15.99
Would match the event:
{
"level": "info",
"message": "Cart Checkout",
"fields": {
"user": {
"name": "Tobi",
"email": "tobi@example.com"
},
"cart": {
"items": [
{
"name": "Ferret Food",
"cost": 15.99
}
],
"total": 15.99
},
"source": {
"host": "api-01"
}
}
}
Array elements are accessed using the [] operator with an index:
cart.items[0].name = "Ferret Food"
cart.items[0].cost > 10
You may group expressions using the ( and ) characters:
message = "Uploaded File" and (region = "us-west-2" or region = "us-east-1")
Show logs with an error severity or greater:
error
{
"level": "error",
"message": "Upload Failed",
"fields": {
"error": "Operation not permitted",
"path": "/tmp/image.png"
}
}
Show missing file errors in plain-text Apache error logs from the US regions:
"File does not exist" region = "us-*"
{
"level": "info",
"message": "[Thu Mar 13 19:04:13 2014] [error] [client 50.0.134.125]
File does not exist: /var/www/favicon.ico
",
"fields": {
"host": "api-01",
"region": "us-west-2",
"source": "apache"
}
}
Show a specific Lambda request by ID:
REPORT 4b3ab4c4-75e4-431f-81dc-783f4b74a243
{
"level": "info",
"message": "REPORT RequestId: 4b3ab4c4-75e4-431f-81dc-783f4b74a243
Duration: 4687.99ms Billed Duration: 4700 ms Memory Size: 128 MB
Max Memory Used: 51 MB",
"fields": {
"aws": {
"region": "us-west-2",
"lambda": {
"function": "api",
"version": 9
}
}
}
}
Errors caused by a particular Lambda function and version:
error and function.name = "alert_processor" and function.version = "v1.5.0"
{
"level": "error",
"message": "Processing alert",
"fields": {
"error": "Error connecting to database: no such host",
"function": {
"name": "alert_processor",
"version": "v1.5.0"
}
}
}
Slow responses caused by a particular user.
user.email = "tobi@example.com" and duration >= 1.5s
{
"level": "info",
"message": "HTTP response",
"fields": {
"duration": 2300,
"user": {
"email": "tobi@example.com"
}
}
}
Image upload errors:
error and message = "Upload Failed" and file.type = "image/*"
{
"level": "error",
"message": "Upload Failed",
"fields": {
"file": {
"name": "sloth.png",
"type": "image/png",
"size": 40960
}
}
}
Match slow HTTP responses for GET requests to /profile:
GET /profile response duration >= 1.5s
{
"level": "info",
"message": "response",
"fields": {
"method": "GET",
"path": "/profile",
"duration": 1800,
"user": {
"email": "tobi@example.com"
}
}
}
Match slow HTTP responses for GET requests to /profile using fields:
message = "response"
and method = "GET"
and path = "/profile"
and duration >= 1.5s
{
"level": "info",
"message": "response",
"fields": {
"method": "GET",
"path": "/profile",
"duration": 1800,
"user": {
"email": "tobi@example.com"
}
}
}