For AI agents: the complete documentation index is available at https://docs.us.ovhcloud.com/en/llms.txt, the full documentation bundle is available at https://docs.us.ovhcloud.com/en/llms-full.txt, and this page is available as Markdown at https://docs.us.ovhcloud.com/en/manage-and-operate/logs-data-platform/field-naming-convention.md.
  • English

Logs Data Platform - Field Naming Convention

View as Markdown

Learn how to ensure that your logs are properly parsed.

Now that you can send logs, you may be wondering how to tell Logs Data Platform what kind of data you send. It can be dates, numbers, boolean values, or just plain text strings.

 

Requirements

  • no specific requirement

 

Instructions

What is a valid log for Logs Data Platform?

Each log received on Logs Data Platform is transformed into a GELF-formatted log. What is GELF? A standardized JSON way to send logs. GELF stands for Graylog Extended Log Format. Using this format gives us two advantages: it is directly compatible with Graylog and it is still extensible enough to enrich your logs as you would like to.

This format imposes a few conventions that if you don't follow can have many consequences:

  • Logs Data Platform will rewrite your field as an incorrect one (with a _fixit suffix).
  • Your log will be rejected.

First please check the table below to know which fields are available and their meaning. Note that some of these fields are mandatory and have to be set by whichever library you use to send logs to Logs Data Platform. Refer to the documentation of the library or to some of our awesome tutorials to know how to send them.

WordES typeAbout
versionStringGELF spec version – “1.1”; MUST be set by client library.
hostStringThe name of the host, source, or application that sent this message; MUST be set by client library.
short_messageStringA short descriptive message; MUST be set by client library.
full_messageStringA long message that can i.e. contain a backtrace; optional.
timestampNumberSeconds since UNIX epoch with optional decimal places for milliseconds; SHOULD be set by client library. Will be set to NOW by server if absent.
levelNumberThe level equal to the standard syslog levels; optional, default is 1 (ALERT).
lineNumberWe consider this value to be a standard in the logs messages so we force it to be a Number.
X-OVH-TOKENStringMandatory on direct access, don't try to forge strange ones, you will be banished.
X-OVH-CONTENT-SIZENumberSize in bytes of the current log.
X-OVH-TO-FREEZEStringIf given, will build another archive containing only its value (separated by a line break).

 

Warning

NOTE: You cannot use the field source since it is replaced by the content of the field host in Graylog.

 

Can I go deeper?

Of course. As we said before, you can send some additional fields as long as you prepend them with the _ (underscore) character. You can use any JSON valid character for your field, except the . (dot) character. But don't worry, if you do so, we will rewrite your '.' as an underscore. So how can you send special type as number, date, and boolean? Here is the answer:

Case sensitive suffixES TypeAbout
_num, _double, *_floatdoublefloating value in double in java representation: double-precision 64-bit IEEE 754 floating point
_int, _longlong64-bit signed long type, which has a minimum value of -263 and a maximum value of 263-1
*_datedatean ISO 8601 date with optional Time or Milliseconds since UNIX epoch in Integer.
*_boolbooleanExpected values: "true" or "false". WARNING: GELF does not support boolean types you will have to send "true" or "false" in String
*_geolocationStringA pair of two float numbers separated by a comma ','. This pair must represent Latitude and Longitude. For OpenSearch Dashboards & Grafana compatibility, the value is also copied to a GeoHash: *_geolocation.geo
*_ipStringA valid IPv4 or IPv6. This will allows you to search by range (dst_ip:[10.0.0.0 TO 10.255.255.255]) or netmask (dst_ip:10.0.0.0\/8)
Everything elseStringAnything else will be considered a string

As you can see, it is fairly straightforward. Suffix your field with the right value and you will be able to send anything you want. For reference, here is a full example of a valid GELF message with every type we have:

{
 "version":"1.1",
 "host":"my_host",
 "_some_num":87.6,
 "_some_user_id_float":123,
 "a_good_date":"2016-01-01T17:04:25.000",
 "short_message":"A short message that can save your life",
 "full_message":"all the things you want up to 32768 characters",
 "_line":18,
 "level":1,
 "_power_level_int":"9001",
 "_some_info":"info",
 "_ovh_is_wonderful_bool":"true",
 "_dst_ip":"51.38.195.65"
}

Specifying the correct numeric suffix type is the only way to generate numeric Widgets for your Dashboards. Here is an example of a graph you can generate with a numeric value:

Operate ingest names - step 1

Our platform limits the usage of IP addresses as field keys. IP addresses have a high cardinality and thus are not allowed to be used as keys (of course they are supported and enriched as values, as you can see above). If you use an IP address as a key, it will be changed. For example:

This...

{
 "version":"1.1",
 "host":"my_host",
 "_some_user_id_float":123,
 "short_message":"A short message that can save your life",
 "192.168.1.1":"SSL Handshake Failures"
}

becomes this...

{
 "version":"1.1",
 "host":"my_host",
 "_some_user_id_float":123,
 "short_message":"A short message that can save your life",
 "invalid_ip_fields":"192.168.1.1",
 "invalid_ip_fields_values":"SSL Handshake Failures",
 "ovh_warn_ip_as_field":"One of your field name is an IP"
}

 

Go further

Browse all the Logs Data Platform guides.

Browse the rest of the OVHcloud US documentation for guides on our other products and services.

Learn more about Logs Data Platform on the OVHcloud US website.