Nginx logs to Graylog, input extractors

04 Oct 2021

To get the most out of Graylog we need to filter our log messages into separate fields. Those fields will later be used to create different searches which can be saved as a Dashboard widget.

Graylog Input extractors

Lets get started by going to System -> Inputs and press Manage extractors for our Nginx Access logs input.

Inputs - Manage Extractors
In the Manage Extractors page we just hit Get started to begin our journey to create our first extractor
Add extractor - Get started
On the next page, just press Load Message to load the last log message received to that input.
Load Message
There already exists some fields in our log. But the part we are interested in is all contained in the message field.
Select extractor type - Regular Expression
Click select extractor type on the message field and select Regular expression.

Our first extractor will just remove the first “crap” nginx logs, the result is a pretty JSON string we can use in our next extractor. The result from our regular expression filter is stored in the new field we call json. Use the Try button to make sure your regular expression works as intended. When you are done just hit Create extractor.

JSON extractor
Now we will get a new json field on our message. It does not work retroactive though, so you need to receive a new message for the field to appear.

When you have received a new message with our new json field. Go back to Manage Extractors and create a new one, after Load Message step click Select extractor type on the json field, and select JSON.

Select extractor type - JSON
JSON fields extractor
Source field extractor
Message field replacement extractor
Now you should have lots of fields in each of your messages. Makes it way easier to search and filter for stuff you want.