lnav is a wonderful log file viewer in terminal for Mac and Linux. This repo provides a CFML log format definition file for lnav so that you can use lnav properly to view CF & Lucee log files.
To use, download the cflog.json definition file from the lnav-format folder then install it with lnav using the following command:
lnav -i cflog.json
lnav is powerful. To take advantage of its full features, refer to the documentations. Below is a partial list of what it can do:
lnav /path/to/log/folderto view all log files in a folder, orlnav /path/to/log/file.logto view a single log file- Basic log content navigation:
Left/Right,PgUp/PgDn,Home/End - Note: for some hotkeys, use it with
shiftto do the opposite function (e.g.fto jump to the next file andshift + fto jump to previous file) ctrl + wto toggle word-wrappto show log line parsing result (display the log line into items so they are easier to read)shift+pto format the log content for better display or printing (for example if you have json or xml in log, it will format them with proper indention)- Navigate through log entries base on time period:
1=10 mins,2=20 mins, ..,6=60 mins,d=24hrs ito show histogrammto bookmark the line, and then useuto go through bookmarks/to start searching with regexp, then usento go through search result line:to trigger command mode:filter-inand:filter-outto filter the log content:commentto comment on a line,:clear-commentto remove:tagto tag a line,:un-tagto remove,:delete-tagsto remove tag(s):save-sessionto save the current stage/config to a named session, and use:load-sessionto load it back
- lnav will use a session to remember your commands such filter-in/out and markers. You can use
ctrl + rto reset the session ;to run SQL query on the log data. Do aselect * from cflogto see what column names you can use for query, some of them are:log_time,log_level,log_mark,log_comment,log_tag
- Use
tabto go into filter panel (useqto exist the panel), and while in the filter panel:ito add a filter-in,oto add a filter-out,tto flip your filter (e.g. filter IN becomes filter OUT and vice versa),spacebarto enable/disable the filter
- If the view contains multiple files, use
fto go to next file - Change theme (default themes are: default, eldar, grayscale, monocai, night-owl, solarized-dark, and solarized-light)
:config /ui/theme/ <theme_name>
If you want to write your own log format file, see lnav log formats docs. Below is a brief explanation on how the one in this repo is written.
The regexp pattern in the format definition file might look massive, but it is mostly due to we need to escape all the double quotes used in a CF log file as well as the regexp character classes (such as \d). Below is an explanation on how the one in this repo is constructed.
A CF log line can be something likes:
"INFO","XNIO-1 task-2","05/18/2022","09:48:55","lucee.runtime.CFMLFactoryImpl","Reset 6 Unused PageContexts"
So the regexp to parse a CF log line could be likes:
^"([A-Z]+)","([^\"]+)","(\d{2}\/\d{2}\/\d{4}","\d{2}:\d{2}:\d{2})","([^\"]+)","([^\"]+)"$
The regexp will give us 5 groups, from the line example above they are:
- INFO
- XNIO-1 task-2
- 05/18/2022","09:48:55
- lucee.runtime.CFMLFactoryImpl
- Reset 6 Unused PageContexts
Now we can add our lnav fields into the regexp pattern. The lnav fields are written as ?<field> and will be placed before the group regexp. The fields we used for the 5 groups matched by regexp are:
?<severity>- Our custom field name forlevel. Note that if you use a custom name then you must define it in thelevel-fielditem (see the section "Other items" below)?<thread_id>- Our custom field name?<timestamp>- This is the field name lnav used to parse date & time, you must use this name exactly?<application>- Our custom field name?<body>- Custom field name for log message (most people usebody, so we keep it this way)
So the final regexp pattern for lnav will be:
^"(?<severity>[A-Z]+)","(?<thread_id>[^"]+)","(?<timestamp>\d{2}\/\d{2}\/\d{4}","\d{2}:\d{2}:\d{2})","(?<application>[^"]+)","(?<body>[^"]+)"$
And to use that pattern string in a json file you need to escape double quotes and slashes, hence the final pattern string will be:
^\"(?<severity>[A-Z]+)\",\"(?<thread_id>[^\"]+)\",\"(?<timestamp>\\d{2}\/\\d{2}\/\\d{4}\",\"\\d{2}:\\d{2}:\\d{2})\",\"(?<application>[^\"]+)\",\"(?<body>[^\"]+)\"$
Note: for the ?<timestamp> field, the matched value contains date & time value in the format likes dd/mm/yyyy","hh:mm:ss (notice the double quotes and comma between the date & time value), so in the timestamp-format item (see the section "Other items" below) we can tell lnav how to properly parse the date and time out of this string as %m/%d/%Y\",\"%H:%M:%S
level-field- This defines the column name from your regexp pattern that lnav will use to determine the alert level value (i.e. Info, Warning, Error, Fatal etc).
level- A structure defines the alert levels and their corresponding text value in the column
value- A structure defines the column types you used in the regexp pattern.
timestamp-format- A string tells lnav how to parse the date & time from the
?<timestamp>column value it get from the regexp pattern.
- A string tells lnav how to parse the date & time from the
sample- A log line example. If lnav have trouble parsing this sample then it won't load the log files and will throw an error telling you what went wrong.
- https://github.com/hagfelsh/lnav_formats.git
- https://github.com/PaulWay/lnav-formats.git
- https://github.com/penntaylor/lnav-ruby-logger-format.git
- https://github.com/aspiers/lnav-formats.gi
- Improve the log format definition regular express
- It currently cannot handel the log body message that contains double quote