A fast and powerful log viewer that translates JSON logs into a pretty human-readable format. High performance and convenient features are the main goals.
- Automatic usage of the less pager by default for convenience.
- Log streaming with the
-P
flag that disables the pager. - Log record filtering by field key/value pairs with the
-f
option with support for hierarchical keys. - Quick and easy filtering by level with the
-l
option. - Quick and easy filtering by timestamp range using the
--since
and--until
options and intuitive formats:- RFC-3339 timestamp format.
- Current configured timestamp output format with the
-t
option or environment variable. - Human friendly shortcuts like
today
,yesterday
,friday
or relative offsets like-3h
or-14d
.
- Quick and easy hiding and unhiding of fields with the
-h
option. - Hide empty fields with the
-E
flag. - Lightning fast message sorting with automatic indexing for local files using the
-s
flag.- Handles ~1 GiB/s for the first scan and allows fast filtering by timestamp range and level without scanning the data afterwards.
- Works fast with hundreds of local files containing hundreds of gigabytes of data.
- Reindexes large, growing files at lightning speed, skipping unmodified blocks, ~10 GiB/s.
- Follow mode with live message sorting by timestamp from different sources using the
-F
flag and preview of several recent messages with the--tail
option. - Custom complex queries that can include and/or conditions and much more.
- Non-JSON prefixes with
--allow-prefix
flag. - Displays timestamps in UTC by default and supports easy timezone switching with the
-Z
option and the-L
flag for a local timezone. - Customizable via configuration file and environment variables, supports easy theme switching and custom themes.
- See performance section for more details.
-
Download latest release from download page
-
Download and extract using
curl
andtar
on Linuxcurl -sSfL https://github.com/pamburus/hl/releases/latest/download/hl-linux-x86_64-musl.tar.gz | tar xz
-
Download and extract using
curl
andtar
on macOScurl -sSfL https://github.com/pamburus/hl/releases/latest/download/hl-macos.tar.gz | tar xz
-
Install AUR package on Arch Linux
yay -S hl-log-viewer-bin
-
Install using cargo
cargo install --locked --git https://github.com/pamburus/hl.git
-
Concatenate all log files
Command
$ hl $(ls -tr /var/log/example/*.log)
Concatenates and humanizes all
*.log
files found in/var/log/example/
.
-
Concatenate all log files including gzipped log files
Command
$ hl $(ls -tr /var/log/example/*.{log,log.gz})
Concatenates and humanizes all
*.log
and*.log.gz
files found in/var/log/example/
.
-
Use default pager with default parameters
Command
$ hl example.log
Automatically opens
less
pager with default parameters. -
Override options for default pager
Command
$ LESS=-SR hl example.log
Opens
less
pager with disabled line wrapping. -
Use custom pager
Command
$ PAGER=bat hl example.log
Opens
bat
pager.
-
Errors only
Command
$ hl -l e
Displays only error log level messages.
-
Errors and warnings
Command
$ hl -l w
Displays only warning and error log level messages.
-
Errors, warnings and informational
Command
$ hl -l i
Displays all log messages except debug level messages.
-
Command
$ tail -f example.log | hl -P
Tracks changes in the example.log file and displays them immediately. Flag
-P
disables automatic using of pager in this case.
-
Command
$ hl example.log --filter component=tsdb
Displays only messages where the
component
field has the valuetsdb
. -
Command
$ hl example.log -f component!=tsdb -f component!=uninteresting
Displays only messages where the
component
field has a value other thantsdb
oruninteresting
. -
Command
$ hl example.log -f provider~=string
Displays only messages where the
provider
field contains thestring
sub-string.
-
Command
$ hl my-service.log --query 'level > info or status-code >= 400 or duration > 0.5'
Displays messages that either have a level higher than info (i.e. warning or error) or have a status code field with a numeric value >= 400 or a duration field with a numeric value >= 0.5.
-
Command
$ hl my-service.log -q '(request in (95c72499d9ec, 9697f7aa134f, bc3451d0ad60)) or (method != GET)'
Displays all messages that have the 'request' field with one of these values, or the 'method' field with a value other than 'GET'.
-
Complete set of supported operators
- Logical operators
- Logical conjunction -
and
,&&
- Logical disjunction -
or
,||
- Logical negation -
not
,!
- Logical conjunction -
- Comparison operators
- Equal -
eq
,=
- Not equal -
ne
,!=
- Greater than -
gt
,>
- Greater or equal -
ge
,>=
- Less than -
lt
,<
- Less or equal -
le
,<=
- Equal -
- String matching operators
- Sub-string check - (
contain
,~=
), (not contain
,!~=
) - Wildcard match - (
like
), (not like
)- Wildcard characters are:
*
for zero or more characters and?
for a single character
- Wildcard characters are:
- Regular expression match - (
match
,~~=
), (not match
,!~~=
)
- Sub-string check - (
- Operators with sets
- Test if value is one of the values in a set -
in (v1, v2)
,not in (v1, v2)
- Test if value is one of the values in a set -
- Logical operators
-
Notes
- Special field names that are reserved for filtering by predefined fields regardless of the actual JSON field names used to load the corresponding value:
level
,message
,caller
andlogger
. - To address a JSON field with one of these names instead of predefined fields, add a period before its name, i.e.,
.level
will perform a match against the "level" JSON field. - To address a JSON field by its exact name, use a JSON-formatted string, i.e.
-q '".level" = info'
. - To specify special characters in field values, also use a JSON-formatted string, i.e.
$ hl my-service.log -q 'message contain "Error:\nSomething unexpected happened"'
- Special field names that are reserved for filtering by predefined fields regardless of the actual JSON field names used to load the corresponding value:
-
Command
$ hl example.log --since 'Jun 19 11:22:33' --until yesterday
Displays only messages that occurred after Jun 19 11:22:33 UTC of the current year (or the previous year if the current date is less than Jun 19 11:22:33) and before yesterday midnight.
-
Command
$ hl example.log --since -3d
Displays only messages from the past 72 hours.
-
Command
$ hl example.log --until '2021-06-01 18:00:00' --local
Displays only messages that occurred before 6 PM local time on June 1, 2021, and shows timestamps in local time.
-
Command
$ hl example.log --hide provider
Hides field
provider
. -
Command
$ hl example.log --hide '*' --hide '!provider'
Hides all fields except
provider
. -
Command
$ hl example.log -h headers -h body -h '!headers.content-type'
Hides fields
headers
andbody
but shows a single sub-fieldcontent-type
inside fieldheaders
.
-
Command
$ hl -s *.log
Displays log messages from all log files in the current directory sorted in chronological order.
-
Command
$ hl --sync-interval-ms 500 -F <(kubectl logs -l app=my-app-1 -f) <(kubectl logs -l app=my-app-2 -f)
Runs without a pager in follow mode by merging messages from the outputs of these 2 commands and sorting them chronologically within a custom 500ms interval.
-
Command
$ hl -F --tail 100 app1.log app2.log app3.log
Runs without a pager in follow mode, following the changes in three log files in the current directory and sorting them chronologically at a default interval of 100ms. Preloads 100 lines from the end of each file before filtering.
-
Configuration file is automatically loaded if found in a predefined platform-specific location.
OS Location macOS ~/.config/hl/config.yaml Linux ~/.config/hl/config.yaml Windows %USERPROFILE%\AppData\Roaming\hl\config.yaml -
All parameters in the configuration file are optional and can be omitted. In this case, default values are used.
- Many parameters that are defined in command line arguments and configuration files can also be specified by environment variables.
- Configuration file
- Environment variables
- Command-line arguments
HL_TIME_FORMAT='%y-%m-%d %T.%3N'
overrides time format specified in configuration file.HL_TIME_ZONE=Europe/Berlin
overrides time zone specified in configuration file.HL_CONCURRENCY=4
overrides concurrency limit specified in configuration file.HL_PAGING=never
specified default value for paging option but it may be overridden by command-line arguments.
- Using
theme
value in the configuration file. - Using environment variable, i.e.
HL_THEME=classic
, overrides the value specified in configuration file. - Using command-line argument, i.e.
--theme classic
, overrides all other values.
-
Custom themes are automatically loaded when found in a predefined platform-specific location.
OS Location macOS ~/.config/hl/themes/*.yaml Linux ~/.config/hl/themes/*.yaml Windows %USERPROFILE%\AppData\Roaming\hl\themes*.yaml -
Format description
- Section
elements
contains styles for predefined elements. - Section
levels
contains optional overrides for styles defined inelements
sections per logging level, which are [debug
,info
,warning
,error
]. - Each element style contains optional
background
,foreground
andmodes
parameters. - Example
elements: <element>: foreground: <color> background: <color> modes: [<mode>, <mode>, ...] levels: <level>: <element>: foreground: <color> background: <color> modes: [<mode>, <mode>, ...]
- Color format is one of
- Keyword
default
specifies default color defined by the terminal. - ASCII basic color name, one of
black
red
green
yellow
blue
magenta
cyan
white
bright-black
bright-red
bright-green
bright-yellow
bright-blue
bright-magenta
bright-cyan
bright-white
- 256-color palette code, from
0
to255
. - RGB color in hex web color format, i.e.
#FFFF00
for bright yellow color.
- Keyword
- Modes is a list of additional styles, each of them is one of
bold
faint
italic
underline
slow-blink
rapid-blink
reverse
conceal
crossed-out
- Section
- One Dark Neo
- Built-in "Light Background" color scheme
- One Dark Neo
- Note: It is recommended to use
draw_bold_text_with_bright_colors: true
setting
- Note: It is recommended to use
- Light
- Note: It is recommended to use
draw_bold_text_with_bright_colors: false
setting
- Note: It is recommended to use
JSON log converter to human readable representation
Usage: hl [OPTIONS] [FILE]...
Arguments:
[FILE]... Files to process
Options:
--color <COLOR> Color output options [env: HL_COLOR=] [default: auto] [possible values: auto, always, never]
-c Handful alias for --color=always, overrides --color option
--paging <PAGING> Output paging options [env: HL_PAGING=] [default: auto] [possible values: auto, always, never]
-P Handful alias for --paging=never, overrides --paging option
--theme <THEME> Color theme [env: HL_THEME=] [default: universal]
-r, --raw Output raw JSON messages instead of formatter messages, it can be useful for applying filters and saving results in original format
--no-raw Disable raw JSON messages output, overrides --raw option
--raw-fields Disable unescaping and prettifying of field values
--allow-prefix Allow non-JSON prefixes before JSON messages [env: HL_ALLOW_PREFIX=]
--interrupt-ignore-count <INTERRUPT_IGNORE_COUNT> Number of interrupts to ignore, i.e. Ctrl-C (SIGINT) [env: HL_INTERRUPT_IGNORE_COUNT=] [default: 3]
--buffer-size <BUFFER_SIZE> Buffer size [env: HL_BUFFER_SIZE=] [default: "256 KiB"]
--max-message-size <MAX_MESSAGE_SIZE> Maximum message size [env: HL_MAX_MESSAGE_SIZE=] [default: "64 MiB"]
-C, --concurrency <CONCURRENCY> Number of processing threads [env: HL_CONCURRENCY=]
-f, --filter <FILTER> Filtering by field values in one of forms [k=v, k~=v, k~~=v, 'k!=v', 'k!~=v', 'k!~~=v'] where ~ does substring match and ~~ does regular expression match
-q, --query <QUERY> Custom query, accepts expressions from --filter and supports '(', ')', 'and', 'or', 'not', 'in', 'contain', 'like', '<', '>', '<=', '>=', etc
-h, --hide <HIDE> Hide or unhide fields with the specified keys, prefix with ! to unhide, specify !* to unhide all
-l, --level <LEVEL> Filtering by level [env: HL_LEVEL=]
--since <SINCE> Filtering by timestamp >= the value (--time-zone and --local options are honored)
--until <UNTIL> Filtering by timestamp <= the value (--time-zone and --local options are honored)
-t, --time-format <TIME_FORMAT> Time format, see https://man7.org/linux/man-pages/man1/date.1.html [env: HL_TIME_FORMAT=] [default: "%y-%m-%d %T.%3N"]
-Z, --time-zone <TIME_ZONE> Time zone name, see column "TZ identifier" at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones [env: HL_TIME_ZONE=] [default: UTC]
-L, --local Use local time zone, overrides --time-zone option
--no-local Disable local time zone, overrides --local option
-e, --hide-empty-fields Hide empty fields, applies for null, string, object and array fields only [env: HL_HIDE_EMPTY_FIELDS=]
-E, --show-empty-fields Show empty fields, overrides --hide-empty-fields option [env: HL_SHOW_EMPTY_FIELDS=]
--input-info <INPUT_INFO> Show input number and/or input filename before each message [default: auto] [possible values: auto, none, full, compact, minimal]
--list-themes List available themes and exit
-s, --sort Sort messages chronologically
-F, --follow Follow input streams and sort messages chronologically during time frame set by --sync-interval-ms option
--tail <TAIL> Number of last messages to preload from each file in --follow mode [default: 10]
--sync-interval-ms <SYNC_INTERVAL_MS> Synchronization interval for live streaming mode enabled by --follow option [default: 100]
-o, --output <OUTPUT> Output file
--delimiter <DELIMITER> Log message delimiter, [NUL, CR, LF, CRLF] or any custom string
--dump-index Dump index metadata and exit
--help Print help
-V, --version Print version
- MacBook Pro (16-inch, 2021)
- CPU: Apple M1 Max CPU
- OS: macOS Sonoma 14.2.1
- Data: ~ 2.3 GiB log file, 6 000 000 lines
-
hl v0.25.1 ~ 1.2 seconds
$ time hl example.log -c >/dev/null hl example.log -c > /dev/null 10.61s user 0.53s system 887% cpu 1.256 total
-
hlogf v1.41.1 ~ 8.5 seconds
$ time hlogf example.log --color always >/dev/null hlogf example.log --color always > /dev/null 6.70s user 1.83s system 99% cpu 8.563 total
-
humanlog v0.7.6 ~ 75 seconds
$ time humanlog <example.log --color always >/dev/null humanlog> reading stdin... humanlog --color always < example.log > /dev/null 78.51s user 4.60s system 109% cpu 1:15.65 total
-
fblog v4.8.0 ~ 33 seconds
$ time fblog example.log >/dev/null fblog example.log > /dev/null 31.25s user 2.07s system 99% cpu 33.448 total
-
fblog with
-d
flag v4.8.0 ~ 146 seconds$ time fblog -d example.log >/dev/null fblog -d example.log > /dev/null 130.64s user 14.70s system 99% cpu 2:25.98 total
-