LogDNA LogSpout

A Docker LogSpout image to stream logs from your containers to LogDNA.

Change Log

v1.2.0 - Released on May 29, 2019

• Added Tagged Build;
• Added Semantic Versioning;
• Added CHANGELOG.md;
• Updated LICENSE;
• Enriched the Adapter Configuration;
• Added 11 New Environment Variable Options;
• Implemented Retry Mechanism;
• Added Message Sanitization;
• Added Capturing m.Container.State.Pid;
• Changed Buffer Limit from the Length to the Byte Size;
• Polished Some Debug Statements.

v1.1.0 - Released on December 06, 2018

• Getting Tags from Templates

v1.0.0 - Released on August 29, 2018

• Initial Release

How to Use

Environment Variables

The following variables can be used to tune LogSpout for specific use cases.

Log Router Specific

• FILTER_NAME: Filter by Container Name with Wildcards, Optional
• FILTER_ID: Filter by Container ID with Wildcards, Optional
• FILTER_SOURCES: Filter by Comma-Separated List of Sources, Optional
• FILTER_LABELS: Filter by Comma-Separated List of Labels, Optional

Note: More information can be found here.

Ingestion Specific

• LOGDNA_KEY: LogDNA Ingestion Key, Required
• HOSTNAME: Alternative Hostname, Optional
• TAGS: Comma-Separated List of Tags, Optional
• LOGDNA_URL: Specific Endpoint to Stream Log into, Optional
  • Default: logs.logdna.com/logs/ingest
• VERBOSE: Enabling or Disabling to Log LogSpout Container, Optional
  • Default: Enabled
  • Note: 0 to Disable

Note: Logging the LogSpout Container is recommended to keep track of HTTP Request Errors or Exceptions.

Related to HTTP Client

• DIAL_KEEP_ALIVE: The interval (in seconds) between keep-alive probes for an active network connection.
  • Default: 60
  • Source: net/dial.go#Timeout
• DIAL_TIMEOUT: The maximum amount of time (in seconds) a dial will wait for a connect to complete.
  • Default: 60
  • Source: net/dial.go#Timeout
• IDLE_CONN_TIMEOUT: The maximum amount of time (in seconds) an idle (keep-alive) connection will remain idle before closing itself.
  • Default: 60
  • Source: net/http/transport.go#IdleConnTimeout
• HTTP_CLIENT_TIMEOUT: Time limit (in seconds) for requests made by this HTTP Client, Optional
  • Default: 60
  • Source: net/http/client.go#Timeout
• TLS_HANDSHAKE_TIMEOUT: The maximum amount of time (in seconds) allowed to wait for a TLS handshake, Optional
  • Default: 30
  • Source: net/http/transport.go#TLSHandshakeTimeout

Limits

• FLUSH_INTERVAL: How frequently batches of logs are sent (in milliseconds), Optional
  • Default: 250
• INACTIVITY_TIMEOUT: How long to wait for inactivity before declaring failure in the Docker API and restarting, Optional
  • Default: 1m
  • Note: More information about the possible values can be found here. Also see time.ParseDuration for valid format as recommended here.
• MAX_BUFFER_SIZE: The maximum size (in mb) of batches to ship to LogDNA, Optional
  • Default: 2
• MAX_LINE_LENGTH: The maximum character length for each line, Optional
  • Default: 16000
• MAX_REQUEST_RETRY: The maximum number of retries for sending a line when there are network failures, Optional
  • Default: 10

Docker

Create and run container named logdna from this image using CLI:

sudo docker run --name="logdna" --restart=always \
-d -v=/var/run/docker.sock:/var/run/docker.sock \
-e LOGDNA_KEY="<LogDNA Ingestion Key>" \
logdna/logspout:latest

Docker Cloud

Append the following to your Docker Cloud stackfile:

logdna:
  autoredeploy: true
  deployment_strategy: every_node
  environment:
    - LOGDNA_KEY="<LogDNA Ingestion Key>"
    - TAGS='{{.Container.Config.Hostname}}'
  image: 'logdna/logspout:latest'
  restart: always
  volumes:
    - '/var/run/docker.sock:/var/run/docker.sock'

Elastic Container Service (ECS)

Modify your ECS Cloud Configuration file to have LogDNA Service as described below:

services:
  logdna:
    environment:
        - LOGDNA_KEY="<LogDNA Ingestion Key>"
        - TAGS='{{ if .Container.Config.Labels }}{{index .Container.Config.Labels "com.amazonaws.ecs.task-definition-family"}}:{{index .Container.Config.Labels "com.amazonaws.ecs.container-name"}}{{ else }}{{.ContainerName}}{{ end }}'
    image: logdna/logspout:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      mode: global

Rancher

Modify your Rancher Compose Stackfile to have LogDNA Service as described below:

version: '2'
services:
  logdna:
    image: logdna/logspout:latest
    environment:
      LOGDNA_KEY="<LogDNA Ingestion Key>"
    restart: always
    labels:
      io.rancher.container.hostname_override: container_name
      io.rancher.container.pull_image: always
      io.rancher.os.scope: system
    volumes:
    - /var/run/docker.sock:/tmp/docker.sock

Docker Swarm

Modify your Docker Swarm Compose file to have LogDNA Service as described below:

version: "3"
networks:
  logging:
services:
  logdna:
    image: logdna/logspout:latest
    networks:
      - logging
    volumes:
      - /etc/hostname:/etc/host_hostname:ro
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - LOGDNA_KEY="<LogDNA Ingestion Key>"
    deploy:
      mode: global

Notes

Do not forget to add -u root (in CLI) or user: root (in YAML) in case of having permission issues.

Contributing

Contributions are always welcome. See the contributing guide to learn how you can help. Build instructions for the agent are also in the guide.