Specify template for elasticsearch to get . notation working

[Marcussk]

Hello I have been trying to resolve problem as specified in #33.
The problem is:
I have tags which consist from "." separated fields and i want to be able to filter messages according to this field.
More specifically I have these tags:
host1.server.sender
host1.server.receiver
host2...

What I want to achieve is to be able to search for host1.server.* or host1.* or *.server (wildcard search)

My solution was to put template into elasticsearch as follows:

{
  "template" : "logstash-*",
  "settings" : {
    "index.number_of_shards" : "1",
    "index.number_of_replicas" : "1"
  },
  "mappings" : {
    "_default_" : {
      "properties" : {
        "@app" : {"type" : "string","index" : "analyzed"},
        "@severity" : { "type" : "string", "index" : "analyzed" },
        "@timestamp" : { "type" : "date", "index" : "not_analyzed" },
        "@thread" : { "type" : "string", "index" : "analyzed" },
        "@tag" : { "type" : "string", "index" : "not_analyzed" },
        "@message" : { "type" : "string", "index" : "not_analyzed" }
      }
    }
  }
}

When checking index pattern in Kibana i see that tag field and others are correctly unchecked -> not_analyzed.

Searching for tag:"application.*.server" (simplified) returns no results. I am not entirely sure whether my template is wrong or whether something else is wrong

[pitr]

Is your field @tag or tag?

[Marcussk]

Hello, it is tag:
Example message:

{
  "_index": "logstash-2016.07.11",
  "_type": "fluentd",
  "_id": "AVXbk1yOSYqKcxJrYmvI",
  "_score": null,
  "_source": {
    "app": "RECEIVER",
    "thread": "140290763708160",
    "severity": " MSG ",
    "message": "Message :<?xml version=\"1.0\" encoding=\"UTF-8\"?>",
    "tag": "beeeon.ant-2.ada_server",
    "@timestamp": "2016-07-11T22:07:56+02:00"
  },
  "fields": {
    "@timestamp": [
      1468267676000
    ]
  },
  "sort": [
    1468267676000
  ]
}

[pitr]

right, so mapping should be "tag" : { "type" : "string", "index" : "not_analyzed" }, not @tag

[pitr]

same for other fields, except for @timestamp

[Marcussk]

curl -XGET 'http://localhost:9200/_template/?pretty'
{
  "simple-template" : {
    "order" : 0,
    "template" : "logstash-*",
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    },
    "mappings" : {
      "_default_" : {
        "properties" : {
          "app" : {
            "index" : "analyzed",
            "type" : "string"
          },
          "severity" : {
            "index" : "analyzed",
            "type" : "string"
          },
          "@timestamp" : {
            "index" : "not_analyzed",
            "type" : "date"
          },
          "thread" : {
            "index" : "analyzed",
            "type" : "string"
          },
          "tag" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "message" : {
            "index" : "not_analyzed",
            "type" : "string"
          }
        }
      }
    },
    "aliases" : { }
  }
}

I checked server and this is current mapping. No success.

[pitr]

Don't know how else to help here, it seems that logs are indexed properly.

[cosmo0920]

No response and seems to be working properly.
Closing.