SQL: Add initial geo support (elastic#42031)

Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2. 

## Queries that are supported as a result of this initial implementation

###  Metadata commands

- `DESCRIBE table`  - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly. 

### Returning geoshapes and geopoints from elasticsearch

- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;

### Using geopoints to elasticsearch

- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes. 
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.

## Limitations:

Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in elastic#37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.

Relates to elastic#29872

docs/reference/sql/functions/geo.asciidoc

@@ -0,0 +1,192 @@
+[role="xpack"]
+[testenv="basic"]
+[[sql-functions-geo]]
+=== Geo Functions
+
+The geo functions work with geometries stored in `geo_point` and `geo_shape` fields, or returned by other geo functions.
+
+==== Limitations
+
+Both <<geo-point, `geo_point`>> and <<geo-shape, `geo_shape`>> types are represented in SQL as geometry and can be used
+interchangeably with the following exceptions:
+
+* `geo_shape` fields don't have doc values, therefore these fields cannot be used for filtering, grouping or sorting.
+
+* `geo_points` fields are indexed and have doc values by default, however only latitude and longitude are stored and
+  indexed with some loss of precision from the original values (4.190951585769653E-8 for the latitude and
+  8.381903171539307E-8 for longitude). The altitude component is accepted but not stored in doc values nor indexed.
+  Therefore calling `ST_Z` function in the filtering, grouping or sorting will return `null`.
+
+==== Geometry Conversion
+
+[[sql-functions-geo-st-as-wkt]]
+===== `ST_AsWKT`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_AsWKT(geometry<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> geometry
+
+*Output*: string
+
+.Description:
+
+Returns the WKT representation of the `geometry`.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[aswkt]
+--------------------------------------------------
+
+
+[[sql-functions-geo-st-wkt-to-sql]]
+===== `ST_WKTToSQL`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_WKTToSQL(string<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> string WKT representation of geometry
+
+*Output*: geometry
+
+.Description:
+
+Returns the geometry from WKT representation.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[aswkt]
+--------------------------------------------------
+
+==== Geometry Properties
+
+[[sql-functions-geo-st-geometrytype]]
+===== `ST_GeometryType`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_GeometryType(geometry<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> geometry
+
+*Output*: string
+
+.Description:
+
+Returns the type of the `geometry` such as POINT, MULTIPOINT, LINESTRING, MULTILINESTRING, POLYGON, MULTIPOLYGON, GEOMETRYCOLLECTION, ENVELOPE or CIRCLE.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[geometrytype]
+--------------------------------------------------
+
+[[sql-functions-geo-st-x]]
+===== `ST_X`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_X(geometry<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> geometry
+
+*Output*: double
+
+.Description:
+
+Returns the longitude of the first point in the geometry.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[x]
+--------------------------------------------------
+
+[[sql-functions-geo-st-y]]
+===== `ST_Y`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_Y(geometry<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> geometry
+
+*Output*: double
+
+.Description:
+
+Returns the the latitude of the first point in the geometry.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[y]
+--------------------------------------------------
+
+[[sql-functions-geo-st-z]]
+===== `ST_Z`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_Z(geometry<1>)
+--------------------------------------------------
+
+*Input*:
+
+<1> geometry
+
+*Output*: double
+
+.Description:
+
+Returns the altitude of the first point in the geometry.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[z]
+--------------------------------------------------
+
+[[sql-functions-geo-st-distance]]
+===== `ST_Distance`
+
+.Synopsis:
+[source, sql]
+--------------------------------------------------
+ST_Distance(geometry<1>, geometry<2>)
+--------------------------------------------------
+
+*Input*:
+
+<1> source geometry
+<2> target geometry
+
+*Output*: Double
+
+.Description:
+
+Returns the distance between geometries in meters. Both geometries have to be points.
+
+["source","sql",subs="attributes,macros"]
+--------------------------------------------------
+include-tagged::{sql-specs}/docs/geo.csv-spec[distance]
+--------------------------------------------------

docs/reference/sql/functions/index.asciidoc

@@ -136,6 +136,14 @@
 ** <<sql-functions-conditional-least>>
 ** <<sql-functions-conditional-nullif>>
 ** <<sql-functions-conditional-nvl>>
+* <<sql-functions-geo>>
+** <<sql-functions-geo-st-as-wkt>>
+** <<sql-functions-geo-st-distance>>
+** <<sql-functions-geo-st-geometrytype>>
+** <<sql-functions-geo-st-wkt-to-sql>>
+** <<sql-functions-geo-st-x>>
+** <<sql-functions-geo-st-y>>
+** <<sql-functions-geo-st-z>>
 * <<sql-functions-system>>
 ** <<sql-functions-system-database>>
 ** <<sql-functions-system-user>>
@@ -149,5 +157,6 @@ include::search.asciidoc[]
 include::math.asciidoc[]
 include::string.asciidoc[]
 include::type-conversion.asciidoc[]
+include::geo.asciidoc[]
 include::conditional.asciidoc[]
 include::system.asciidoc[]

docs/reference/sql/language/data-types.asciidoc

@@ -81,6 +81,8 @@ s|SQL precision
 | interval_hour_to_minute   | 23
 | interval_hour_to_second   | 23
 | interval_minute_to_second | 23
+| geo_point                 | 52
+| geo_shape                 | 2,147,483,647
 
 |===
 

docs/reference/sql/limitations.asciidoc

@@ -150,3 +150,14 @@ SELECT count(*) FROM test GROUP BY MINUTE((CAST(date_created AS TIME));
 -------------------------------------------------------------
 SELECT HISTOGRAM(CAST(birth_date AS TIME), INTERVAL '10' MINUTES) as h, COUNT(*) FROM t GROUP BY h
 -------------------------------------------------------------
+
+[float]
+[[geo-sql-limitations]]
+=== Geo-related functions
+
+Since `geo_shape` fields don't have doc values these fields cannot be used for filtering, grouping or sorting.
+
+By default,`geo_points` fields are indexed and have doc values. However only latitude and longitude are stored and
+indexed with some loss of precision from the original values (4.190951585769653E-8 for the latitude and
+8.381903171539307E-8 for longitude). The altitude component is accepted but not stored in doc values nor indexed.
+Therefore calling `ST_Z` function in the filtering, grouping or sorting will return `null`.

server/src/main/java/org/elasticsearch/common/geo/parsers/ShapeParser.java

@@ -20,12 +20,18 @@
 
 import org.elasticsearch.ElasticsearchParseException;
 import org.elasticsearch.common.ParseField;
+import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.geo.builders.ShapeBuilder;
+import org.elasticsearch.common.xcontent.LoggingDeprecationHandler;
+import org.elasticsearch.common.xcontent.NamedXContentRegistry;
 import org.elasticsearch.common.xcontent.XContent;
+import org.elasticsearch.common.xcontent.XContentBuilder;
 import org.elasticsearch.common.xcontent.XContentParser;
+import org.elasticsearch.common.xcontent.json.JsonXContent;
 import org.elasticsearch.index.mapper.BaseGeoShapeFieldMapper;
 
 import java.io.IOException;
+import java.io.InputStream;
 
 /**
  * first point of entry for a shape parser
@@ -67,4 +73,20 @@ static ShapeBuilder parse(XContentParser parser, BaseGeoShapeFieldMapper shapeMa
     static ShapeBuilder parse(XContentParser parser) throws IOException {
         return parse(parser, null);
     }
+
+    static ShapeBuilder parse(Object value) throws IOException {
+        XContentBuilder content = JsonXContent.contentBuilder();
+        content.startObject();
+        content.field("value", value);
+        content.endObject();
+
+        try (InputStream stream = BytesReference.bytes(content).streamInput();
+             XContentParser parser = JsonXContent.jsonXContent.createParser(
+                 NamedXContentRegistry.EMPTY, LoggingDeprecationHandler.INSTANCE, stream)) {
+            parser.nextToken(); // start object
+            parser.nextToken(); // field name
+            parser.nextToken(); // field value
+            return parse(parser);
+        }
+    }
 }

x-pack/plugin/sql/build.gradle

@@ -16,6 +16,7 @@ ext {
     // SQL test dependency versions
     csvjdbcVersion="1.0.34"
     h2Version="1.4.197"
+    h2gisVersion="1.5.0"
 }
 
 configurations {

x-pack/plugin/sql/jdbc/build.gradle

@@ -21,6 +21,9 @@ dependencies {
     compile (project(':libs:x-content')) {
         transitive = false
     }
+    compile (project(':libs:elasticsearch-geo')) {
+        transitive = false
+    }
     compile project(':libs:core')
     runtime "com.fasterxml.jackson.core:jackson-core:${versions.jackson}"
     testCompile "org.elasticsearch.test:framework:${version}"

x-pack/plugin/sql/jdbc/src/main/java/org/elasticsearch/xpack/sql/jdbc/EsType.java

@@ -44,7 +44,9 @@ public enum EsType implements SQLType {
     INTERVAL_DAY_TO_SECOND(   ExtraTypes.INTERVAL_DAY_SECOND),
     INTERVAL_HOUR_TO_MINUTE(  ExtraTypes.INTERVAL_HOUR_MINUTE),
     INTERVAL_HOUR_TO_SECOND(  ExtraTypes.INTERVAL_HOUR_SECOND),
-    INTERVAL_MINUTE_TO_SECOND(ExtraTypes.INTERVAL_MINUTE_SECOND);
+    INTERVAL_MINUTE_TO_SECOND(ExtraTypes.INTERVAL_MINUTE_SECOND),
+    GEO_POINT(                ExtraTypes.GEOMETRY),
+    GEO_SHAPE(                ExtraTypes.GEOMETRY);
 
     private final Integer type;
 

x-pack/plugin/sql/jdbc/src/main/java/org/elasticsearch/xpack/sql/jdbc/ExtraTypes.java

@@ -29,5 +29,6 @@ private ExtraTypes() {}
     static final int INTERVAL_HOUR_MINUTE = 111;
     static final int INTERVAL_HOUR_SECOND = 112;
     static final int INTERVAL_MINUTE_SECOND = 113;
+    static final int GEOMETRY = 114;
 
 }

x-pack/plugin/sql/jdbc/src/main/java/org/elasticsearch/xpack/sql/jdbc/JdbcColumnInfo.java

@@ -3,6 +3,7 @@
  * or more contributor license agreements. Licensed under the Elastic License;
  * you may not use this file except in compliance with the Elastic License.
  */
+
 package org.elasticsearch.xpack.sql.jdbc;
 
 import java.util.Objects;
@@ -89,4 +90,4 @@ public boolean equals(Object obj) {
     public int hashCode() {
         return Objects.hash(name, type, table, catalog, schema, label, displaySize);
     }
-}
+}

x-pack/plugin/sql/jdbc/src/main/java/org/elasticsearch/xpack/sql/jdbc/JdbcConfiguration.java

@@ -35,7 +35,7 @@
  / Additional properties can be specified either through the Properties object or in the URL. In case of duplicates, the URL wins.
  */
 //TODO: beef this up for Security/SSL
-class JdbcConfiguration extends ConnectionConfiguration {
+public class JdbcConfiguration extends ConnectionConfiguration {
     static final String URL_PREFIX = "jdbc:es://";
     public static URI DEFAULT_URI = URI.create("http://localhost:9200/");
 
@@ -47,7 +47,7 @@ class JdbcConfiguration extends ConnectionConfiguration {
     // can be out/err/url
     static final String DEBUG_OUTPUT_DEFAULT = "err";
 
-    static final String TIME_ZONE = "timezone";
+    public static final String TIME_ZONE = "timezone";
     // follow the JDBC spec and use the JVM default...
     // to avoid inconsistency, the default is picked up once at startup and reused across connections
     // to cater to the principle of least surprise