Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure Joni warning are logged at debug #57302

Merged
merged 7 commits into from
Jun 9, 2020
Merged

Ensure Joni warning are logged at debug #57302

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2f64eb7
Ensure Joni warning are logged at debug
jakelandis May 28, 2020
fa0493a
Revert "Ensure Joni warning are logged at debug"
jakelandis May 31, 2020
4068dc4
use callback instead of direct log4j dependency
jakelandis May 31, 2020
5643762
add on construction warning
jakelandis May 31, 2020
eea0231
fix import
jakelandis May 31, 2020
2b2f269
remove compile dependency on log4jApi and remove license
jakelandis Jun 8, 2020
9e800e4
Merge branch 'master' into route_joni_warnings_to_debug_log
elasticmachine Jun 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "Ensure Joni warning are logged at debug" 
This reverts commit 2f64eb7.
  • Loading branch information
@jakelandis
jakelandis committed May 31, 2020
commit fa0493a9e29ba7a57086afec68ffe7396039612f
21 changes: 0 additions & 21 deletions docs/reference/ingest/processors/grok.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,24 +321,3 @@ settings:
| `ingest.grok.watchdog.interval` | 1s | How often to check whether there are grok evaluations that take longer than the maximum allowed execution time.
| `ingest.grok.watchdog.max_execution_time` | 1s | The maximum allowed execution of a grok expression evaluation.
|======

[[grok-debugging]]
==== Grok debugging

It is advised to use the {kibana-ref}/xpack-grokdebugger.html[Grok Debugger] to debug grok patterns. From there you can test one or more
patterns in the UI against sample data. Under the covers it uses the same engine as ingest node processor. The Grok Debugger is
an {xpack} feature under the Basic License and is therefore *free to use*.

Additionally, it is recommended to enable debug logging for Grok so that any additional messages may also be seen in the Elasticsearch
server log.

[source,js]
--------------------------------------------------
PUT _cluster/settings
{
"transient": {
"logger.org.elasticsearch.grok.Grok" : "debug"
}
}
--------------------------------------------------
// NOTCONSOLE
15 changes: 0 additions & 15 deletions libs/grok/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
*/

dependencies {
compile "org.apache.logging.log4j:log4j-api:${versions.log4j}"
compile 'org.jruby.joni:joni:2.1.29'
// joni dependencies:
compile 'org.jruby.jcodings:jcodings:1.0.44'
Expand All @@ -31,17 +30,3 @@ dependencies {
tasks.named('forbiddenApisMain').configure {
replaceSignatureFiles 'jdk-signatures'
}


thirdPartyAudit.ignoreMissingClasses(
// from log4j
'org.osgi.framework.AdaptPermission',
'org.osgi.framework.AdminPermission',
'org.osgi.framework.Bundle',
'org.osgi.framework.BundleActivator',
'org.osgi.framework.BundleContext',
'org.osgi.framework.BundleEvent',
'org.osgi.framework.SynchronousBundleListener',
'org.osgi.framework.wiring.BundleWire',
'org.osgi.framework.wiring.BundleWiring'
)
1 change: 0 additions & 1 deletion libs/grok/licenses/log4j-api-2.11.1.jar.sha1
View file
Open in desktop

This file was deleted.

202 changes: 0 additions & 202 deletions libs/grok/licenses/log4j-api-LICENSE.txt
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions libs/grok/licenses/log4j-api-NOTICE.txt
View file
Open in desktop

This file was deleted.

10 changes: 2 additions & 8 deletions libs/grok/src/main/java/org/elasticsearch/grok/Grok.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@

package org.elasticsearch.grok;

import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.jcodings.specific.UTF8Encoding;
import org.joni.Matcher;
import org.joni.NameEntry;
Expand All @@ -46,7 +43,6 @@

public final class Grok {

private static final Logger logger = LogManager.getLogger(Grok.class);
private static final String NAME_GROUP = "name";
private static final String SUBNAME_GROUP = "subname";
private static final String PATTERN_GROUP = "pattern";
Expand All @@ -62,8 +58,7 @@ public final class Grok {
")" +
")?" + "\\}";
private static final Regex GROK_PATTERN_REGEX = new Regex(GROK_PATTERN.getBytes(StandardCharsets.UTF_8), 0,
GROK_PATTERN.getBytes(StandardCharsets.UTF_8).length, Option.NONE, UTF8Encoding.INSTANCE, Syntax.DEFAULT,
message -> logger.log(Level.DEBUG, message));
GROK_PATTERN.getBytes(StandardCharsets.UTF_8).length, Option.NONE, UTF8Encoding.INSTANCE, Syntax.DEFAULT);

private static final Map<String, String> builtinPatterns;
private static final int MAX_TO_REGEX_ITERATIONS = 100_000; //sanity limit
Expand Down Expand Up @@ -106,8 +101,7 @@ private Grok(Map<String, String> patternBank, String grokPattern, boolean namedC

String expression = toRegex(grokPattern);
byte[] expressionBytes = expression.getBytes(StandardCharsets.UTF_8);
this.compiledExpression = new Regex(expressionBytes, 0, expressionBytes.length, Option.DEFAULT, UTF8Encoding.INSTANCE,
message -> logger.log(Level.DEBUG, message));
this.compiledExpression = new Regex(expressionBytes, 0, expressionBytes.length, Option.DEFAULT, UTF8Encoding.INSTANCE);
}

/**
Expand Down