Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scripting: Cache script results if deterministic #50106

Merged
Merged
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
cf07b27
Notes
stu-elastic Dec 6, 2019
593da60
Wiring from whitelist annotations to PainlessMethod and PainlessConst…
stu-elastic Dec 6, 2019
d1ae37f
Generate isResultDeterministic
stu-elastic Dec 6, 2019
45e3aed
formatting, parseInt is deterministic
stu-elastic Dec 6, 2019
bbc74fe
Merge branch 'master' of github.com:elastic/elasticsearch into featur…
stu-elastic Dec 9, 2019
db77cde
WIP
stu-elastic Dec 10, 2019
16a2f44
update whitelist based on core-infra discussion, map/set iterators ar…
stu-elastic Dec 11, 2019
e13698c
Merge branch 'master' of github.com:elastic/elasticsearch into featur…
stu-elastic Dec 11, 2019
91883d3
Revert LinkedHashMap
stu-elastic Dec 11, 2019
a5468ac
MaxAggregatorTests.testDontCacheScripts -> testCacheScripts
stu-elastic Dec 11, 2019
5b14a33
remove stale TODOs
stu-elastic Dec 11, 2019
25220c4
handle imported and static methods
stu-elastic Dec 11, 2019
da28b3c
Merge branch 'master' of github.com:elastic/elasticsearch into featur…
stu-elastic Dec 12, 2019
39337fb
setDeterministic -> markNonDeterministic
stu-elastic Dec 12, 2019
fec15a6
MultiValuesSourceFieldConfig, use {read,write}OptionalString for fiel…
stu-elastic Dec 12, 2019
91eaf70
Merge branch 'master' of github.com:elastic/elasticsearch into featur…
stu-elastic Dec 13, 2019
91062b4
this.deterministic &= deterministic -> this.deterministic &= !nondete…
stu-elastic Dec 13, 2019
681d271
guard writeOptional against pre-v7.6.0
stu-elastic Dec 13, 2019
c5ccb6f
disable bwc testing until 7.6 backport
stu-elastic Dec 13, 2019
e26999e
add pr link
stu-elastic Dec 13, 2019
b11dd3a
System.nanoTime/currentTimeMillis non, Collections.shuffle with rand …
stu-elastic Dec 13, 2019
2d45bed
Update script cache testing in HDRPercentilesIT, TDigestPercentileRan…
stu-elastic Dec 13, 2019
0b58b9d
Add MockDeterministicScript and use it in MockScriptEngine.
stu-elastic Dec 16, 2019
dd2c3c3
Update DateRangeIT
stu-elastic Dec 16, 2019
9020cdd
Update DoubleTermsIT
stu-elastic Dec 16, 2019
b0fc18b
Update HistogramIT
stu-elastic Dec 16, 2019
81d6df9
Update LongTermsIT
stu-elastic Dec 16, 2019
fcb6ac6
Update RangeIT
stu-elastic Dec 16, 2019
79641f7
Update SignificantTermsSignificanceScoreIT
stu-elastic Dec 16, 2019
edb16ba
Update StringTermsIT
stu-elastic Dec 16, 2019
a9c9256
Update AvgAggregatorTests
stu-elastic Dec 16, 2019
4b9c283
CardinalityIT
stu-elastic Dec 16, 2019
528a6a4
Update ExtendedStatsIT
stu-elastic Dec 17, 2019
53be9e4
Update HDRPercentileRanksIT
stu-elastic Dec 17, 2019
3306461
Update HDRPercentilesIT
stu-elastic Dec 17, 2019
e464221
Update MaxAggregatorTests MedianAbsoluteDeviationIT
stu-elastic Dec 17, 2019
cf5585f
Update MinIT
stu-elastic Dec 17, 2019
b2a113d
Update ScriptedMetricIT
stu-elastic Dec 17, 2019
f21c252
Update StatsIT
stu-elastic Dec 17, 2019
7acb0fe
Update SumIT
stu-elastic Dec 17, 2019
45a9f07
Update TDigestPercentileRanksIT
stu-elastic Dec 17, 2019
872bed4
Update TDigestPercentilesIT
stu-elastic Dec 17, 2019
e0ed60a
Update TopHitsIT
stu-elastic Dec 17, 2019
40292e0
Update ValueCountIT
stu-elastic Dec 17, 2019
af9df68
MinIT merged into MinAggregatorTests
stu-elastic Dec 17, 2019
04e5530
Update MinAggregatorTests
stu-elastic Dec 17, 2019
a1a572e
Use reproducable random methods instead of Math.random
stu-elastic Dec 17, 2019
e7c1171
Remove unnecessary publics, rearrange imports, correct indent
stu-elastic Dec 17, 2019
b808b39
Mark Calendar and Clock methods nondeterministic
stu-elastic Dec 17, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update StatsIT
  • Loading branch information
stu-elastic committed Dec 17, 2019
commit f21c252448da81e77037bfb00b4b27261564a920
Original file line number Diff line number Diff line change
Expand Up @@ -488,10 +488,10 @@ private void assertShardExecutionState(SearchResponse response, int expectedFail
}

/**
* Make sure that a request using a script does not get cached and a request
* not using a script does get cached.
* Make sure that a request using a deterministic script or not using a script get cached.
* Ensure requests using nondeterministic scripts do not get cached.
*/
public void testDontCacheScripts() throws Exception {
public void testScriptCaching() throws Exception {
assertAcked(prepareCreate("cache_test_idx").addMapping("type", "d", "type=long")
.setSettings(Settings.builder().put("requests.cache.enable", true).put("number_of_shards", 1).put("number_of_replicas", 1))
.get());
Expand All @@ -504,25 +504,35 @@ public void testDontCacheScripts() throws Exception {
assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getMissCount(), equalTo(0L));

// Test that a request using a script does not get cached
// Test that a request using a nondeterministic script does not get cached
SearchResponse r = client().prepareSearch("cache_test_idx").setSize(0).addAggregation(
stats("foo").field("d").script(
new Script(ScriptType.INLINE, AggregationTestScriptsPlugin.NAME, "_value + 1", Collections.emptyMap()))).get();
new Script(ScriptType.INLINE, AggregationTestScriptsPlugin.NAME, "Math.random()", Collections.emptyMap()))).get();
assertSearchResponse(r);

assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getHitCount(), equalTo(0L));
assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getMissCount(), equalTo(0L));

// To make sure that the cache is working test that a request not using
// a script is cached
r = client().prepareSearch("cache_test_idx").setSize(0).addAggregation(stats("foo").field("d")).get();
// Test that a request using a deterministic script gets cached
r = client().prepareSearch("cache_test_idx").setSize(0).addAggregation(
stats("foo").field("d").script(
new Script(ScriptType.INLINE, AggregationTestScriptsPlugin.NAME, "_value + 1", Collections.emptyMap()))).get();
assertSearchResponse(r);

assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getHitCount(), equalTo(0L));
assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getMissCount(), equalTo(1L));

// Ensure that non-scripted requests are cached as normal
r = client().prepareSearch("cache_test_idx").setSize(0).addAggregation(stats("foo").field("d")).get();
assertSearchResponse(r);

assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getHitCount(), equalTo(0L));
assertThat(client().admin().indices().prepareStats("cache_test_idx").setRequestCache(true).get().getTotal().getRequestCache()
.getMissCount(), equalTo(2L));
}
}