Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use Caffeine instead of Guava Cache #9166

Closed
wants to merge 12 commits into from
Closed

use Caffeine instead of Guava Cache #9166

wants to merge 12 commits into from

Conversation

adrian-wang
Copy link

Fixes #8399 .

Description

Turn to Caffeine instead of Guava's Cache in ResourcePool, OnHeapLoadingCache, and SegmentLoadDropHandler, and interface changes in OffHeapLoadingCache.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths.
  • added integration tests.
  • been tested in a test Druid cluster.

@drcrallen
Copy link
Contributor

Thanks for the PR! One note: catching blanket Exception often breaks InterruptedException intentions, especially if the catch just continues on through the loop. Can you please make sure you are more prescriptive on the exceptions caught?

leventov
leventov reviewed Jan 16, 2020
View reviewed changes
...ed-single/src/main/java/org/apache/druid/server/lookup/cache/loading/OnHeapLoadingCache.java Outdated

private final Cache<K, V> cache;
private final AtomicBoolean isClosed = new AtomicBoolean(false);
@JsonProperty
private final int concurrencyLevel;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure a warning is printed if this legacy JSON property is still specified.

...re/lookups-cached-single/src/test/java/org/apache/druid/server/lookup/LoadingLookupTest.java
@@ -96,28 +95,6 @@ public void testClose()
EasyMock.verify(lookupCache, reverseLookupCache);
}

@Test
public void testApplyWithExecutionError() throws ExecutionException
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could these tests be retained?

...s-cached-single/src/main/java/org/apache/druid/server/lookup/cache/loading/LoadingCache.java

/**
* A semi-persistent mapping from keys to values. Cache entries are added using
* {@link #get(Object, Callable)} and stored in the cache until either evicted or manually invalidated.
* {@link #get(Object, Function)} and stored in the cache until either evicted or manually invalidated.
* <p>
* <p>Implementations of this interface are expected to be thread-safe, and can be safely accessed
* by multiple concurrent threads.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure a warning is printed when legacy "guava" type is specified, and add a new "onHeap" type.

core/src/test/java/org/apache/druid/java/util/common/parsers/TimestampParserTest.java Outdated
@@ -140,6 +140,7 @@ public void testTimeStampParserWithLongTimeZone()

long millis1 = new DateTime(1994, 11, 9, 4, 0, DateTimeZone.forOffsetHours(-8)).getMillis();
long millis2 = new DateTime(1994, 11, 9, 4, 0, DateTimeZone.forOffsetHours(-6)).getMillis();
// java.util.Locale.setDefault(java.util.Locale.US);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove

core/src/test/java/org/apache/druid/java/util/common/parsers/TimestampParserTest.java Outdated
@@ -157,6 +158,7 @@ public void testTimeStampParserWithLongTimeZone()
@Test
public void testTimeZoneAtExtremeLocations()
{
// java.util.Locale.setDefault(java.util.Locale.US);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this

leventov
leventov reviewed Jan 16, 2020
View reviewed changes
Copy link
Member

@leventov leventov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of this change, forbidden-apis entries should also be added.

@leventov
Copy link
Member

Labelling Design Review because the new name for cache property should be vetted (I suggest "onHeap" not to tie to a specific cache implementation, whether it is Guava or Caffeine).

Labelling Release Notes because it entails changes that cluster operators should apply: replace "guava" with "onHeap" (or whatever config name is chosen) and removing the concurrencyLevel property.

@adrian-wang
Copy link
Author

@leventov @drcrallen Thanks for the review, I'm new to druid, for now I just resolved the conflicts. I'll take time to address your comments, thanks!

@clintropolis
Copy link
Member

Looks like there is a legit checkstyle CI failure:

[ERROR] /home/travis/build/apache/druid/extensions-core/lookups-cached-single/src/main/java/org/apache/druid/server/lookup/LoadingLookup.java:33:8: Unused import - java.util.concurrent.Callable. [UnusedImports]

@adrian-wang
Copy link
Author

@leventov I've updated the code, now the failed checks are not relevant. can you please re-trigger the CI?

@ccaominh
Copy link
Contributor

ccaominh commented Feb 7, 2020

@adrian-wang I retriggered the jobs for you

@jihoonson
Copy link
Contributor

@adrian-wang sorry for the delayed review. Would you please fix the conflicts?

@adrian-wang
Copy link
Author

@jihoonson Sorry for the late response, I just merged this with master.

@stale
Copy link

stale bot commented Jun 4, 2020

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

@stale stale bot added the stale label Jun 4, 2020
@adrian-wang
Copy link
Author

gently ping @leventov @jihoonson

@stale
Copy link

stale bot commented Jun 5, 2020

This issue is no longer marked as stale.

@stale stale bot removed the stale label Jun 5, 2020
@stale
Copy link

stale bot commented Aug 8, 2020

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

@stale stale bot added the stale label Aug 8, 2020
@stale
Copy link

stale bot commented Sep 5, 2020

This pull request/issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@stale stale bot closed this Sep 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leventov leventov leventov left review comments

Assignees
No one assigned
Labels
Design Review Improvement Release Notes stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Migrate from Guava's Cache to Caffeine and prohibit Guava's Cache using forbidden-apis
6 participants
@adrian-wang @drcrallen @leventov @clintropolis @ccaominh @jihoonson