Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Java agent to use new version of log4j 2 #605

Closed
6 tasks done
kford-newrelic opened this issue Dec 10, 2021 · 20 comments · Fixed by #603
Closed
6 tasks done

Update Java agent to use new version of log4j 2 #605

kford-newrelic opened this issue Dec 10, 2021 · 20 comments · Fixed by #603
Assignees
Labels
GTSE There is an associated support escalation with this issue.

Comments

@kford-newrelic
Copy link
Contributor

kford-newrelic commented Dec 10, 2021

Is your feature request related to a problem? Please describe.

A well-publicized vulnerability has been discovered with certain versions of the log4j 2 framework. Some references:

Feature Description

Need to publish updated maint releases for the following major agent versions (these are still under support):

  • Java Agent 7.4.1
  • Java Agent 6.5.1
  • Java Agent 7.4.2
  • Java Agent 6.5.2
  • Java Agent 7.4.3
  • Java Agent 6.5.3

Describe Alternatives

A workaround to the issue has been described, to disable logging by setting the log level to off.
See security bulletin NR21-03 for the latest mitigation actions.

Additional context

Older versions of the Java Agent that are not currently supported will not be updated, in alignment with our published EOL policy.

Priority

Critical

@kford-newrelic kford-newrelic added the GTSE There is an associated support escalation with this issue. label Dec 10, 2021
@meiao meiao linked a pull request Dec 10, 2021 that will close this issue
@edwardlee1
Copy link

By turning the logging level to 'off' will it temporarily remediate the vulnerability?

@aSapien
Copy link

aSapien commented Dec 12, 2021

Java Agent 5.x should also still be supported and receive security patches. Is this fix being backported to older agents?

@Firefishy
Copy link

The updated Java Agent 6.5.1 breaks JDK7 compatibility.

@JamesUoM
Copy link

The updated Java Agent 6.5.1 breaks JDK7 compatibility.

6.5.0 states it's the last version to support JDK7 but presumably the implication was from 7.x onwards. We unfortunately do need JDK7 support.

@Firefishy
Copy link

The updated Java Agent 6.5.1 breaks JDK7 compatibility.

6.5.0 states it's the last version to support JDK7 but presumably the implication was from 7.x onwards. We unfortunately do need JDK7 support.

https://discuss.newrelic.com/t/log4j-zero-day-vulnerability-and-the-new-relic-java-agent/170322 says use 6.5.1 for JDK7 compatibility, but it appear v6.5.1 is NOT compatible with JDK7 because log4j 2.15.0 is not compatible with JDK7.

@tbradellis
Copy link
Contributor

tbradellis commented Dec 13, 2021

The updated Java Agent 6.5.1 breaks JDK7 compatibility.

We are looking at how to best address this in the agent. For immediate remediation the thing to do will be to use the workaround recommendations indicated by log4j in conjunction with Agent 6.5.0 or other appropriate agent version that will run on Java 7:

https://logging.apache.org/log4j/2.x/security.html

Mitigation: In releases >=2.10, this behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true. For releases >=2.7 and <=2.14.1, all PatternLayout patterns can be modified to specify the message converter as %m{nolookups} instead of just %m. For releases >=2.0-beta9 and <=2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.
Credit: This issue was discovered by Chen Zhaojun of Alibaba Cloud Security Team.

@Stephan202
Copy link
Contributor

@tbradellis I suppose the New Relic Agent build could be updated to exclude JndiLookup.class from the final artifact.

In any case, New Relic isn't the only one with this issue: apache/logging-log4j2#608 (comment). Let's see what the response will be 👀.

@tbradellis
Copy link
Contributor

Looks like we have a good path for the Java 7 supported releases. We'll do a 6.5.x point release with the backported patch 2.12.2.
https://github.com/apache/logging-log4j2/commits/log4j-2.12

@kford-newrelic
Copy link
Contributor Author

Updating agent release to 7.4.2, to leverage version 2.16.0 of log4j

Will update agent release 6.5.2 when an updated log4j 2.12.2 is available (to support Java 7 customers)

@jinzishuai
Copy link

Any ETA for 6.5.2 ? Thanks.

@yunusevren
Copy link

yunusevren commented Dec 15, 2021

A second CVE was found in log4j 2.15.0. I see that it is already fixed for 7.4.2 (#610) and log4j version is bumped to 2.16.0. Could this also be fixed in a new 6.5.x version?

@jasonjkeller
Copy link
Contributor

@jinzishuai @yunusevren We plan to do another 6.x point release but we're waiting for the Log4j folks to release the back-ported fix to a version of Log4j that supports Java 7, as the 6.x range of agents should remain Java 7 compatible. Unfortunately we broke that backwards compatibility with the 6.5.1 patch but we'd like to remedy that now that we know that a back-port should happen.

As summarized in this comment from @benders we don't believe the agent to be vulnerable to CVE-2021-45046:

Speaking for New Relic: Our analysis does not show the Java Agent to be vulnerable to CVE-2021-45046, which is harder to exploit than the original issue. Any application running Agent 6.5.1 or 7.4.1, or using -Dlog4j2.formatMsgNoLookups=true should still be protected. Because of this, we feel comfortable waiting for the (hopefully imminent) Java 7 compatible backport for our 6.x series.

@jasonjkeller
Copy link
Contributor

jasonjkeller commented Dec 16, 2021

All, Java agent 6.5.2 has been released. This provides a Log4j security patch that is back-ported for compatibility with Java 7.

Changed log4j version to 2.12.2 to mitigate the security vulnerability CVE-2021-45046.

You can find it on maven central:
https://repo1.maven.org/maven2/com/newrelic/agent/java/newrelic-java/6.5.2/

@dylanmei
Copy link

Can something be done to remove JNDI as a threat vector entirely?

$ unzip -l newrelic-agent-7.4.2.jar | grep JndiLookup
     3354  2021-12-09 11:25   com/newrelic/agent/deps/org/apache/logging/log4j/core/lookup/JndiLookup.class

@hpoettker
Copy link

Another CVE has been published: https://logging.apache.org/log4j/2.x/security.html#CVE-2021-45105
A fix has been released with log4j 2.17.0. Will there be an according release for the agent?

@skjelmo
Copy link
Contributor

skjelmo commented Dec 18, 2021

The described alternative, to set the log4j2.formatMsgNoLookups system property to true, seems to have been discredited by Apache as a mitigation measure (See "Older (discredited) mitigation measures").

@meiao
Copy link
Contributor

meiao commented Dec 20, 2021

@skjelmo we've updated our mitigation actions, but had forgotten to update this ticket. This has been fixed.

@meiao
Copy link
Contributor

meiao commented Dec 20, 2021

@hpoettker agent 7.4.3 was released this morning with log4j 2.17.0.

@kford-newrelic
Copy link
Contributor Author

@skjelmo from the information we have on CVE-2021-45105 and our analysis of the code, NR agents are NOT affected by this vulnerability.

@kford-newrelic
Copy link
Contributor Author

Waiting to see if the Apache project releases a log4j 2.12.3 - if so, we'll issue an agent release 6.5.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GTSE There is an associated support escalation with this issue.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.