[Backport release-22.05] python310Packages.nbconvert: use mistune 2.x

[github-actions]

Bot-based backport to release-22.05, triggered by a label in #186804.

• Before merging, ensure that this backport complies with the Criteria for Backporting.
  • Even as a non-commiter, if you find that it does not comply, leave a comment.

[dotlambda]

Need to use mistune_2_0.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/jupyter-broken-in-22-05/21108/4

[FRidh]

This requires mistune_2_0 to become the package set default. In principle we don't do such upgrades.

To clarify. The package is publicly available in the package set. Changing major version likely changes the API causing breakage for not only our packages that depend on it (which could be fixable) but also for third-party users that use the package directly. That is, they have to deal with an unexpected change in behavior.

This package probably should have been upgraded before the 22.05 release, judging from the history of releases of of the package.

Now, if we deem the CVE to be so important and it has a massive impact on the package set and it is considered unlikely and acceptable that direct users of mistune 0.8.4 have their packages break because they're relying on this old version, we could do the upgrade.

[dotlambda]

We don't have to make mistune_2_0 the default, we can just use it for nbconvert.

[FRidh]

No, we cannot use multiple versions of a package in the package set.

[dotlambda]

No, we cannot use multiple versions of a package in the package set.

Fact is that we're already doing so in the case of mistune since @SuperSandro2000 merged #117917.

[FRidh]

No, we cannot use multiple versions of a package in the package set.

Fact is that we already do so in the case of mistune since @SuperSandro2000 merged #117917.

Then that's an error. It does not make it an excuse to make the same mistake elsewhere.

[sersorrel]

On nixos-22.05 (currently 00e376e), I count:

• 8 packages using mistune (mirage, giara, nbconvert, m2r, lektor, schema-salad, mrkd, karton-dashboard) (plus the version of m2r from github rather than pypi)
• 3 packages using mistune_2_0 (present, md2gemini, hyperkitty)

(0 packages use mistune_0_8 directly)

It seems like 2.0 is a significant change lepture/mistune#290, unfortunately. Best I can tell, the vulnerability is a ReDoS, and hence probably not severe enough to justify the breakage of upgrading mistune for everyone (though you could make the argument that with a CVSS of 9.8, it's upgrade-the-world-right-now serious! how the hell anyone labelled a ReDoS as having confidentiality impact, I will never know).

Though frankly I'm not sure whether mistune 0.8.4 is actually vulnerable in the first place; the regex that was changed in the fix lepture/mistune@a6d4321 was added after v0.8.4's release (in lepture/mistune@ca1e7b5). I am not able to tell with confidence whether the original regex is also vulnerable.

[sersorrel]

ReDoS Checker detects the exponential behaviour in the vulnerable ASTERISK_EMPHASIS regex (\*{1,2})(?=[^\s*])((?:\\[\\*]|[^*])*(?:\\[\\!"#$%&'()*+,./:;<=>?@\[\]^`{}|_~-]|[^\s*]))\1, and detects only polynomial behaviour in the replacement regex (\*{1,2})(?=[^\s*])((?:(?:(?<!\\)(?:\\\\)*\*)|[^*])+)(?<!\\)\1. ReDoS Detector agrees about the vulnerable regex, but is also unhappy about the replacement.

The EMPHASIS regex in 0.8.4, which was replaced by ASTERISK_EMPHASIS and UNDERSCORE_EMPHASIS in lepture/mistune@ca1e7b5, is \b_[^\s_](?:(?<=\\)_)?_|\*[^\s*](?:(?<=\\)\*)?\*|\b_[^\s_][\s\S]*?[^\s_]_(?!_|[^\s\\!"#$%&'()*+,./:;<=>?@\[\]^`{}|_~-])\b|\*[^\s*"<\[][\s\S]*?[^\s*]\*. ReDoS Checker detects only polynomial behaviour, and ReDoS Detector believes that it is safe.

Obviously there are more regexes in mistune than just that one, but I'm not sure that mistune 0.8.4 is vulnerable.

[FRidh]

Thinking about the implication it has on nbconvert. The CVE is about catastrophic backtracking, a potentially endless search. This clearly could be an issue. nbconvert is a tool and library for converting notebooks. What's the worst case I can come up with? DDOS of an nbconvert service.

To me this is a perfect case where having the vulnerability being mentioned is good, and users simply should accept the risk.

[sersorrel]

Since mistune 0.8.4 turned out not to be vulnerable (#188031), I think there's little point backporting this change. May as well keep stable as stable as possible.