Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Implement Intl.Segmenter #2840

Closed
wants to merge 3 commits into from
Closed

Conversation

jedel1043
Copy link
Member

The new ICU4X release stabilized the icu_segmenter component, so this PR implements Intl.Segmenter using that as a base.

Also, I opted for importing itertools instead of copy-pasting the implementation of TupleWindows because its design is a lot more complex than Intersperse, which we copy-pasted previously. Though, I disabled all std features of itertools to make it a lot more lightweight, so it shouldn't make much difference in compilation times.

@jedel1043 jedel1043 added enhancement New feature or request API labels Apr 18, 2023
@jedel1043 jedel1043 added this to the v0.17.0 milestone Apr 18, 2023
@github-actions
Copy link

github-actions bot commented Apr 18, 2023

Test262 conformance changes

Test result main count PR count difference
Total 94,591 94,591 0
Passed 71,670 71,824 +154
Ignored 17,686 17,530 -156
Failed 5,235 5,237 +2
Panics 0 0 0
Conformance 75.77% 75.93% +0.16%
Fixed tests (154):
test/intl402/Segmenter/ctor-custom-get-prototype-poison-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/ctor-custom-get-prototype-poison-throws.js (previously Ignored)
test/intl402/Segmenter/ctor-custom-prototype.js [strict mode] (previously Ignored)
test/intl402/Segmenter/ctor-custom-prototype.js (previously Ignored)
test/intl402/Segmenter/ctor-default-prototype.js [strict mode] (previously Ignored)
test/intl402/Segmenter/ctor-default-prototype.js (previously Ignored)
test/intl402/Segmenter/proto-from-ctor-realm.js [strict mode] (previously Ignored)
test/intl402/Segmenter/proto-from-ctor-realm.js (previously Ignored)
test/intl402/Segmenter/prototype/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/prop-desc.js (previously Ignored)
test/intl402/Segmenter/prototype/Symbol.toStringTag.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/Symbol.toStringTag.js (previously Ignored)
test/intl402/Segmenter/prototype/toStringTag/toStringTag.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/toStringTag/toStringTag.js (previously Ignored)
test/intl402/Segmenter/prototype/toStringTag/toString.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/toStringTag/toString.js (previously Ignored)
test/intl402/Segmenter/prototype/constructor/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/constructor/prop-desc.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/next-inside-next.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/next-inside-next.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/next-mix-with-containing.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/next-mix-with-containing.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-word-iterable.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-word-iterable.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/prop-desc.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-sentence-iterable.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-sentence-iterable.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-tostring.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-tostring.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/nested-next.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/nested-next.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/name.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/name.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-grapheme-iterable.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/segment-grapheme-iterable.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/length.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/length.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/branding.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/branding.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/iswordlike.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/iswordlike.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/unbreakable-input.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/unbreakable-input.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/prop-desc.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/index-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/index-throws.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/zero-index.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/zero-index.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/one-index.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/one-index.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/name.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/name.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/word-iswordlike.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/word-iswordlike.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/length.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/length.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/breakable-input.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/breakable-input.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/out-of-bound-index.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/out-of-bound-index.js (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/branding.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/segment/containing/branding.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/order.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/order.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/prop-desc.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/type-without-lbs.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/type-without-lbs.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/name.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/name.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/length.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/length.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/caching.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/caching.js (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/branding.js [strict mode] (previously Ignored)
test/intl402/Segmenter/prototype/resolvedOptions/branding.js (previously Ignored)
test/intl402/Segmenter/constructor/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/prop-desc.js (previously Ignored)
test/intl402/Segmenter/constructor/name.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/name.js (previously Ignored)
test/intl402/Segmenter/constructor/length.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/length.js (previously Ignored)
test/intl402/Segmenter/constructor/prototype.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/prototype.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-toString-abrupt-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-toString-abrupt-throws.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-valid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-valid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-abrupt-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-abrupt-throws.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-order.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-order.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-abrupt-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-granularity-abrupt-throws.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-valid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-valid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-getoptionsobject.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-getoptionsobject.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-undefined.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-undefined.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-valid-combinations.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-valid-combinations.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-throwing-getters.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-throwing-getters.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/proto-from-ctor-realm.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/proto-from-ctor-realm.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-toString-abrupt-throws.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-localeMatcher-toString-abrupt-throws.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/locales-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/locales-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/subclassing.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/subclassing.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/newtarget-undefined.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/newtarget-undefined.js (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-null.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/constructor/options-null.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/prop-desc.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/prop-desc.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-localeMatcher-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-localeMatcher-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/result-type.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/result-type.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-undefined.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-undefined.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/name.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/name.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-specific.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-specific.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-empty.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-empty.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-invalid.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/locales-invalid.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/length.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/length.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-toobject.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-toobject.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/branding.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/branding.js (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-null.js [strict mode] (previously Ignored)
test/intl402/Segmenter/constructor/supportedLocalesOf/options-null.js (previously Ignored)
test/intl402/Segmenter/instance/extensibility.js [strict mode] (previously Ignored)
test/intl402/Segmenter/instance/extensibility.js (previously Ignored)
test/intl402/Segmenter/instance/prototype.js [strict mode] (previously Ignored)
test/intl402/Segmenter/instance/prototype.js (previously Ignored)
test/intl402/String/prototype/localeCompare/throws-same-exceptions-as-Collator.js [strict mode] (previously Failed)
test/intl402/String/prototype/localeCompare/throws-same-exceptions-as-Collator.js (previously Failed)

@codecov
Copy link

codecov bot commented Apr 18, 2023

Codecov Report

Merging #2840 (b9d43d2) into main (d288b63) will decrease coverage by 0.30%.
The diff coverage is 20.30%.

@@            Coverage Diff             @@
##             main    #2840      +/-   ##
==========================================
- Coverage   51.38%   51.09%   -0.30%     
==========================================
  Files         417      419       +2     
  Lines       41328    41602     +274     
==========================================
+ Hits        21238    21258      +20     
- Misses      20090    20344     +254     
Impacted Files Coverage Δ
boa_engine/src/builtins/intl/segmenter/options.rs 0.00% <0.00%> (ø)
boa_engine/src/object/mod.rs 32.89% <3.70%> (-0.90%) ⬇️
boa_engine/src/builtins/intl/segmenter/mod.rs 7.84% <6.12%> (-35.02%) ⬇️
boa_engine/src/context/icu.rs 30.15% <12.90%> (-9.43%) ⬇️
boa_engine/src/builtins/intl/segmenter/iterator.rs 14.51% <14.51%> (ø)
boa_engine/src/builtins/intl/segmenter/segments.rs 15.09% <15.09%> (ø)
boa_engine/src/builtins/intl/locale/utils.rs 51.87% <70.00%> (+1.53%) ⬆️
boa_engine/src/builtins/json/mod.rs 85.26% <75.00%> (+0.58%) ⬆️
boa_engine/src/builtins/iterable/mod.rs 67.85% <90.90%> (+0.70%) ⬆️
boa_engine/src/builtins/mod.rs 100.00% <100.00%> (ø)
... and 1 more

... and 6 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@raskad raskad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

boa_engine/src/builtins/intl/segmenter/mod.rs Outdated Show resolved Hide resolved
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! :)

@jedel1043
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request Apr 20, 2023
The new ICU4X release stabilized the `icu_segmenter` component, so this PR implements `Intl.Segmenter` using that as a base.

Also, I opted for importing `itertools` instead of copy-pasting the implementation of `TupleWindows` because its design is a lot more complex than `Intersperse`, which we copy-pasted previously. Though, I disabled all `std` features of `itertools` to make it a lot more lightweight, so it shouldn't make much difference in compilation times.
@bors
Copy link

bors bot commented Apr 20, 2023

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Implement Intl.Segmenter [Merged by Bors] - Implement Intl.Segmenter Apr 20, 2023
@bors bors bot closed this Apr 20, 2023
@bors bors bot deleted the intl-segmenter branch April 20, 2023 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants