Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Insurance policy in case iter.size_hint() lies. #65749

Merged
merged 1 commit into from
Oct 25, 2019

Conversation

Centril
Copy link
Contributor

@Centril Centril commented Oct 24, 2019

Follow up to https://github.com/rust-lang/rust/pull/64949/files#r334235076.
(If the perf impact is bad we can use debug_assert! instead.)

The good news is that the UI tests pass locally so iter.size_hint() seems to be honest thus far.
On the other hand, with the status quo we do not have an insurance policy should that change in some case. This is problematic because a) this could possibly make some program be accepted which shouldn't, b) the compiler itself could have memory unsafety if the correctness of the iterator is assumed in unsafe { ... } code (even though the blame lies with the unsafe { ... } block in question.)

r? @RalfJung
cc @nnethercote

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 24, 2019
@Centril
Copy link
Contributor Author

Centril commented Oct 24, 2019

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Oct 24, 2019

⌛ Trying commit 256335b with merge bc4dafa90d4ba113c682edfe266ea94a0e2fbc2f...

@bors
Copy link
Contributor

bors commented Oct 24, 2019

☀️ Try build successful - checks-azure
Build commit: bc4dafa90d4ba113c682edfe266ea94a0e2fbc2f (bc4dafa90d4ba113c682edfe266ea94a0e2fbc2f)

@rust-timer
Copy link
Collaborator

Queued bc4dafa90d4ba113c682edfe266ea94a0e2fbc2f with parent 4a8c5b2, future comparison URL.

Copy link
Member

@RalfJung RalfJung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me if perf looks good.

f(&[t0, t1])
}
(0, Some(0)) => {
assert!(iter.next().is_none());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it matters this could be written without unwrap's or asserts.

let a = match iter.next() {
    None => return Ok(f(&[])),
    Some(a) => a?,
};
let b = match iter.next() {
    None => return Ok(f(&[a])),
    Some(b) => b?,
};
let c = match iter.next() {
    None => return Ok(f(&[a, b])),
    Some(c) => c?,
};
let mut vec: SmallVec<[_; 8]> = SmallVec::with_capacity(iter.size_hint().0 + 3);
vec.push(a);
vec.push(b);
vec.push(c);
for result in iter {
    vec.push(result?);
}
Ok(f(&vec))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! OTOH, the original order of the match was chosen by @nnethercote to reflect the frequency of the cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RalfJung Should we perf test this variant?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No opinion. @nnethercote what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vec construction could be more concise, e.g. using extend. And I would use t0, t1, t2 instead of a, b, c. And the comment needs updating. Otherwise, fine by me to use this form if the perf isn't hurt.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extend would be better but there isn't a way in std to extend from a Result iterator so you need a workaround like https://github.com/mitsuhiko/redis-rs/blob/418512bf9171589b814da4c5ecb500e8747781e0/src/parser.rs#L21-L54

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit bc4dafa90d4ba113c682edfe266ea94a0e2fbc2f, comparison URL.

@Centril
Copy link
Contributor Author

Centril commented Oct 24, 2019

Perf looks like noise / possibly an improvement.

@Centril
Copy link
Contributor Author

Centril commented Oct 25, 2019

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Oct 25, 2019

⌛ Trying commit c85bfc5 with merge 2beb6a3...

bors added a commit that referenced this pull request Oct 25, 2019
Insurance policy in case `iter.size_hint()` lies.

Follow up to https://github.com/rust-lang/rust/pull/64949/files#r334235076.
(If the perf impact is bad we can use `debug_assert!` instead.)

The good news is that the UI tests pass locally so `iter.size_hint()` seems to be honest *thus far*.
On the other hand, with the status quo we do not have an insurance policy should that change in some case. This is problematic because a) this could possibly make some program be accepted which shouldn't, b) the compiler itself could have memory unsafety if the correctness of the iterator is assumed in `unsafe { ... }` code (even though the blame lies with the `unsafe { ... }` block in question.)

r? @RalfJung
cc @nnethercote
@bors
Copy link
Contributor

bors commented Oct 25, 2019

☀️ Try build successful - checks-azure
Build commit: 2beb6a3 (2beb6a387a95fb690924f39416f3cee585149506)

@rust-timer
Copy link
Collaborator

Queued 2beb6a3 with parent 10a52c2, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 2beb6a3, comparison URL.

@Centril
Copy link
Contributor Author

Centril commented Oct 25, 2019

@nnethercote Thoughts on ^---? It looks like a slight regression to me.

@nnethercote
Copy link
Contributor

Let's go with the first one! :)

@mati865
Copy link
Contributor

mati865 commented Oct 25, 2019

I think Clippy even has lint against:

for result in iter {
    vec.push(result?);
}

IIRC this performs poorly and could be main reason of this regression.

lies, underreporting the number of elements.
@Centril
Copy link
Contributor Author

Centril commented Oct 25, 2019

Dropped the second commits in favor of the first solution which had clean perf; @bors r=RalfJung

@bors
Copy link
Contributor

bors commented Oct 25, 2019

📌 Commit dfcfca2 has been approved by RalfJung

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 25, 2019
Centril added a commit to Centril/rust that referenced this pull request Oct 25, 2019
Insurance policy in case `iter.size_hint()` lies.

Follow up to https://github.com/rust-lang/rust/pull/64949/files#r334235076.
(If the perf impact is bad we can use `debug_assert!` instead.)

The good news is that the UI tests pass locally so `iter.size_hint()` seems to be honest *thus far*.
On the other hand, with the status quo we do not have an insurance policy should that change in some case. This is problematic because a) this could possibly make some program be accepted which shouldn't, b) the compiler itself could have memory unsafety if the correctness of the iterator is assumed in `unsafe { ... }` code (even though the blame lies with the `unsafe { ... }` block in question.)

r? @RalfJung
cc @nnethercote
bors added a commit that referenced this pull request Oct 25, 2019
Rollup of 7 pull requests

Successful merges:

 - #63810 (Make <*const/mut T>::offset_from `const fn`)
 - #65705 (Add {String,Vec}::into_raw_parts)
 - #65749 (Insurance policy in case `iter.size_hint()` lies.)
 - #65799 (Fill tracking issue number for `array_value_iter`)
 - #65800 (self-profiling: Update measureme to 0.4.0 and remove non-RAII methods from profiler.)
 - #65806 (Add [T]::as_ptr_range() and [T]::as_mut_ptr_range().)
 - #65810 (SGX: Clear additional flag on enclave entry)

Failed merges:

r? @ghost
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Oct 25, 2019
Insurance policy in case `iter.size_hint()` lies.

Follow up to https://github.com/rust-lang/rust/pull/64949/files#r334235076.
(If the perf impact is bad we can use `debug_assert!` instead.)

The good news is that the UI tests pass locally so `iter.size_hint()` seems to be honest *thus far*.
On the other hand, with the status quo we do not have an insurance policy should that change in some case. This is problematic because a) this could possibly make some program be accepted which shouldn't, b) the compiler itself could have memory unsafety if the correctness of the iterator is assumed in `unsafe { ... }` code (even though the blame lies with the `unsafe { ... }` block in question.)

r? @RalfJung
cc @nnethercote
bors added a commit that referenced this pull request Oct 25, 2019
Rollup of 6 pull requests

Successful merges:

 - #65705 (Add {String,Vec}::into_raw_parts)
 - #65749 (Insurance policy in case `iter.size_hint()` lies.)
 - #65799 (Fill tracking issue number for `array_value_iter`)
 - #65800 (self-profiling: Update measureme to 0.4.0 and remove non-RAII methods from profiler.)
 - #65806 (Add [T]::as_ptr_range() and [T]::as_mut_ptr_range().)
 - #65810 (SGX: Clear additional flag on enclave entry)

Failed merges:

r? @ghost
@bors bors merged commit dfcfca2 into rust-lang:master Oct 25, 2019
@Centril Centril deleted the insurance-policy branch October 26, 2019 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants