Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit the number of ping upload requests per minute #974

Merged
merged 3 commits into from
Jun 23, 2020

Conversation

brizental
Copy link
Contributor

@brizental brizental commented Jun 12, 2020

Related to Bug 1543612

I wanted to get some feedback on my approach to this problem before I continued. My goal here was to try and replicate what Firefox does to rate limit ping uploads, specifically:

Only allow up to 10 ping uploads per minute.

This turned out to be tricky. The challenges on the current implementation and how I decided to solve them are:

  • It doesn't really know exactly when the ping is uploaded.
    • My solution here was to ignore this fact and just try to limit the number of times get_upload_task can be called per minute.
  • It sends pings to upload one at a time, not in batches.
    • I created a counter to know how many pings have been sent out in the past minute.

NOTE: I am aware this breaks the ffi code and all the bindings, I'll deal with that after.

@brizental brizental requested a review from Dexterp37 June 12, 2020 10:23
@Dexterp37 Dexterp37 requested a review from badboy June 12, 2020 12:38
Copy link
Contributor

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this approach, but I'd like @badboy input on it as well.

glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
Copy link
Member

@badboy badboy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed that looks like a good start.
I have some comments, but not much against the approach by itself.

glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
@brizental brizental marked this pull request as ready for review June 22, 2020 08:03
@auto-assign auto-assign bot requested a review from mdboom June 22, 2020 08:03
@brizental brizental requested review from badboy and Dexterp37 and removed request for mdboom June 22, 2020 08:03
Copy link
Contributor

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me overall. Before merging this, can we please add a documentation page describing it? I'm fine with it living within the Glean internal docs. I'd like to see some documentation explaining:

  • the overview of how this is supposed to work;
  • how are we enforcing the rate limitation;
  • what are the default choices.

glean-core/src/upload/mod.rs Show resolved Hide resolved
glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
Copy link
Member

@badboy badboy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, see one comment.
Once we got the docs that Dexter asked for we should be good to go.

glean-core/src/upload/mod.rs Outdated Show resolved Hide resolved
docs/user/pings/index.md Outdated Show resolved Hide resolved
@brizental brizental requested a review from Dexterp37 June 22, 2020 16:25
Copy link
Contributor

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! r+wc :)

docs/user/pings/index.md Outdated Show resolved Hide resolved
glean-core/src/lib.rs Show resolved Hide resolved
@brizental brizental merged commit c6e1349 into mozilla:main Jun 23, 2020
@brizental brizental deleted the 1543612 branch June 23, 2020 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants