Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flows: model simple scoring function to prioritize running flows in the same time slot #399

Open
zaychenko-sergei opened this issue Dec 27, 2023 · 0 comments
Labels
enhancement New feature or request rust Pull requests that update Rust code

Comments

@zaychenko-sergei
Copy link
Contributor

zaychenko-sergei commented Dec 27, 2023

Currently all flows are equal, and only the activation time and order of arrival define the order of tasks scheduling.

Prototype a smarter algorithm that provides more fairness:

  • the activation time for flows should be rounded to the reasonable period (i.e. 1 minute), so that a minor difference like 1s won't be breaking the scoring functions' logic
  • time wheel's priority queue could use a more complex key, that represents both rounded activation time + a priority score, so that flows are properly grouped by rounded activation time, but stay sorted by score within the same activation time
  • implement demo of score computing logic:
    • rarely executed flow goes first (compare schedule periods)
    • flow with higher derived fanout goes first

In future, the scoring function will be more sophisticated, and will include preference of premium datasets or some sort of QoS guarantee (weight of paid datasets, delay penalty, fairness improvements, average work amount stats)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request rust Pull requests that update Rust code
Projects
None yet
Development

No branches or pull requests

1 participant