Skip to content

Commit

Permalink
Add option to adjust restart randomizer
Browse files Browse the repository at this point in the history
* Add option to adjust the rolling restart randomization.
* Increase the default random factor from 10 seconds to 300 seconds.
* Make sure rand() gets a float to avoid producing integer second.
* Adjust tests to use shortened restart splay.

Signed-off-by: Ben Kochie <superq@gmail.com>
  • Loading branch information
SuperQ committed Aug 26, 2020
1 parent 45b55a1 commit 8a0ef25
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 4 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## Master

- Add option to adjust restart randomizer (#78)
- Simplify workers memory calculation in PumaMemory‘s `get_total` method #81
- Add rubocop in gemspec and CI, with offenses corrected and unnecessary cops disabled.
- Add `pre_term`-like `rolling_pre_term` config for terminations caused by rolling restart (#86)
Expand Down
8 changes: 6 additions & 2 deletions lib/puma_worker_killer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ module PumaWorkerKiller
extend self

attr_accessor :ram, :frequency, :percent_usage, :rolling_restart_frequency,
:rolling_restart_splay_seconds,
:reaper_status_logs, :pre_term, :rolling_pre_term, :on_calculation

self.ram = 512 # mb
self.frequency = 10 # seconds
self.percent_usage = 0.99 # percent of RAM to use
self.rolling_restart_frequency = 6 * 3600 # 6 hours in seconds
self.rolling_restart_splay_seconds = 0.0..300.0 # 0 to 5 minutes in seconds
self.reaper_status_logs = true
self.pre_term = nil
self.rolling_pre_term = nil
Expand All @@ -30,8 +32,10 @@ def start(frequency = self.frequency, reaper = self.reaper)
enable_rolling_restart(rolling_restart_frequency) if rolling_restart_frequency
end

def enable_rolling_restart(frequency = rolling_restart_frequency)
frequency += rand(0..10.0) # so all workers don't restart at the exact same time across multiple machines
def enable_rolling_restart(frequency = rolling_restart_frequency,
splay_seconds = rolling_restart_splay_seconds)
# Randomize so all workers don't restart at the exact same time across multiple machines.
frequency += rand(splay_seconds)
AutoReap.new(frequency, RollingRestart.new(nil, rolling_pre_term)).start
end
end
Expand Down
2 changes: 1 addition & 1 deletion test/fixtures/rolling_pre_term.ru
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ load File.expand_path('fixture_helper.rb', __dir__)
PumaWorkerKiller.config do |config|
config.rolling_pre_term = ->(worker) { puts("About to terminate (rolling) worker: #{worker.pid}") }
end
PumaWorkerKiller.enable_rolling_restart(1) # 1 second
PumaWorkerKiller.enable_rolling_restart(1, 0..5.0) # 1 second, short 1-5s splay.

run HelloWorldApp
2 changes: 1 addition & 1 deletion test/fixtures/rolling_restart.ru
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

load File.expand_path('fixture_helper.rb', __dir__)

PumaWorkerKiller.enable_rolling_restart(1) # 1 second
PumaWorkerKiller.enable_rolling_restart(1, 0..5.0) # 1 second, short 1-5s splay.

run HelloWorldApp

0 comments on commit 8a0ef25

Please sign in to comment.