SQLite3::BusyException: database is locked

[dmke]

Problem

From time to time, the commit hook fails like so:

$ git push
[...]
remote: /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/resultset.rb:108:in `step': database is locked (SQLite3::BusyException)
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/resultset.rb:108:in `next'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/resultset.rb:133:in `each'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:202:in `to_a'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:202:in `block in execute'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:151:in `prepare'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:193:in `execute'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:650:in `commit'
remote:         from /var/lib/gems/2.5.0/gems/sqlite3-1.4.1/lib/sqlite3/database.rb:638:in `transaction'
remote:         from /opt/dnsgit/lib/backend/sqlite.rb:242:in `set_metadata'
remote:         from /opt/dnsgit/lib/backend/sqlite.rb:170:in `block in update_database'
remote:         from /opt/dnsgit/lib/backend/sqlite.rb:169:in `each'
remote:         from /opt/dnsgit/lib/backend/sqlite.rb:169:in `update_database'
remote:         from /opt/dnsgit/lib/backend/sqlite.rb:79:in `deploy'
remote:         from /opt/dnsgit/lib/zone_generator.rb:29:in `deploy'
remote:         from /opt/dnsgit/lib/zone_generator.rb:6:in `run'
remote:         from hooks/../../bin/run.rb:5:in `<main>'
[...]
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '<remote>:/opt/dnsgit/data'

While this is normal and expected behaviour from SQLite, this is annoying for the user, who need to push again and manually execute the hooks on the server.

Solution

1. add a timeout to the Backend::SQLite initializer
   (add a config variable to set the number of retries)
2. add a retry mechanism (balance retry and timout; short timeout = more retries)
   (not needed, SQLite3 already ships a retry mechanism)
3. run an early test to see whether the DB file is locked or not (something along select 1 from domains)
   (dito, SQLite does this automagically)