Avoid constantly attempting to acquire a lock

[ddaspit]

• ignore expiration timeout on WaitForChangesAsync when there are queued writer locks

---

This change is 

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 76.92308% with 3 lines in your changes missing coverage. Please review.

Project coverage is 57.06%. Comparing base (8c5f802) to head (80f4c6e).

Files with missing lines	Patch %	Lines
...ine.Shared/Services/DistributedReaderWriterLock.cs	76.92%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #491   +/-   ##
=======================================
  Coverage   57.05%   57.06%           
=======================================
  Files         277      277           
  Lines       14165    14168    +3     
  Branches     1901     1902    +1     
=======================================
+ Hits         8082     8085    +3     
  Misses       5495     5495           
  Partials      588      588           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[johnml1135]

This appears to prevent the infinite cycling. Still, if there is an expired writer on the queue (with a different lock Id), it will never be cleared away. Such could happen if the database is inaccessible when trying to remove the lock.

I see that you don't put any ExpiresAt on the WriterQueue, leaving it null - they will hold forever (unless there is something else that causes it to crash out). Would this be plausible for the WriterQueue?

• Have a timeout refresh interval of 10 seconds
• Set the ExpiresAt interval to 31 seconds
• Every time we try to acquire the writer lock and fail, we upsert the queued item with a new ExpiresAt time (31 seconds later)
• When we want to check for nothing queued, we look for non-expired items in the WriterQueue.

Is that overkill? Should we try for this fix and see if it resolves it?

[johnml1135]

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @ddaspit)

[ddaspit]

There are circumstances where the writer queue doesn't get cleared properly. Your proposed solution would definitely work. It would result in extra queries/updates to the database. We would be trying to acquire the lock every 10 seconds instead of waiting until the currently acquired lock expired. Also, we would need to do an extra update for the ExpiresAt property. Another option would be to add a timeout when attempting to acquire a lock. The timeout value could be used to set the ExpiresAt property for the writer queue entry. If we want to deal with this possible case, we can do it in another PR.

Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @ddaspit)

[johnml1135]

Sure.