Track last seen post timestamp per wiki #100
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
2a85c2c fixed a bug where a post would be skipped being downloaded if a post was stored with a later timestamp than it, which because posts are downloaded ordered by wiki and not by timestamp, was causing almost all posts to be skipped. It fixed it by limiting the 'latest timestamp' search only to posts from the given wiki.
While effective, this is not efficient. Most downloaded posts are deleted during the very same run, meaning on the next run, many of those same posts will be downloaded again, wasting time and networking resources.
This PR fixes this by recording the last time that posts were downloaded from a wiki in the context_wiki table. This recording will be permanent, even if the corresponding post is deleted.