Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[server] Cache problems #15

Open
michaelfromyeg opened this issue Jun 6, 2021 · 0 comments
Open

[server] Cache problems #15

michaelfromyeg opened this issue Jun 6, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@michaelfromyeg
Copy link
Owner

michaelfromyeg commented Jun 6, 2021

It takes a long time to fetch and scrape the problem each time. It also opens up the project to uncaught "runtime" errors (i.e., bad problem formatting after scrape).

Instead, we could cache problems. The very first time a user requests a problem with a given ID, we could query and see if we've stored it already (adds milliseconds to their request) and after it's sent to the user, store the problem information for subsequent calls.

Feasibility

  • Project Euler's copyright seems to allow for this
  • Reddit users own their post so this one is trickier, but I think it's okay to save and reuse the posts
  • CodingBat copyright belongs to Nick Parlante, so I'll have to reach out to him/the website before I proceed

Design

The changes are minor. In each handler (sans coding bat), we include the following logic

  1. Check DB for post ID (--if user does not provide a post ID, check random ID against the DB)
    2a. If post exists, return cached post
    2b. If post does not exist, query site, create payload, save to DB, and return payload

Occasionally, a Cron job should probably clean and validate problems.

As for design, I think a NoSQL design is fine here since we'll only ever return a post at a time, and the post format could change. MongoDB makes the most sense out of current choices.

@michaelfromyeg michaelfromyeg added the enhancement New feature or request label Jun 6, 2021
@michaelfromyeg michaelfromyeg self-assigned this Jun 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant