Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meilisearch for prefetched courses #833

Merged
merged 4 commits into from
Dec 3, 2022
Merged

Conversation

joschahenningsen
Copy link
Sponsor Member

@joschahenningsen joschahenningsen commented Dec 2, 2022

Motivation and Context

The current sql fulltext search is not user friendly. To overcome this issue we can try using meilisearch as discussed here: #814 It even handles typos very well.

Description

The courses are prefetched and stored in a newly introduced search engine instead of the database. From there we can simply query them.

Steps for Testing

Run a local instance of meilisearch like this:

docker run -it --rm \
    -p 7700:7700 \
    -e MEILI_MASTER_KEY='MASTER_KEY'\
    -v $(pwd)/meili_data:/meili_data \
    getmeili/meilisearch:v0.30 \
    meilisearch --env="development"

Download and extract this json document:
prefetchedCourses.zip

Push it to meilisearch as demo data:

curl \                                      
  -H "Authorization: Bearer MASTER_KEY" -X POST 'http://localhost:7700/indexes/PREFETCHED_COURSES/documents?primaryKey=courseID' \
  -H 'Content-Type: application/json' \
  --data-binary @prefetchedCourses.json

Test the search when creating a course

The import can only be tested with a tum online token so you'll need to trust me on this :P

Screenshots

2022-12-02.15-13-22.mp4

@github-actions
Copy link

github-actions bot commented Dec 2, 2022

Your Testserver will be ready at https://833.test.live.mm.rbg.tum.de in a few minutes.

@joschahenningsen
Copy link
Sponsor Member Author

Thanks to @CommanderStorm for the input on this :)

@alexanderstephan
Copy link
Collaborator

alexanderstephan commented Dec 2, 2022

When run meilisearch and add the json file, I get no results (although all requests are status 200). Do you have any idea what could be wrong?
I get [2022-12-02T20:06:08Z INFO index_scheduler] A batch of tasks was successfully completed., so I assume it is added.

@alexanderstephan
Copy link
Collaborator

I think I get the error missing_document_id, which might be related to a broken setup.

@joschahenningsen
Copy link
Sponsor Member Author

Hm I thought the ?primaryKey might do the trick 🤔
I'll check this out tomorrow!

@joschahenningsen
Copy link
Sponsor Member Author

@alexanderstephan I updated the zip file, it should work with the new one :)

Copy link
Collaborator

@alexanderstephan alexanderstephan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, it really works great. 🎉 The code looks also flawless. 👍

@alexanderstephan alexanderstephan merged commit 4e7371f into dev Dec 3, 2022
@alexanderstephan alexanderstephan deleted the meli-prefetched-courses branch December 3, 2022 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants