Allow natural language search on user content like notes, images, transactions using transformer based models
All data is processed locally. User can interface with semantic-search app via Emacs, API or Commandline
git clone https://github.com/debanjum/semantic-search && cd semantic-search
Add Content Directories for Semantic Search to Docker-Compose Update docker-compose.yml to mount your images, org-mode notes, ledger/beancount directories If required, edit config settings in docker_sample_config.yml.
docker-compose up -d
- Install Python3 [Required[
- Install Conda [Required]
- Install Exiftool [Optional]
sudo apt-get -y install libimage-exiftool-perl
git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml
conda activate semantic-search
Configure application search types and their underlying data source/files in sample_config.yml
Use the sample_config.yml
as reference
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML
python3 -m src.main -c=sample_config.yml -vv
- Semantic Search via Emacs
- Install semantic-search.el
- Run
M-x semantic-search <user-query>
- Semantic Search via API
- Query:
GET
http://localhost:8000/search?q=”What is the meaning of life”&t=notes - Regenerate Embeddings:
GET
http://localhost:8000/regenerate - Semantic Search API Docs
- Query:
- UI to Edit Config
docker-compose up
cd semantic-search
git pull origin master
conda env update -f environment.yml
conda activate semantic-search
- MiniLM Model for Asymmetric Text Search. See SBert Documentation
- OpenAI CLIP Model for Image Search. See SBert Documentation
- Charles Cave for OrgNode Parser
- Sven Marnach for PyExifTool