DDR

This repository provides codes for paper "Dense Retrieval as Indirect Supervision for Large-space Decision Making" accepted by EMNLP 2023

Data Preparation and Evaluation

Ultra-fine Entity Typing

1. Download the dataset

NOTE: please refer to https://www.cs.utexas.edu/~eunsol/html_pages/open_entity.html for original UFET dataset

mkdir -p datasets && cd datasets
gdown https://drive.google.com/file/d/1HYqRFDbctj6VkqAZt1IVenBnUk7j65pd/view?usp=sharing && unzip UFET.zip
cd ..

2. Prepare DPR data format

python explore_ufet.py --stage data_preparation

files are saved at ./datasets/UFET/crowd/dpr_processed/query_context_direct_256 3. Sample evaluate codes

python explore_ufet.py --stage evaluation

Few-shot Intent Classification

Download the dataset

mkdir -p repositories && cd repositories
git clone https://github.com/jianguoz/Few-Shot-Intent-Detection.git
cd ..
gdown https://drive.google.com/file/d/1iGtJqMXHLIXb8BU5fbUAZdjGpD_92Cdm/view?usp=sharing && unzip results.zip

BANKING

python explore_banking77.py
# 5-shot train data: ./datasets/BANKING77/train_5/dpr_processed/query_context_direct_256_train_5_gtr-t5-xl_-1examples/trn.json
# 10-shot train data: ./datasets/BANKING77/train_10/dpr_processed/query_context_direct_256_train_10_gtr-t5-xl_-1examples/trn.json

CLINC

python explore_clinc150.py
# 5-shot train data: ./datasets/CLINC150/train_5/dpr_processed/query_context_direct_256_train_5_gtr-t5-xl_-1examples/trn.json
# 10-shot train data: ./datasets/CLINC150/train_10/dpr_processed/query_context_direct_256_train_10_gtr-t5-xl_-1examples/trn.json

HWU

python explore_hwu64.py
# 5-shot train data: ./datasets/HWU64/train_5/dpr_processed/query_context_direct_256_train_5_gtr-t5-xl_-1examples/trn.json
# 10-shot train data: ./datasets/HWU64/train_10/dpr_processed/query_context_direct_256_train_10_gtr-t5-xl_-1examples/trn.json

Acknowledgement

This repo wouldn't be possible without the awesome UFET, Context-TE, Few-Shot-Intent-Detection

Citation

If you find DDR useful for your research and applications, please consider starring this repository and citing:

@article{xu2023dense,
  title={Dense Retrieval as Indirect Supervision for Large-space Decision Making},
  author={Xu, Nan and Wang, Fei and Dong, Mingtao and Chen, Muhao},
  journal={arXiv preprint arXiv:2310.18619},
  year={2023}
}