🆕 Personal Infromation Tagger Based on Named entity recognition

Named entity recognition (NER) helps you easily identify the key elements in a text, like names of people, places, brands, monetary values, and more.Extracting the main entities in a text helps sort unstructured data and detect important information, which is crucial if you have to deal with large datasets.

💽 Dataset

XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 typologically diverse languages and includes nine tasks.

📚 Approach

1. Get data and properly create text and label (Can be done using https://explosion.ai/demos/displacy-ent.
2. Use trasnformer Roberta architecture for training the ner tagger
3. Use hugging face for Robereta Tokenizer
4. Train and Deploy model for use-cases

🚀 API

🧑‍💻 How to setup

create fresh conda environment

conda create -p ./env python=3.8 -y

activate conda environment

conda activate ./env

Install requirements

pip install -r requirements.txt

To run train pipeline

python ner/pipeline/train_pipeline.py

To run inferencing

python app.py

To launch swagger ui

http://localhost:8085/docs

🧑‍💻 Tech Used

1. Natural Language processing
2. Pytorch
3. Transformer
4. FastApi

🏭 Industrial Use-cases

1. Search and Recommendation system
2. Content Classification
3. Customer Support
4. Research Paper Screening
5. Automatically Summarizing Resumes

👋 Conclusion

We have shown how to train our own name entity tagger along with proper inplementaion of train and predict pipeline.