Implementing NLP tasks using Finite State Automata

In this Project we have looked into the application of Finite Automata in generating algorithms for Tokenization, Stop Word Removal and Pattern Searching, which are some of the common steps involved in NLP and Genomics. We have also discussed the efficiency of each algorithm by testing over various test cases.

🛠 Tools

Python

Algorithms

1) For performing Tokenization/lexical analysis

State-transition table for the finite state machine

2) For performing Pattern Search

3) For Stop Word Removal

FSA designed for Stop Word removal