The Open Assistant inference stack includes a toggleable safety pipeline which can be used to mitigate harmful requests and outputs to and from the model.
This directory was initially used for development of the safety pipeline, but it has now been migrated to a standalone repository called blade2blade.