Optimus is the missing framework for cleaning and pre-processing data in a distributed fashion. It uses all the power of Apache Spark (optimized via Catalyst) to do so. It implements several handy tools for data wrangling and munging that will make your life much easier. The first obvious advantage over any other public data cleaning library or framework is that it will work on your laptop or your big cluster, and second, it is amazingly easy to install, use and understand.
Click on Optimus to enter our Website:
Please take a couple of minutes to help shape the Optimus' Roadmap:
The first version of Optimus is out now! Check out the new examples and documentation to see all the changes. We want to thank the projects we rely on: pixiedust and spark-df-profiling and the help from our amazing community. We hope that you find the framework useful and interesting. Have fun!
Optimus main stable branch will work now for Spark 2.2.0 The 1.6.x version is now under maintenance, the last tag release for this Spark version is the 0.4.0. We strongly suggest that you use the >2.x version of the framework because the new improvements and features will be added now on this version.
- Apache Spark 2.2.0
- Python=>3.5
In your terminal just type:
pip install optimuspyspark
- Project Manager: Argenis León
- Original developers: Andrea Rosales, Hugo Reyes, Alberto Bonsanto
- Principal developer and maintainer: Favio Vázquez
Apache 2.0 © Iron
