v0.5.1

numpydocs was left out of requirements.txt and setup.py, so that pip install allennlp wouldn't install it and allennlp wouldn't work (unless you installed it manually). This fixes that.

v0.5.0

Breaking changes:

• Starting with v0.5.0, AllenNLP is based on PyTorch 0.4. This version of PyTorch contains many substantial changes from earlier releases and the internals of AllenNLP required corresponding changes. It is possible that this may break some of your custom models.

• The prior Attention and MatrixAttention modules have been renamed LegacyAttention and LegacyMatrixAttention, and have been replaced with Registrable classes that allow for much more efficient attention implementations. The minimal change to upgrade if you were using these previously is to just change where you have Attention to be LegacyAttention. Though, it'll probably speed up your code and/or reduce memory usage if you switch to using the new Attention.

Large new features:

• We have added a Configuration Wizard that makes it easier to create configuration files for your experiments. Its use is detailed in a new how-to.

Full list of changes:

7128324 (tag: v0.5.0) bump version number to v0.5.0
e17cbd1 fix bug (#1363)
691af4e address PR comments (#1362)
d844a9a fancy version of the configuration wizard (#1344)
6739c31 Make test_trainer_respects_keep_serialized_model_every_num_seconds more patient (#1358)
ac2e0b9 Make optional args actually optional. (#1357)
a08e7fd Move serve functionality out of allennlp command. (#1317)
97484fb Faster Elmo (#1347)
ca4d262 Remove norm from question / table similarity computation (#1354)
2cddc69 Add ability to add tokens to vocabulary namespaces (#1352)
b556e53 Format pytest logging correctly (#1351)
6aaeb70 Add bilinear attention (#1349)
f6d9ba3 Remove what appears to be outdated line (#1348)
6c87ff5 Make vocab (#1339)
0c2d5c7 config explorer webapp (#1329)
b6b8955 Fix link to demo.allennlp.org page (#1336)
4326b59 Fix link to installation instructions (#1335)
03d6fad (initializers-regularizers) command-line configuration explorer (#1309)
b0d0d94 LabelField.as_tensor returns 0-tensor (#1320)
765dc20 Add logging for the original path in load_archive. (#1323)
8eb358b Convert run_with_beaker.py to use Blueprints. (#1199)
427a319 Add version to AllenNLP command. (#1322)
2456983 Add option to keep Viterbi scores when predicting (#1314)
62f701e Remove default arguments from the simple server. (#1319)
50fcb0c how to for debugging using pycharm (#1312)
cebf719 SRL elmo 5b (#1310)
e37a68e Add "Citing" section to README.md (#1305)
0db4bc5 Add ELMo to BCN (#1308)
097ccd7 Update torch and tensorboard versions in setup.py (#1311)
26e7f41 add user contributed models blurb (#1306)
5352a19 make predictors more discoverable (#1302)
4849589 Script for generating logical forms from ERM model for wikitables (#1247)
504964b allow sanitize to work on custom classes (#1307)
200b27e A new model is only considered the best if it is better than others and not simply better or equal (#1246)
ed63b7e upgrade to pytorch 0.4.0 (#1126)
b26031d Ignore tests in coverage metrics (#1293)
986cf17 Improve performance of attention modules (#1235)
69b9af3 In Trainer, require "patience" to be a positive integer or None (meaning "no early stopping"). Set patience=None by default. (#1271)
9cb0ea3 Add LICENSE to source distribution (#1291)
c26a513 Noam lr schedule (#1258)
59a0e5a Fix test-install by moving tests into module (#1232)
c0afe31 Insteall psycopg2-binary (#1286)
1f8125c (cli-params) Update the version of twine. (#1285)
5c06914 Use full version in docs. (#1283)
20da3fd Set long_description_content_type to markdown in setup.py. (#1284)
7e1080e Uptick version to unreleased. (#1281)
3c623c5 Fix links to the allennlp-as-a-library tutorials (#1279)

v0.4.3

This is primarily a bugfix release as well as the final release on PyTorch 0.3.

• Fixes the TQDM bug that was causing jobs to hang in our internal job execution framework (Beaker).

• Moves tests into the source distribution, which a user needed to create a conda installation.

Here is a full list of commits:

66da4ee Update version numbers referred to in tutorials.
1c8b149 (upstream/master, master) Fix crash when validation metric is 0 (#1274)
b4c9fe4 (release-v0.4.2) Migrate gevent's wsgi to pywsgi (#1272)
6b5b017 allennlp train should fail faster if trainer.cuda_device >= 0 (#1196)
40691d3 Fix LR scheduler and test usage inside Trainer (#1255)
ed3697c Remove most mentions of python -m allennlp.run (#1264)
7b9d6a2 Consolidate installation instructions (#1261)
622ea1b (origin/master, origin/HEAD) Tutorial cleanup (#1260)
c2b61c6 make evalb use unique temporary directories per process (#1254)
906b26f make it OK if serialization_dir exists but is empty (#1251)
5ebe103 Add missing close brace, standardize indentation in ELMo howto (#1252)
eabb37f Output predictions in CSV format for official eval (#1249)
d1c36de Constituency Parser Demo Reference Update (#1248)
4f2db42 Replace slicing with chunk along the last dim (#1244)
a4f2fb8 Cache remote datasets with fixed-length filenames (#1230)
9ef013d Fix Constituency Parser / EVALB for pip-installed allennlp (#1233)
3dcde32 Add custom extensions to distribution and install (#1229)
229361c In train_model(), load the best weights at the end so that: (#1234)
76a5a80 changed checklist cost computation (#1231)
d0bd42d edit project root test (#1227)
356829d Add allennlp test-install command (#1213)
820cbfb gitignore EVALB generated files (#1214)
3a8fdb0 Include tests in source distribution (#1208)
10ea3b3 Add the attention visualization to the textual entailment demo (#1219)
b71ef43 Fix for #203 (#1210)
f246da7 Add missing import to setup.py (#1215)
70862e0 Make pytest_runner a conditional requirement in setup.py (#1203)
10e7e99 Remove argparse from requirements (#1202)
e1fdad6 Added a ConstrainedBeamSearch, switched MML to use it (#1189)
7bd52cb Use README.md for PyPI long description (#1201)
2dda95b Changed how checklist balance is used in action prediction (#1204)
1554fb0 Update test requirements in setup.py, remove pytest-pythonpath (#1207)
a8f7ada Added dropout to the NLVR parser (#1205)
b5af3e4 Prune finished states in ERM (#1187)
77b33fb Add compute model number to Compile custom LSTM for Tesla V100 (#1198)
63f2108 Update ELMo command help docstring (#1195)
8ba5867 Added coverage to WikiTables ERM parser (#1181)
69ae074 Rename tutorials so they're clearly connected. (#1186)
65aecdd Add a tutorial for using the pre-trained models. (#1182)
3add61d check for zero grads only if grad is not none (#1188)
7142962 ERM parser for wikitables (#1159)
c369502 Add missing --all flag to ELMo howto (#1183)
ea2e431 Update using_in_your_repo.md
7cc4f72 Update using_in_your_repo.md
19980b5 Add shebang to bin/allennlp (#1179)
58f9033 Clarify the ELMo readme (#1167)
4f8834c fix comment (#1171)
af283d1 hack to stop tqdm hanging (#1168)
a6d4b1b Merge branch 'release-v0.4.2'
15f6102 bump version numbers in README
60e03e3 remove spaces in SpacyWordSplitter.batch_tokenize (#1156)

v0.4.2

The most immediate change is that now PyTorch is installed via pip, and you don't have to do it separately. We also have new code to support a state-of-the-art NER model.

Here's the full list of commits:

d4b286c Mattp/ner (#1162)
d05673a Add a default cuda_device to model. (#1151)
281d032 Added two feature extractors that were in PNP and not in our version (#1155)
f8d64ef add stacked bilm (#1154)
f2257a6 Use separate action embeddings for prediction vs. input feeding (#1153)
cc510fa Allow ablation where linking features are removed (#1145)
121d3e8 unidecode in setup (#1148)
55b0411 Adding ReLUs after linear projections (#1150)
645379e Allow default predictors when calling programatically (#1112)
6d15d17 Allow to restart with optimizers on CUDA (#1144)
c684216 Streamlining the docker build a little bit (#1140)
bfff55f bug fix for empty decoder outputs (#1141)
b9630e6 Fix state scores in WikiTablesDecoderStep (#1129)
a869e0b Move imports to inside of CACHE_MODELS=true condition (#1133)
53ae735 clean EVALB via make, add to Dockerfile (#1132)
812dfcf Added a script to generate data from an ERM trained model (#1115)
2b81da7 Remove the KeyError catch block in the server (#1130)
960f913 Adding WikiTables and NLVR predictors (#1118)
7627a09 Use xavier_uniform for initializing embedding weights (#1120)
0df8978 Adding the wikitables parser (#1114)
f7b736c increase metrics logging precision (#1113)
7dfc342 Adding NLVR model (#1108)
bc8ac1a Adding WikiTables dataset reader (#1106)
cbe5897 Adding NLVR dataset reader (#1105)
f7924bc Adding KnowledgeGraphField and ProductionRuleField (#1104)
245a9af add name argument to run_with_beaker.py (#1103)
2e6f2dd Adding the type system / logical form parsing code from the wikitables branch (#1087)
eb81ac9 Update elmo.md (#1092)
1655f22 Adding the decoding framework from the wikitables branch (#1086)
3ea82b1 Moving over modifications to common / util modules (#1085)
f7b736c increase metrics logging precision (#1113)
7dfc342 Adding NLVR model (#1108)
bc8ac1a Adding WikiTables dataset reader (#1106)
cbe5897 Adding NLVR dataset reader (#1105)
f7924bc Adding KnowledgeGraphField and ProductionRuleField (#1104)
245a9af add name argument to run_with_beaker.py (#1103)
2e6f2dd Adding the type system / logical form parsing code from the wikitables branch (#1087)
eb81ac9 Update elmo.md (#1092)
1655f22 Adding the decoding framework from the wikitables branch (#1086)
3ea82b1 Moving over modifications to common / util modules (#1085)
78dc1df Data cleanup and additions from the wikitables branch (#1084)
6829edb Move the top nav bar to the left pane. (#1082)
14bb4e3 Revert "Top menu -> Left menu (#1074)" (#1081)
a92bf48 Top menu -> Left menu (#1074)
b72c838 Adding an optional projection layer to ElmoTokenEmbedder (#1076)
4c02d92 Reading glove embeddings: strip() --> rstrip() (#1056)
0a918aa read vocab params in model test when available (#1071)
8ddc8cb initial dry run command (#1063)
da1e6f4 Update setup.py
e8e2499 install pytorch via pip (#1062)
bcbc12e don't use mutable sys.stdout for both logs (#1060)
ebf25ea One line update in file_utiles.py (#1059)
9299131 Add a multilabel field (#1054)
db44a25 Add a get_final_encoder_states function that handles bidirectional encoders (#1048)
77da425 don't allow U tags in BIO to spans, use BIOUL in SpanF1Measure (#1045)
e444216 Add an ensemble model for BiDAF (#1000)
a5a827e Add Biattentive Classification Network (#1038)
78aa071 proper configuration of all dropout in self attention (#1042)
d0bb3d2 CoNLL dataset reader that allows BIOLU tags (#1041)
d57717e srl model tweaks (#1040)
1700db4 Minor enhancements to seq2seq data reader (#1037)
2b55bfb re-jig readme (#1034)
395646a Add Stanford Sentiment Treebank dataset reader (#1032)

v0.4.1

496867c add an initial string representation to fields and instances (#1021)
008ceb2 Break up AWS login command in two to catch failures. (#1022)
a0ced89 reset meteric when done evaluation (#1020)
3ea3e64 fix logging error (#1017)
ab3bd0f add optional exclusive start indices flag to Enpoint Extractor (#1005)
f37bc6c add some new optimisers and a cosine LR schedule (#1012)
a327d03 make srl predictor more robust (#1011)
5f722fc Add .pytest_cache folder to gitignore (#1006)
31f4f60 better error for gradient check (#1010)
f06d62f readme: add port publishing for docker (#1009)
760853c Winobias reader (#968)
3c8e299 fix predictor issue for constituency parser (#1004)
d351ac7 Fix simple tagger link in creating a model tutorial (#1003)
e28415c remove PREDICTOR_OVERRIDES and MODEL_OVERRIDES (#999)
1d68f88 (michael/master) add allennlp command (#987)
0c532d5 use correct PTB results on demo page (#990)
dc7a9d6 make hierplane div overflow so it's scrollable (#989)
565b95c (matt/master) Constituency parser demo (#985)
eb79428 Change input name from "source_string" to "source" in Seq2SeqPredictor (#984)
7b2e09d Fix issue #973 and add predictor for SimpleSeq2Seq (#976)
40d337e Keyboard interrupt + training admin (#983)
c7a4e6d don't require that EVALB is compiled on instantiation (#980)
5c663c4 fix span construction (#975)
3d100d3 Fix max_vocab_size bug (#964) (#966)
18eb421 fix sentence encoder from params (#963)
9121eed Move --include-packages flag up to top level (#962)
cc2756d Add a link to the span representations tutorial (#959)
0ef211d fix online metric calculation for EVALB (#956)
c84c714 use biased estimator for layernorm, fix some views in multi head attention (#953)
71c80e3 Install nltk the normal way. (#932)
6044cce Allow to provide a custom Module in Elmo class (#945)
e657353 add label smoothing to loss (#942)
f81e27a get the dimensions correct for the transformer (#941)
c12d435 Automatically install spacy models, if missing (#933)
cdfe9ae add ImportError to lstm import (#938)
c66fac0 pos -> tag (#935)
dcd6487e More parser improvements (#934)
acf21a5 Fix #928 - Minor naming inconsistency in tutorial json files (#929)
0cb2f60 speed up metrics (#927)
394c26d add try except block with helpful message (#923)
97bc7a2 Combined recovery logic, made it not crash on beaker (#925)
de5df62 Fix random hash that's appened to git SHA (#924)
b7a9784 Nested constituents (#920)
085a750 use gold pos tags for tree evaluation (#921)
8b706e4 Constituency js (#916)
740f4fb Constituency Parser Predictor and tests (#914)
6a10857 Constituency training pr (#913)
6ad7d5b Migrating to 0.4.0 (#909)
bca992e Fix span masking (#905)
6f4de85 Update TextualEntailment text to highlight ELMo. (#906)
b891d37 (log-learning-rate) Merge branch 'release-v0.4.0'
5756ff6 Span tutorial (#903)

v0.4.0

New features:

• A major feature in the 0.4 release is the inclusion of ELMo which produces contextualized word embeddings that greatly improve model performance. You can read more in our ELMo HowTo.
• Support for lazy datasets, so you can stream data through the trainer with a lower memory footprint. This is a breaking change for some parts of the API; if you've written a DatasetReader, you will probably need to change a little bit of code. The Dataset class is now gone.
• First-class support for models that operate on spans instead of on tokens.
• Support for programmatically importing additional dependencies so you don't need to write your own run.py script.
• A simple server to create a stand-alone web demo for your model.
• Added constrained decoding to the ConditionalRandomField module (and to the corresponding NER tagger model)

Additional tutorials:

• A tutorial on writing Predictors for using python -m allennlp.run predict and for creating demos.
• Instructions for how to visualize model internals in a live demo of your model, for gaining better insights about what your model is doing.
• A tutorial on how laziness works in AllenNLP.

Additional models / dataset readers:

• A span-based constituency parser that independently predicts a non-terminal label for each span in an input sentence, along with a Penn treebank dataset reader that reads data for this model. Trained model and demo coming soon.

Minor bugfixes and features:

• This release is compatible with pytorch 0.3.1 (and still compatible with pytorch 0.3.0).
• Made it possible to do batch tokenization with spacy inside a DatasetReader.
• Added a make-vocab command to precompute a vocabulary for a dataset.
• Added a fine-tune command to fine-tune a trained model on a new dataset.
• Unified handling of Ontonotes-based datasets, so it is now easier to write new DatasetReaders that use Ontonotes.
• Predictors now support non-json formats for bulk prediction.
• More flexible batching code, and bug fixes when batching / padding using ListFields.
• Added ability to handle different data reading configurations at train and test time.

Breaking Changes

This release contains several breaking changes. Please see the migration guide if you have pre-0.4.0 code you need to update.

v0.3.0

We updated our key dependencies to Spacy 2.0 and PyTorch 0.3, and we have a few additional models and many new features since our 0.2 release.

Additional models. More details for our models are available at http://allennlp.org/models and you can use them interactively online at http://demo.allennlp.org/.

• The baseline NER model from Semi-supervised Sequence Tagging with Bidirectional Language Models.
• A coreference model, based on the publication End-to-end Neural Coreference Resolution, which achieved state-of-the-art performance in early 2017.

Additional examples and tutorials. Find them at http://allennlp.org/tutorials/installation.

• A tutorial on using AllenNLP in your project as a pip dependency.
• Example code for training a seq2seq model.

New features.

• Improved SRL visualization on the demo.
• ListField padding fixes.

v0.2.3

• Fixes bugs in stateful versions of Seq2XXXEncoders.
• Adds an ArrayField for array inputs to models.
• Uses an up to date stand-alone version of tensorboard such that you can use it inside a single environment.
• Adds the ability to read hdf5 formatted embedding matrices.
• Adds a flag to specify what Datasets should be used to build vocabularies in train.

v0.2.0

The first release of AllenNLP using PyTorch.

This release of AllenNLP includes three models:

• Semantic Role Labeling (78.9 dev F1, 78.9 test F1, CoNLL 2012)
• Machine Comprehension (68.7 EM, SQuAD)
• Textual Entailment (84.7 test accuracy, SNLI)