Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HeroTypes in Representation; DataFrame in _types #157

Merged
Merged
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fa342a9
added MultiIndex DF support
mk2510 Aug 18, 2020
59a9f8c
beginning with tests
henrifroese Aug 19, 2020
19c52de
implemented correct sparse support
mk2510 Aug 19, 2020
66e566c
Merge branch 'master_upstream' into change_representation_to_multicolumn
mk2510 Aug 21, 2020
41f55a8
added back list() and rm .tolist()
mk2510 Aug 21, 2020
217611a
rm .tolist() and added list()
mk2510 Aug 21, 2020
6a3b56d
Adopted the test to the new dataframes
mk2510 Aug 21, 2020
b8ff561
wrong format
mk2510 Aug 21, 2020
e3af2f9
Address most review comments.
henrifroese Aug 21, 2020
77ad80e
Add more unittests for representation
henrifroese Aug 21, 2020
f7eb7c3
- Update _types.py with DocumentTermDF
henrifroese Aug 22, 2020
4937a4f
Fix DocumentTermDF example DataFrame column names
henrifroese Aug 22, 2020
e2768b5
implemented the suggested changes
mk2510 Sep 4, 2020
b09f624
fixed messy docstring
mk2510 Sep 4, 2020
508c361
fix black issues
mk2510 Sep 4, 2020
75e955f
fix formatting
mk2510 Sep 4, 2020
9ca244d
begin switch away from multiindex
henrifroese Sep 4, 2020
4ebc266
Merge remote-tracking branch 'origin/change_representation_to_multico…
henrifroese Sep 4, 2020
559a7bd
Finish switch from DocumentTermDF to MatrixDF
henrifroese Sep 4, 2020
75a999c
changed Dataframe name from MatrixDF to DataFrame
mk2510 Sep 4, 2020
a38a32b
merge master
henrifroese Sep 12, 2020
c7b0ece
apply stash
henrifroese Sep 12, 2020
4aeec2a
finish switch to DataFrame type
henrifroese Sep 12, 2020
a304413
Merge remote-tracking branch 'origin/Hero_Types_in_Representation' in…
henrifroese Sep 12, 2020
8d49bff
merge remote
henrifroese Sep 12, 2020
85076b8
Remove check for nlevels
henrifroese Sep 12, 2020
91ed11a
incorporate suggested changes
henrifroese Sep 14, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Adopted the test to the new dataframes
  • Loading branch information
mk2510 committed Aug 21, 2020
commit 6a3b56d1a56401880efa7cfa7dd32668e23b25ea
16 changes: 12 additions & 4 deletions tests/test_representation.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ def _get_multiindex_for_tokenized_output(first_level_name):
_tfidf(x, s_tokenized, 0) # Testing the tfidf formula here
for x in ["!", ".", "?", "TEST", "Test"]
],
[_tfidf(x, s_tokenized, 0) for x in ["!", ".", "?", "TEST", "Test"]],
[_tfidf(x, s_tokenized, 1) for x in ["!", ".", "?", "TEST", "Test"]],
],
index=s_tokenized_output_index,
columns=_get_multiindex_for_tokenized_output("tfidf"),
Expand Down Expand Up @@ -146,20 +146,28 @@ class AbstractRepresentationTest(PandasTestCase):
def test_vectorization_simple(self, name, test_function, correct_output):
s_true = correct_output
result_s = test_function(s_tokenized)
pd.testing.assert_frame_equal(s_true, result_s, check_less_precise=True, check_dtype = False)
pd.testing.assert_frame_equal(
s_true, result_s, check_less_precise=True, check_dtype=False
)

@parameterized.expand(test_cases_vectorization)
def test_vectorization_noncontinuous_index_kept(
self, name, test_function, correct_output=None
):
result_s = test_function(s_tokenized_with_noncontinuous_index)
pd.testing.assert_frame_equal(s_tokenized_output_index_noncontinous, result_s.index, check_dtype = False)
pd.testing.assert_series_equal(
pd.Series(s_tokenized_output_index_noncontinous),
pd.Series(result_s.index),
check_dtype=False,
)

@parameterized.expand(test_cases_vectorization_min_df)
def test_vectorization_min_df(self, name, test_function, correct_output):
s_true = correct_output
result_s = test_function(s_tokenized, min_df=2)
pd.testing.assert_frame_equal(s_true, result_s, check_less_precise=True, check_dtype = False)
pd.testing.assert_frame_equal(
s_true, result_s, check_less_precise=True, check_dtype=False
)

@parameterized.expand(test_cases_vectorization)
def test_vectorization_not_tokenized_yet_warning(self, name, test_function, *args):
Expand Down