Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW] Creating dedicated loader submodule to build TF async dataloader #224

Merged
merged 114 commits into from
Aug 27, 2020
Merged

[REVIEW] Creating dedicated loader submodule to build TF async dataloader #224

Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
c65c269
adding tensorflow example stuff
Jun 22, 2020
041b083
getting workflow working
Jun 23, 2020
54eebaa
training of both workflows works
Jun 23, 2020
3358cb2
notebook updates and addding image from run
Jul 1, 2020
5e27b76
updating workflow for nightly tf build
Jul 2, 2020
5ac8ab1
Create dummy.txt
alecgunny Jul 3, 2020
b899185
Add files via upload
alecgunny Jul 3, 2020
dc315aa
Delete dummy.txt
alecgunny Jul 3, 2020
79ba66f
adding tensorflow example stuff
Jun 22, 2020
5ba7093
getting workflow working
Jun 23, 2020
ecc50e7
training of both workflows works
Jun 23, 2020
5952ae5
notebook updates and addding image from run
Jul 1, 2020
a48bebd
adding root Dockerfile
Jul 20, 2020
f8ac57c
updating root build for 2.3 rc1
Jul 20, 2020
7960ac1
updating Dockerfile for tf 2.3-rc1 and filling out notebook
Jul 20, 2020
d8c4b3f
updating throughput curves in README
Jul 20, 2020
163db09
moving dlrm-train
Jul 20, 2020
7b879e8
cleaning up notebook and layers code, adding cupti symlink to Dockerfile
Jul 22, 2020
7e878e3
getting rid of modprobe install in Dockerfile
Jul 22, 2020
470b2ed
playing with requirements
Jul 22, 2020
3440d9e
updating for tf 2.3 full release
Jul 27, 2020
3bf0c2c
updating notebook
Jul 27, 2020
9f3df35
removing old Dockerfiles, updating environment and README and finishi…
Jul 30, 2020
11833f9
removing old images
Jul 30, 2020
21bc8d3
consolidating data loading code
Aug 17, 2020
59e5a45
cleaning up and blackening
Aug 18, 2020
16f9d75
finished separating loader code
Aug 18, 2020
803aea5
adding fixed Dockerfile
Aug 18, 2020
a83da4b
getting tf data loading running
Aug 18, 2020
f529a43
blackening
Aug 18, 2020
b812b97
fixing bug in torch loader
Aug 18, 2020
56240d5
applying isort fixes
Aug 18, 2020
220d53f
isort fixes
Aug 18, 2020
6543a33
ironing out data loaders
Aug 18, 2020
52586c0
Merge branch 'tfasync' of github.com:alecgunny/NVTabular into tfasync
Aug 18, 2020
7ebac2c
creating parent dataloader class
Aug 18, 2020
c69db1e
playing with thread safe iteration
Aug 19, 2020
c2c42f0
small change
Aug 19, 2020
3836d12
Merge branch 'tfasync' of https://github.com/alecgunny/nvtabular into…
Aug 19, 2020
8327766
moving tensoritr loop into asynciterator
Aug 19, 2020
ad91f4f
fixing syntax error
Aug 19, 2020
a6ab0c5
debugging iter issues
Aug 19, 2020
71da3ad
fixing generator issues
Aug 19, 2020
da557c3
cleaning up backend code
Aug 19, 2020
902f64c
got torch data loader working
Aug 19, 2020
0adebe0
working out tf missing gradient issues
Aug 19, 2020
cf74d8d
working on gradient issues
Aug 19, 2020
aa38d9f
reformatting loader backend to use only 2 classes
Aug 19, 2020
8106f06
undoing changes really quick
Aug 19, 2020
d9da3db
merging
Aug 19, 2020
834976b
backend changes
Aug 19, 2020
54ac62f
getting tf dataloader working
Aug 20, 2020
fb64bea
trying out tensor y
Aug 21, 2020
b8a7df6
tf data loader working
Aug 21, 2020
ff7ea5a
undoing some testing changes to Tensorflow
Aug 21, 2020
ae36087
rerunning tf example for checks
Aug 22, 2020
55921b2
updating tests
Aug 22, 2020
34372d2
blackening
Aug 22, 2020
4ba5e2b
blackening
Aug 22, 2020
90790a4
fixing dataloader bench bug
Aug 22, 2020
7d1d583
fixing unused variables
Aug 22, 2020
2c02a80
isort fixes
Aug 22, 2020
03e880e
adding qsize to chunkedbuffer
Aug 22, 2020
2650cb6
fixed typo in backed
Aug 22, 2020
c5105a7
simplifying and updating DataLoader
Aug 24, 2020
4d229ac
updating dataloader backend
Aug 24, 2020
5029bc7
trying new async scheme
Aug 24, 2020
677ac76
got new implementation working
Aug 24, 2020
157d6f9
Merge branch 'tfasync' of https://github.com/alecgunny/nvtabular into…
Aug 24, 2020
550afa0
fixing merge conflict
Aug 24, 2020
fb77dce
cleaning up
Aug 24, 2020
e24e41d
blackening
Aug 24, 2020
fa2000f
fixing bugs
Aug 24, 2020
74f86b0
updating wait time
Aug 24, 2020
2b0c8ff
isort fixes
Aug 24, 2020
329e270
Merge branch 'tfasync' of https://github.com/alecgunny/nvtabular into…
Aug 24, 2020
dbc70f0
minor aesthetic change
Aug 25, 2020
90b58fa
removing merge conflicts
Aug 25, 2020
a2d280e
merging upstream changes
Aug 25, 2020
ed697bc
bug fixes
Aug 25, 2020
e0ebe3b
fixing merge conflict
Aug 25, 2020
fedb834
trying to update examples
Aug 25, 2020
64d0f3e
adding custom validation callback
Aug 25, 2020
5137031
got examples working
Aug 25, 2020
c8514be
blackening
Aug 25, 2020
ebe03d1
fixing bug and documenting
Aug 25, 2020
9cfc493
gettin criteo most of the way through
Aug 25, 2020
03f8c5b
Merge branch 'tfasync' of github.com:alecgunny/NVTabular into tfasync
Aug 25, 2020
13d2588
fixing merge conflicts
Aug 25, 2020
198cda8
rearranging and adding checks
Aug 26, 2020
6407b88
Merge branch 'tfasync' of github.com:alecgunny/NVTabular into tfasync
Aug 26, 2020
bb21643
adding proper torch documentation
Aug 26, 2020
7d3417a
removing old torch dataloader
Aug 26, 2020
4e8af42
documenting and blackening
Aug 26, 2020
1562287
remove trailing whitespace
Aug 26, 2020
47c9266
updating tests
Aug 26, 2020
cc2d75c
changing cat and cont defaults to empty lists and including checks
Aug 26, 2020
c2d90f7
updating TF example notebook
Aug 26, 2020
f272dc9
Merge branch 'tfasync' of github.com:alecgunny/NVTabular into tfasync
Aug 26, 2020
bb63cc5
adding PARTS_PER_CHUNK to criteo example
Aug 26, 2020
c403e74
adding tf config changes
Aug 26, 2020
760a837
Merge branch 'tfasync' of https://github.com/alecgunny/nvtabular into…
Aug 26, 2020
716825d
fixing tf unit tests
Aug 26, 2020
056155e
blackening
Aug 26, 2020
9c82df5
fixed tf_util bug
Aug 26, 2020
933b719
fixing tf_utils bug
Aug 26, 2020
3db05b0
blackening
Aug 26, 2020
9458a8d
blackening
Aug 26, 2020
6f543f9
fixing bug in loader backend
Aug 26, 2020
1042d96
tests passing
Aug 27, 2020
05b9a25
blackening
Aug 27, 2020
3b974e0
updating rossmann notebook test
Aug 27, 2020
32f15f8
Merge branch 'tfasync' of github.com:alecgunny/NVTabular into tfasync
Aug 27, 2020
372520b
Fix cupy device errors
benfred Aug 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
.git
.dockerignore

# ignore hidden directories created
# by RAPIDS libs
**/.cupy
**/.nv
**/.python_history

# ignore any local files created
# by examples notebooks
examples/tensorflow/logs
examples/tensorflow/docker/Dockerfile*
examples/tensorflow/.*
Expand Down
Loading