Wrapper for TagAs did not work #1462

bschifferer · 2022-03-17T16:39:52Z

I got following error using TagAsUser, TagAsUserId, etc.

ERROR:nvtabular:Failed to transform operator <nvtabular.ops.add_metadata.TagAsItemID object at 0x7f8e180f5070>
Traceback (most recent call last):
  File "/nvtabular/nvtabular/workflow/workflow.py", line 466, in _transform_partition
    output_df = node.op.transform(selection, input_df)
  File "/nvtabular/nvtabular/ops/operator.py", line 45, in transform
    raise NotImplementedError
NotImplementedError

I changed the implementation

First draft of the API overhauls changes. Adds most core functionality, including defining workflow graphs with a ColumnGroup class, the workflow and dataset changes , most operators converted to use the new api, etc.

Also partially fix some tests inside test_workflow

Co-authored-by: Richard (Rick) Zamora <rzamora217@gmail.com>

We should be doing online transforms like ```KerasSequenceLoader(workflow.transform(dataset), ...``` instead of ```KerasSequenceLoader(dataset, workflows=[workflow], ...``` now

* test_minmix * updates test * unittest ops

Re-add get_embedding_sizes . Note that this changes how we support multi-hot columns here (sizes are returned same as single hot, and we don't use this method to distinguish between multi and singlehot columns)

add save_stats/load_stats/clear_stats methods to the workflow, with each statoperator getting called as appropiate

git merge upstream/main Merge branch 'main' of https://github.com/NVIDIA/NVTabular

github-actions · 2022-03-17T16:50:32Z

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1462

nvidia-merlin-bot · 2022-03-17T17:05:51Z

Click to view CI Results

GitHub pull request #1462 of commit 92c7485f6c174eaf16762ccbcb3f6ce216f7cc12, no merge conflicts.
Running as SYSTEM
Setting status of 92c7485f6c174eaf16762ccbcb3f6ce216f7cc12 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4333/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1462/*:refs/remotes/origin/pr/1462/* # timeout=10
 > git rev-parse 92c7485f6c174eaf16762ccbcb3f6ce216f7cc12^{commit} # timeout=10
Checking out Revision 92c7485f6c174eaf16762ccbcb3f6ce216f7cc12 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 92c7485f6c174eaf16762ccbcb3f6ce216f7cc12 # timeout=10
Commit message: "fix tagas"
 > git rev-list --no-walk 7dd03a1868f94851e9d1bb8942e15706e8a0866c # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins141249295964363336.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.1)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+19.ge1f0d6e
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+10.gc2aaca5
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-i11u42j2/merlin-core_5ba526bb12ed42d1a88dd29faec8fa44
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-i11u42j2/merlin-core_5ba526bb12ed42d1a88dd29faec8fa44
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 53b1ffc0b135c1dddc7f89c6686da062cedf8dab
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: pandas>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.4.1)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.19.4)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.0)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (60.10.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.3)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.55.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.1.0)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.1.1+12.g53b1ffc-py3-none-any.whl size=128879 sha256=3aa236dadd538238acddd9b4e17e0e056305d8fa83e7889681453cb477cad596
  Stored in directory: /tmp/pip-ephem-wheel-cache-7510e_ar/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
Successfully installed merlin-core-0.1.1+12.g53b1ffc
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'merlin/systems/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/cpp
creating build/temp.linux-x86_64-3.8/cpp/nvtabular
creating build/temp.linux-x86_64-3.8/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+52.g92c7485 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+52.g92c7485 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+52.g92c7485 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+52.g92c7485 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-3.8
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 0.11.0+52.g92c7485 is already the active version in easy-install.pth
Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

165 files would be left unchanged.

Running flake8

Running isort

Skipped 3 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.0, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1422 items / 1 skipped
tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

................................                                         [ 39%]

tests/unit/ops/test_column_similarity.py ........................        [ 41%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 41%]

tests/unit/ops/test_fill.py ............................................ [ 44%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py .........                                [ 45%]

tests/unit/ops/test_hash_bucket.py .........................             [ 47%]

tests/unit/ops/test_join.py ............................................ [ 50%]

........................................................................ [ 55%]

..................................                                       [ 58%]

tests/unit/ops/test_lambda.py ..........                                 [ 58%]

tests/unit/ops/test_normalize.py ....................................... [ 61%]

..                                                                       [ 61%]

tests/unit/ops/test_ops.py ............................................. [ 64%]

....................                                                     [ 66%]

tests/unit/ops/test_ops_schema.py ...................................... [ 68%]

........................................................................ [ 74%]

........................................................................ [ 79%]

........................................................................ [ 84%]

.......................................                                  [ 86%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 87%]

tests/unit/ops/test_target_encode.py .....................               [ 88%]

tests/unit/systems/test_ensemble.py ....                                 [ 88%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 88%]

tests/unit/systems/test_export.py .                                      [ 89%]

tests/unit/systems/test_graph.py .                                       [ 89%]

tests/unit/systems/test_inference_ops.py ..                              [ 89%]

tests/unit/systems/test_op_runner.py ....                                [ 89%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):
../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)
../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)
../../../../../core/merlin/io/fsspec_utils.py:26

/core/merlin/io/fsspec_utils.py:26: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(fsspec.version).version[:3] > [2021, 11, 0]:
tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 731 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 97 warnings

tests/unit/ops/test_categorify.py: 65 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 96 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

tests/unit/workflow/test_workflow.py: 44 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(
tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(
tests/unit/test_tools.py: 12 warnings

tests/unit/ops/test_ops.py: 68 warnings

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(
tests/unit/test_triton_inference.py: 10 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/inference/triton/data_conversions.py:125: FutureWarning: The series.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.

offsets = pd.Series([0]).append(col.map(len).cumsum()).values
tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(
tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(
tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(
tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_fill.py:72: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_join.py: 96 warnings

/var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg/partd/pandas.py:113: DeprecationWarning: The Index._get_attributes_dict method is deprecated, and will be removed in a future version

header = (type(ind), ind._get_attributes_dict(), values.dtype, cat)
tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_ops.py:135: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(
tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(
tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing
examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

merlin/systems/dag/init.py                                     3      0      0      0   100%

merlin/systems/dag/ensemble.py                                    64      2     30      2    96%   79, 81

merlin/systems/dag/node.py                                        30     10     12      1    69%   31-32, 35-36, 50, 57-62

merlin/systems/dag/op_runner.py                                   21      0      8      0   100%

merlin/systems/dag/ops/init.py                                 0      0      0      0   100%

merlin/systems/dag/ops/operator.py                                62      5     18      1    90%   24-25, 30, 50, 59

merlin/systems/dag/ops/session_filter.py                          43     11      8      3    73%   35-38, 56, 61, 68, 84-88

merlin/systems/dag/ops/softmax_sampling.py                        50     17      4      1    67%   23-29, 60, 78-108

merlin/systems/dag/ops/tensorflow.py                              73     10     24      4    84%   39-40, 48->57, 86, 130-139

merlin/systems/dag/ops/workflow.py                                26      0      2      0   100%

merlin/systems/triton/init.py                                 36     12     14      1    58%   42-49, 68, 72, 76-82

merlin/systems/triton/conversions.py                              87     73     58      0    10%   32-33, 52-84, 88-94, 98-105, 109-115, 119-136, 140-150

merlin/systems/triton/export.py                                  266    154     82      9    42%   89-93, 156-195, 239-287, 380-388, 414, 417-433, 437-441, 454-455, 485-495, 547-587, 593-609, 613-680, 686-693, 732, 738-757, 763-787, 794

merlin/systems/triton/model_config_pb2.py                        299      0      2      0   100%

merlin/systems/workflow/init.py                               22      5     12      3    65%   43, 51, 63-65

nvtabular/init.py                                             19      2      0      0    89%   40-41

nvtabular/dispatch.py                                              1      0      0      0   100%

nvtabular/framework_utils/init.py                              0      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     12     89      6    91%   60, 68->49, 122, 179, 231-239, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 1      0      0      0   100%

nvtabular/inference/init.py                                    0      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      52     52     22      0     0%   27-124

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    1      0      0      0   100%

nvtabular/loader/init.py                                       0      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   32->35, 35->37, 42->44, 46, 47->68, 53-54, 62-64, 70-74, 89-94, 104-117

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      657     71    350     50    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 389->393, 397-404, 450, 458, 481-482, 559-564, 635, 731, 748, 793, 871-872, 887-891, 892->856, 910, 918, 925->exit, 949, 952->955, 1004->1002, 1066-1068, 1071, 1073->1075, 1092->1096, 1098->1051, 1104-1107, 1119, 1123, 1127, 1134, 1139-1142, 1220, 1222, 1292->1315, 1298->1315, 1316-1321, 1366, 1379->1382, 1386->1391, 1390, 1396, 1399, 1407-1417

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   91->93, 95, 97->87

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             21      0      8      1    97%   41->46

nvtabular/ops/dropna.py                                            8      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         130      4     84      5    96%   72, 83, 93->95, 131, 221

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   73, 112->118

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   59, 63, 81, 93-108

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                44      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   45, 70-72

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 130-133, 143-147, 149, 171-172, 323, 333, 359, 362-371

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 1      0      0      0   100%

nvtabular/worker.py                                                1      0      0      0   100%

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   218     17     94     12    91%   28-29, 51, 84, 167, 173->187, 200-202, 326, 341-342, 378, 455, 485, 493-495, 508
TOTAL                                                           6387   1512   2371    239    74%

Coverage XML written to file coverage.xml
Required test coverage of 70% reached. Total coverage: 73.78%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========= 1421 passed, 2 skipped, 1612 warnings in 1377.39s (0:22:57) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins4127065212251164864.sh

nvidia-merlin-bot · 2022-03-17T19:36:22Z

Click to view CI Results

GitHub pull request #1462 of commit 81aebae373ac1503b297a766ef6cb884fabad7fb, no merge conflicts.
Running as SYSTEM
Setting status of 81aebae373ac1503b297a766ef6cb884fabad7fb to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4338/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1462/*:refs/remotes/origin/pr/1462/* # timeout=10
 > git rev-parse 81aebae373ac1503b297a766ef6cb884fabad7fb^{commit} # timeout=10
Checking out Revision 81aebae373ac1503b297a766ef6cb884fabad7fb (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 81aebae373ac1503b297a766ef6cb884fabad7fb # timeout=10
Commit message: "Merge branch 'main' into fix_tagas"
 > git rev-list --no-walk 6697f9d784fb69986af87e54d2cba49c0b3e5204 # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins6912570038030099360.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.1)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+19.ge1f0d6e
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+10.gc2aaca5
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-u5qykkrv/merlin-core_abb81bfa777c4c3687b8ac6208887ed7
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-u5qykkrv/merlin-core_abb81bfa777c4c3687b8ac6208887ed7
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 53b1ffc0b135c1dddc7f89c6686da062cedf8dab
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.19.4)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: pandas>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.4.1)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (60.10.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.3)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.55.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.1.0)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.1.1+12.g53b1ffc-py3-none-any.whl size=128879 sha256=9a568d2f509ac6e9e0265a0b40dc6170c5d7cbe106fd0db9190d81ac4c47afae
  Stored in directory: /tmp/pip-ephem-wheel-cache-goou_96o/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
Successfully installed merlin-core-0.1.1+12.g53b1ffc
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'merlin/systems/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/cpp
creating build/temp.linux-x86_64-3.8/cpp/nvtabular
creating build/temp.linux-x86_64-3.8/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+54.g81aebae -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+54.g81aebae -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+54.g81aebae -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+54.g81aebae -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-3.8
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 0.11.0+54.g81aebae is already the active version in easy-install.pth
Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

165 files would be left unchanged.

Running flake8

Running isort

Skipped 3 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.0, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1422 items / 1 skipped
tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

................................                                         [ 39%]

tests/unit/ops/test_column_similarity.py ........................        [ 41%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 41%]

tests/unit/ops/test_fill.py ............................................ [ 44%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py .........                                [ 45%]

tests/unit/ops/test_hash_bucket.py .........................             [ 47%]

tests/unit/ops/test_join.py ............................................ [ 50%]

........................................................................ [ 55%]

..................................                                       [ 58%]

tests/unit/ops/test_lambda.py ..........                                 [ 58%]

tests/unit/ops/test_normalize.py ....................................... [ 61%]

..                                                                       [ 61%]

tests/unit/ops/test_ops.py ............................................. [ 64%]

....................                                                     [ 66%]

tests/unit/ops/test_ops_schema.py ...................................... [ 68%]

........................................................................ [ 74%]

........................................................................ [ 79%]

........................................................................ [ 84%]

.......................................                                  [ 86%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 87%]

tests/unit/ops/test_target_encode.py .....................               [ 88%]

tests/unit/systems/test_ensemble.py ....                                 [ 88%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 88%]

tests/unit/systems/test_export.py .                                      [ 89%]

tests/unit/systems/test_graph.py .                                       [ 89%]

tests/unit/systems/test_inference_ops.py ..                              [ 89%]

tests/unit/systems/test_op_runner.py ....                                [ 89%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):
../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)
../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)
../../../../../core/merlin/io/fsspec_utils.py:26

/core/merlin/io/fsspec_utils.py:26: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(fsspec.version).version[:3] > [2021, 11, 0]:
tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 731 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 97 warnings

tests/unit/ops/test_categorify.py: 65 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 96 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

tests/unit/workflow/test_workflow.py: 44 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(
tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(
tests/unit/test_tools.py: 12 warnings

tests/unit/ops/test_ops.py: 68 warnings

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(
tests/unit/test_triton_inference.py: 10 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/inference/triton/data_conversions.py:125: FutureWarning: The series.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.

offsets = pd.Series([0]).append(col.map(len).cumsum()).values
tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(
tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(
tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(
tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_fill.py:72: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_join.py: 96 warnings

/var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg/partd/pandas.py:113: DeprecationWarning: The Index._get_attributes_dict method is deprecated, and will be removed in a future version

header = (type(ind), ind._get_attributes_dict(), values.dtype, cat)
tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_ops.py:134: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(
tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(
tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing
examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

merlin/systems/dag/init.py                                     3      0      0      0   100%

merlin/systems/dag/ensemble.py                                    64      2     30      2    96%   79, 81

merlin/systems/dag/node.py                                        30     10     12      1    69%   31-32, 35-36, 50, 57-62

merlin/systems/dag/op_runner.py                                   21      0      8      0   100%

merlin/systems/dag/ops/init.py                                 0      0      0      0   100%

merlin/systems/dag/ops/operator.py                                62      5     18      1    90%   24-25, 30, 50, 59

merlin/systems/dag/ops/session_filter.py                          43     11      8      3    73%   35-38, 56, 61, 68, 84-88

merlin/systems/dag/ops/softmax_sampling.py                        50     17      4      1    67%   23-29, 60, 78-108

merlin/systems/dag/ops/tensorflow.py                              73     10     24      4    84%   39-40, 48->57, 86, 130-139

merlin/systems/dag/ops/workflow.py                                26      0      2      0   100%

merlin/systems/triton/init.py                                 36     12     14      1    58%   42-49, 68, 72, 76-82

merlin/systems/triton/conversions.py                              87     73     58      0    10%   32-33, 52-84, 88-94, 98-105, 109-115, 119-136, 140-150

merlin/systems/triton/export.py                                  266    154     82      9    42%   89-93, 156-195, 239-287, 380-388, 414, 417-433, 437-441, 454-455, 485-495, 547-587, 593-609, 613-680, 686-693, 732, 738-757, 763-787, 794

merlin/systems/triton/model_config_pb2.py                        299      0      2      0   100%

merlin/systems/workflow/init.py                               22      5     12      3    65%   43, 51, 63-65

nvtabular/init.py                                             19      2      0      0    89%   40-41

nvtabular/dispatch.py                                              1      0      0      0   100%

nvtabular/framework_utils/init.py                              0      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     12     89      6    91%   60, 68->49, 122, 179, 231-239, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 1      0      0      0   100%

nvtabular/inference/init.py                                    0      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      52     52     22      0     0%   27-124

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    1      0      0      0   100%

nvtabular/loader/init.py                                       0      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 125, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   32->35, 35->37, 42->44, 46, 47->68, 53-54, 62-64, 70-74, 89-94, 104-117

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      657     71    350     50    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 389->393, 397-404, 450, 458, 481-482, 559-564, 635, 731, 748, 793, 871-872, 887-891, 892->856, 910, 918, 925->exit, 949, 952->955, 1004->1002, 1066-1068, 1071, 1073->1075, 1092->1096, 1098->1051, 1104-1107, 1119, 1123, 1127, 1134, 1139-1142, 1220, 1222, 1292->1315, 1298->1315, 1316-1321, 1366, 1379->1382, 1386->1391, 1390, 1396, 1399, 1407-1417

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   91->93, 95, 97->87

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             21      0      8      1    97%   41->46

nvtabular/ops/dropna.py                                            8      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         130      4     84      5    96%   72, 83, 93->95, 131, 221

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   73, 112->118

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   59, 63, 81, 93-108

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                44      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   45, 70-72

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 130-133, 143-147, 149, 171-172, 323, 333, 359, 362-371

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 1      0      0      0   100%

nvtabular/worker.py                                                1      1      0      0     0%   20

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   218     17     94     12    91%   28-29, 51, 84, 167, 173->187, 200-202, 326, 341-342, 378, 455, 485, 493-495, 508
TOTAL                                                           6387   1513   2371    239    74%

Coverage XML written to file coverage.xml
Required test coverage of 70% reached. Total coverage: 73.77%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========= 1421 passed, 2 skipped, 1612 warnings in 1421.45s (0:23:41) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins4221035422464450553.sh

nvidia-merlin-bot · 2022-03-17T21:55:11Z

Click to view CI Results

GitHub pull request #1462 of commit f2232512e916a6934bea5b361b9567153690dcb5, no merge conflicts.
Running as SYSTEM
Setting status of f2232512e916a6934bea5b361b9567153690dcb5 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4340/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1462/*:refs/remotes/origin/pr/1462/* # timeout=10
 > git rev-parse f2232512e916a6934bea5b361b9567153690dcb5^{commit} # timeout=10
Checking out Revision f2232512e916a6934bea5b361b9567153690dcb5 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2232512e916a6934bea5b361b9567153690dcb5 # timeout=10
Commit message: "Merge branch 'main' into fix_tagas"
 > git rev-list --no-walk bb8f3754887e29b5e060a90a544262996acc4262 # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins3594931688306975324.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.10.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.1)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+19.ge1f0d6e
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+10.gc2aaca5
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-fuk032cx/merlin-core_09df8c0f61b04416b15e22ab3cbb2cee
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-fuk032cx/merlin-core_09df8c0f61b04416b15e22ab3cbb2cee
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 53b1ffc0b135c1dddc7f89c6686da062cedf8dab
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pandas>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.4.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.19.4)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (60.10.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.3)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.55.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.1.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.1.1+12.g53b1ffc-py3-none-any.whl size=128879 sha256=088cbf2f5bdd05799772303d385ccddfbee0d257df31fc1c41bf9a4624f3ee1e
  Stored in directory: /tmp/pip-ephem-wheel-cache-p80clq2e/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
Successfully installed merlin-core-0.1.1+12.g53b1ffc
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'merlin/systems/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/cpp
creating build/temp.linux-x86_64-3.8/cpp/nvtabular
creating build/temp.linux-x86_64-3.8/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+56.gf223251 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+56.gf223251 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+56.gf223251 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=0.11.0+56.gf223251 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-3.8
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-3.8/cpp/nvtabular/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-3.8/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.8/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 0.11.0+56.gf223251 is already the active version in easy-install.pth
Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

165 files would be left unchanged.

Running flake8

Running isort

Skipped 3 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.0, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1422 items / 1 skipped
tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

................................                                         [ 39%]

tests/unit/ops/test_column_similarity.py ........................        [ 41%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 41%]

tests/unit/ops/test_fill.py ............................................ [ 44%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py .........                                [ 45%]

tests/unit/ops/test_hash_bucket.py .........................             [ 47%]

tests/unit/ops/test_join.py ............................................ [ 50%]

........................................................................ [ 55%]

..................................                                       [ 58%]

tests/unit/ops/test_lambda.py ..........                                 [ 58%]

tests/unit/ops/test_normalize.py ....................................... [ 61%]

..                                                                       [ 61%]

tests/unit/ops/test_ops.py ............................................. [ 64%]

....................                                                     [ 66%]

tests/unit/ops/test_ops_schema.py ...................................... [ 68%]

........................................................................ [ 74%]

........................................................................ [ 79%]

........................................................................ [ 84%]

.......................................                                  [ 86%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 87%]

tests/unit/ops/test_target_encode.py .....................               [ 88%]

tests/unit/systems/test_ensemble.py ....                                 [ 88%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 88%]

tests/unit/systems/test_export.py .                                      [ 89%]

tests/unit/systems/test_graph.py .                                       [ 89%]

tests/unit/systems/test_inference_ops.py ..                              [ 89%]

tests/unit/systems/test_op_runner.py ....                                [ 89%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):
../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)
../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)
../../../../../core/merlin/io/fsspec_utils.py:26

/core/merlin/io/fsspec_utils.py:26: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(fsspec.version).version[:3] > [2021, 11, 0]:
tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 731 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 97 warnings

tests/unit/ops/test_categorify.py: 65 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 96 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

tests/unit/workflow/test_workflow.py: 44 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(
tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(
tests/unit/test_tools.py: 12 warnings

tests/unit/ops/test_ops.py: 68 warnings

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(
tests/unit/test_triton_inference.py: 10 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/inference/triton/data_conversions.py:125: FutureWarning: The series.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.

offsets = pd.Series([0]).append(col.map(len).cumsum()).values
tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(
tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(
tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(
tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_fill.py:72: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_join.py: 96 warnings

/var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg/partd/pandas.py:113: DeprecationWarning: The Index._get_attributes_dict method is deprecated, and will be removed in a future version

header = (type(ind), ind._get_attributes_dict(), values.dtype, cat)
tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/tests/unit/ops/test_ops.py:134: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df[col].iloc[idx] = None
tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(
tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(
tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing
examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

merlin/systems/dag/init.py                                     3      0      0      0   100%

merlin/systems/dag/ensemble.py                                    64      2     30      2    96%   79, 81

merlin/systems/dag/node.py                                        30     10     12      1    69%   31-32, 35-36, 50, 57-62

merlin/systems/dag/op_runner.py                                   21      0      8      0   100%

merlin/systems/dag/ops/init.py                                 0      0      0      0   100%

merlin/systems/dag/ops/operator.py                                62      5     18      1    90%   24-25, 30, 50, 59

merlin/systems/dag/ops/session_filter.py                          43     11      8      3    73%   35-38, 56, 61, 68, 84-88

merlin/systems/dag/ops/softmax_sampling.py                        50     17      4      1    67%   23-29, 60, 78-108

merlin/systems/dag/ops/tensorflow.py                              73     10     24      4    84%   39-40, 48->57, 86, 130-139

merlin/systems/dag/ops/workflow.py                                26      0      2      0   100%

merlin/systems/triton/init.py                                 36     12     14      1    58%   42-49, 68, 72, 76-82

merlin/systems/triton/conversions.py                              87     73     58      0    10%   32-33, 52-84, 88-94, 98-105, 109-115, 119-136, 140-150

merlin/systems/triton/export.py                                  266    154     82      9    42%   89-93, 156-195, 239-287, 380-388, 414, 417-433, 437-441, 454-455, 485-495, 547-587, 593-609, 613-680, 686-693, 732, 738-757, 763-787, 794

merlin/systems/triton/model_config_pb2.py                        299      0      2      0   100%

merlin/systems/workflow/init.py                               22      5     12      3    65%   43, 51, 63-65

nvtabular/init.py                                             19      2      0      0    89%   40-41

nvtabular/dispatch.py                                              1      0      0      0   100%

nvtabular/framework_utils/init.py                              0      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     12     89      6    91%   60, 68->49, 122, 179, 231-239, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 1      0      0      0   100%

nvtabular/inference/init.py                                    0      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      52     52     22      0     0%   27-124

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    1      0      0      0   100%

nvtabular/loader/init.py                                       0      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   32->35, 35->37, 42->44, 46, 47->68, 53-54, 62-64, 70-74, 89-94, 104-117

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      657     71    350     50    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 389->393, 397-404, 450, 458, 481-482, 559-564, 635, 731, 748, 793, 871-872, 887-891, 892->856, 910, 918, 925->exit, 949, 952->955, 1004->1002, 1066-1068, 1071, 1073->1075, 1092->1096, 1098->1051, 1104-1107, 1119, 1123, 1127, 1134, 1139-1142, 1220, 1222, 1292->1315, 1298->1315, 1316-1321, 1366, 1379->1382, 1386->1391, 1390, 1396, 1399, 1407-1417

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   91->93, 95, 97->87

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             21      0      8      1    97%   41->46

nvtabular/ops/dropna.py                                            8      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         130      4     84      5    96%   72, 83, 93->95, 131, 221

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   73, 112->118

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   59, 63, 81, 93-108

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                44      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   45, 70-72

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 130-133, 143-147, 149, 171-172, 323, 333, 359, 362-371

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 1      0      0      0   100%

nvtabular/worker.py                                                1      1      0      0     0%   20

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   218     17     94     12    91%   28-29, 51, 84, 167, 173->187, 200-202, 326, 341-342, 378, 455, 485, 493-495, 508
TOTAL                                                           6387   1513   2371    239    74%

Coverage XML written to file coverage.xml
Required test coverage of 70% reached. Total coverage: 73.77%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========= 1421 passed, 2 skipped, 1612 warnings in 1461.39s (0:24:21) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins8313928970765867780.sh

benfred and others added 30 commits December 14, 2020 09:15

API Overhaul

d708172

First draft of the API overhauls changes. Adds most core functionality, including defining workflow graphs with a ColumnGroup class, the workflow and dataset changes , most operators converted to use the new api, etc.

remove debug print statement

abc0cb4

Fix test_io unittest

b69ad67

Also partially fix some tests inside test_workflow

Merge remote-tracking branch 'origin/main' into new_api

2fd3db9

Handle multi-column joint/combo categorify

2e918cd

Update JoinGroupby

23a4eb9

Fix differencelag

711efd8

add dependencies method (NVIDIA-Merlin#498)

fd3f35a

Convert TargetEncoding op

9b78a46

Merge branch 'new_api' of github.com:NVIDIA/NVTabular into new_api

82f1c17

Update nvtabular/workflow.py

5c28e85

Co-authored-by: Richard (Rick) Zamora <rzamora217@gmail.com>

Update nvtabular/workflow.py

6a84e7d

Co-authored-by: Richard (Rick) Zamora <rzamora217@gmail.com>

Remove workflow code from dataloaders

b21ffd8

We should be doing online transforms like ```KerasSequenceLoader(workflow.transform(dataset), ...``` instead of ```KerasSequenceLoader(dataset, workflows=[workflow], ...``` now

Merge branch 'new_api' of github.com:NVIDIA/NVTabular into new_api

5467d86

Unittest ops + bugfix in Bucketize (NVIDIA-Merlin#496)

f216edf

* test_minmix * updates test * unittest ops

First draft get_embedding_sizes support

b44dfa6

Re-add get_embedding_sizes . Note that this changes how we support multi-hot columns here (sizes are returned same as single hot, and we don't use this method to distinguish between multi and singlehot columns)

isort

27b4e33

Remove groupbystatistics

a95a4d9

implement serialization of statistics

0e55c2a

add save_stats/load_stats/clear_stats methods to the workflow, with each statoperator getting called as appropiate

Fix TF dataloader unittests

ee9367c

test_torch_dataloader fixes

7bf624f

doc strings

4c99186

resovled

aee86e2

Merge remote-tracking branch 'upstream/main'

ef52f5a

Merge remote-tracking branch 'upstream/main'

a2cdcc6

merge

1783f8c

t checkout master

3ebf331

git merge upstream/main Merge branch 'main' of https://github.com/NVIDIA/NVTabular

Merge branch 'main' of https://github.com/NVIDIA/NVTabular

a3c9695

Merge branch 'main' of https://github.com/NVIDIA-Merlin/NVTabular

4d53a25

Merge branch 'main' of https://github.com/NVIDIA/NVTabular

5f8c040

bschifferer added 2 commits March 17, 2022 16:35

Merge branch 'main' of https://github.com/NVIDIA/NVTabular

55ee5c8

fix tagas

92c7485

bschifferer requested a review from benfred March 17, 2022 16:39

karlhigley added the bug Something isn't working label Mar 17, 2022

karlhigley added this to the Merlin 22.04 milestone Mar 17, 2022

karlhigley self-assigned this Mar 17, 2022

karlhigley approved these changes Mar 17, 2022

View reviewed changes

Merge branch 'main' into fix_tagas

81aebae

karlhigley assigned bschifferer and unassigned karlhigley Mar 17, 2022

Merge branch 'main' into fix_tagas

f223251

karlhigley merged commit 154ed10 into NVIDIA-Merlin:main Mar 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrapper for TagAs did not work #1462

Wrapper for TagAs did not work #1462

bschifferer commented Mar 17, 2022

github-actions bot commented Mar 17, 2022

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

Wrapper for TagAs did not work #1462

Wrapper for TagAs did not work #1462

Conversation

bschifferer commented Mar 17, 2022

github-actions bot commented Mar 17, 2022

Documentation preview

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 ----------- Name Stmts Miss Branch BrPart Cover Missing

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 ----------- Name Stmts Miss Branch BrPart Cover Missing

nvidia-merlin-bot commented Mar 17, 2022

---------- coverage: platform linux, python 3.8.10-final-0 ----------- Name Stmts Miss Branch BrPart Cover Missing

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing