Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Move pympler back into the all extras #12652

Merged
merged 2 commits into from
May 7, 2022
Merged

Move pympler back into the all extras #12652

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
108ee83
Move `pympler` back into the `all` extras
May 6, 2022
bfeb294
Debian changelog entry to keep the linter happy
May 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/12652.misc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Move `pympler` back in to the `all` extras.
1 change: 0 additions & 1 deletion debian/build_virtualenv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ poetry export \
--extras all \
--extras test \
--extras systemd \
--extras cache_memory \
-o exported_requirements.txt
deactivate
rm -rf "$TEMP_VENV"
Expand Down
8 changes: 8 additions & 0 deletions debian/changelog
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
matrix-synapse-py3 (1.58.2) UNRELEASED; urgency=medium

* Adjust how the `exported-requirements.txt` file is generated as part of
the process of building these packages. This affects the package
maintainers only; end-users are unaffected.

-- Synapse Packaging team <packages@matrix.org> Fri, 06 May 2022 13:49:29 +0100

matrix-synapse-py3 (1.58.1) stable; urgency=medium

* Include python dependencies from the `systemd` and `cache_memory` extras package groups, which
Expand Down
4 changes: 2 additions & 2 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 4 additions & 3 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,10 +231,11 @@ all = [
"jaeger-client", "opentracing",
# jwt
"pyjwt",
#redis
"txredisapi", "hiredis"
# redis
"txredisapi", "hiredis",
# cache_memory
"pympler",
Copy link
Contributor

@MadLittleMods MadLittleMods May 10, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to break the Docker builds in complement.sh in some way I don't understand.

Here is a git bisect of sorts:

❌ d80a7ab151afd6919b4e14a258105ab59146d528

❌ 0ce2201932488e98cab8a7d81788e5bcf8f8dd5e

✅ 051a1c3f220938a0ea1a5b328c268bdb3d1ad592

✅ 2607b3e1816341b3b8534077bd5d3a4daf3a3d15

✅ c2d50e9f6c5f7b01cbd8bf1dca36cb8c0e7b007f

✅ 07fa53ec40106b97ba2c12a2dcc5446325e5fb61

And how it breaks:

Relevant part of the full-run:

#18 2.864 ERROR: Could not find a version that satisfies the requirement canonicaljson==1.6.0 (from versions: 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.2.0, 1.3.0, 1.4.0, 1.5.0)
#18 2.864 ERROR: No matching distribution found for canonicaljson==1.6.0

Full-run:

$ COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1 COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestSendAndFetchMessage
[+] Building 4.1s (17/24)
 => [internal] load build definition from Dockerfile                                                                                                             0.0s
 => => transferring dockerfile: 37B                                                                                                                              0.0s
 => [internal] load .dockerignore                                                                                                                                0.0s
 => => transferring context: 35B                                                                                                                                 0.0s
 => resolve image config for docker.io/docker/dockerfile:1                                                                                                       0.3s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:443aab4ca21183e069e7d8b2dc68006594f40bddf1b15bbd83f5137bd93e80e2                                  0.0s
 => [internal] load build definition from Dockerfile                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.9-slim                                                                                               0.3s
 => [internal] load build context                                                                                                                                0.1s
 => => transferring context: 259.54kB                                                                                                                            0.1s
 => [requirements 1/6] FROM docker.io/library/python:3.9-slim@sha256:ba3b77ddbc953cdb8d998b2052088d4af4b8805805e5b01975a05af4e19855ea                            0.0s
 => CACHED [requirements 2/6] RUN    --mount=type=cache,target=/var/cache/apt,sharing=locked    --mount=type=cache,target=/var/lib/apt,sharing=locked  apt-get   0.0s
 => CACHED [requirements 3/6] RUN --mount=type=cache,target=/root/.cache/pip   pip install --user git+https://github.com/python-poetry/poetry.git@fb13b3a676f47  0.0s
 => CACHED [requirements 4/6] WORKDIR /synapse                                                                                                                   0.0s
 => CACHED [requirements 5/6] COPY pyproject.toml poetry.lock /synapse/                                                                                          0.0s
 => CACHED [requirements 6/6] RUN /root/.local/bin/poetry export --extras all -o /synapse/requirements.txt                                                       0.0s
 => CACHED [builder 2/7] RUN    --mount=type=cache,target=/var/cache/apt,sharing=locked    --mount=type=cache,target=/var/lib/apt,sharing=locked  apt-get updat  0.0s
 => CACHED [builder 3/7] COPY --from=requirements /synapse/requirements.txt /synapse/                                                                            0.0s
 => ERROR [builder 4/7] RUN --mount=type=cache,target=/root/.cache/pip   pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requir  3.0s
------
 > [builder 4/7] RUN --mount=type=cache,target=/root/.cache/pip   pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requirements.txt:
#18 2.309 Ignoring importlib-metadata: markers 'python_full_version >= "3.7.1" and python_version < "3.8"' don't match your environment
#18 2.310 Ignoring importlib-resources: markers 'python_full_version >= "3.7.1" and python_version < "3.9"' don't match your environment
#18 2.315 Ignoring psycopg2cffi-compat: markers 'python_full_version >= "3.7.1" and python_full_version < "4.0.0" and platform_python_implementation == "PyPy"' don't match your environment
#18 2.316 Ignoring psycopg2cffi: markers 'python_full_version >= "3.7.1" and python_full_version < "4.0.0" and platform_python_implementation == "PyPy"' don't match your environment
#18 2.320 Ignoring twisted-iocpsupport: markers 'python_full_version >= "3.7.1" and python_full_version < "4.0.0" and platform_system == "Windows"' don't match your environment
#18 2.321 Ignoring zipp: markers 'python_full_version >= "3.7.1" and python_version < "3.8" or python_full_version >= "3.7.1" and python_version < "3.9"' don't match your environment
#18 2.444 Collecting attrs==21.4.0
#18 2.446   Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
#18 2.495 Collecting authlib==0.15.5
#18 2.498   Using cached Authlib-0.15.5-py2.py3-none-any.whl (203 kB)
#18 2.542 Collecting automat==20.2.0
#18 2.544   Using cached Automat-20.2.0-py2.py3-none-any.whl (31 kB)
#18 2.650 Collecting bcrypt==3.2.0
#18 2.652   Using cached bcrypt-3.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (61 kB)
#18 2.712 Collecting bleach==4.1.0
#18 2.715   Using cached bleach-4.1.0-py2.py3-none-any.whl (157 kB)
#18 2.864 ERROR: Could not find a version that satisfies the requirement canonicaljson==1.6.0 (from versions: 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.2.0, 1.3.0, 1.4.0, 1.5.0)
#18 2.864 ERROR: No matching distribution found for canonicaljson==1.6.0
------
executor failed running [/bin/sh -c pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requirements.txt]: exit code: 1

Copy link
Contributor

@MadLittleMods MadLittleMods May 10, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right when I have gathered all of the info and the results were consistent for all my testing, it started working (even on the latest develop). I'm assuming this is some Docker build caching layer problem?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caching would be my guess, or maybe a temporary network failure that means the build process can't contact PyPI. I'm happy to investigate if you see it again though!

# omitted:
# - cache_memory: this is an experimental option
# - test: it's useful to have this separate from dev deps in the olddeps job
# - systemd: this is a system-based requirement
]
Expand Down