Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 5.0 to master #2849

Merged
merged 24 commits into from
Jul 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[flake8]
max-line-length = 88
exclude =
*.egg-info,
*.pyc,
.git,
.tox,
.venv*,
build,
docs/*,
dist,
docker,
venv*,
.venv*,
whitelist.py,
tasks.py
ignore =
F405
W503
E203
E126
75 changes: 62 additions & 13 deletions .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ on:
schedule:
- cron: '0 1 * * *' # nightly build

concurrency:
group: ${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read # to fetch code (actions/checkout)

Expand Down Expand Up @@ -48,7 +52,7 @@ jobs:

run-tests:
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 60
strategy:
max-parallel: 15
fail-fast: false
Expand All @@ -68,32 +72,77 @@ jobs:
- name: run tests
run: |
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
tox -e ${{matrix.test-type}}-${{matrix.connection-type}}
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install hiredis
fi
invoke devenv
sleep 5 # time to settle
invoke ${{matrix.test-type}}-tests

- uses: actions/upload-artifact@v2
if: success() || failure()
with:
name: pytest-results-${{matrix.test-type}}
name: pytest-results-${{matrix.test-type}}-${{matrix.connection-type}}-${{matrix.python-version}}
path: '${{matrix.test-type}}*results.xml'

- name: Upload codecov coverage
uses: codecov/codecov-action@v3
if: ${{matrix.python-version == '3.11'}}
with:
fail_ci_if_error: false
# - name: View Test Results
# uses: dorny/test-reporter@v1
# if: success() || failure()
# with:
# name: Test Results ${{matrix.python-version}} ${{matrix.test-type}}-${{matrix.connection-type}}
# path: '${{matrix.test-type}}*results.xml'
# reporter: java-junit
# list-suites: failed
# list-tests: failed
# max-annotations: 10

- name: View Test Results
uses: dorny/test-reporter@v1
if: success() || failure()
continue-on-error: true
with:
name: Test Results ${{matrix.python-version}} ${{matrix.test-type}}-${{matrix.connection-type}}
path: '*.xml'
reporter: java-junit
list-suites: all
list-tests: all
max-annotations: 10
fail-on-error: 'false'

resp3_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.7', '3.11']
test-type: ['standalone', 'cluster']
connection-type: ['hiredis', 'plain']
protocol: ['3']
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
name: RESP3 [${{ matrix.python-version }} ${{matrix.test-type}}-${{matrix.connection-type}}]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: run tests
run: |
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install hiredis
fi
invoke devenv
sleep 5 # time to settle
invoke ${{matrix.test-type}}-tests
invoke ${{matrix.test-type}}-tests --uvloop

build_and_test_package:
name: Validate building and installing the package
runs-on: ubuntu-latest
needs: [run-tests]
strategy:
fail-fast: false
matrix:
extension: ['tar.gz', 'whl']
steps:
Expand Down
5 changes: 5 additions & 0 deletions .isort.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[settings]
profile=black
multi_line_output=3
src_paths = ["redis", "tests"]
skip_glob=benchmarks/*
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,9 @@ Here's how to get started with your code contribution:
a. python -m venv .venv
b. source .venv/bin/activate
c. pip install -r dev_requirements.txt
c. pip install -r requirements.txt

4. If you need a development environment, run `invoke devenv`
4. If you need a development environment, run `invoke devenv`. Note: this relies on docker-compose to build environments, and assumes that you have a version supporting [docker profiles](https://docs.docker.com/compose/profiles/).
5. While developing, make sure the tests pass by running `invoke tests`
6. If you like the change and think the project could use it, send a
pull request
Expand All @@ -59,7 +60,6 @@ can execute docker and its various commands.
- Three sentinel Redis nodes
- A redis cluster
- An stunnel docker, fronting the master Redis node
- A Redis node, running unstable - the latest redis

The replica node, is a replica of the master node, using the
[leader-follower replication](https://redis.io/topics/replication)
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/socket_read_size.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from base import Benchmark

from redis.connection import HiredisParser, PythonParser
from redis.connection import PythonParser, _HiredisParser


class SocketReadBenchmark(Benchmark):

ARGUMENTS = (
{"name": "parser", "values": [PythonParser, HiredisParser]},
{"name": "parser", "values": [PythonParser, _HiredisParser]},
{
"name": "value_size",
"values": [10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000],
Expand Down
5 changes: 2 additions & 3 deletions dev_requirements.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
click==8.0.4
black==22.3.0
flake8==5.0.4
flake8-isort==6.0.0
flynt~=0.69.0
isort==5.10.1
mock==4.0.3
packaging>=20.4
pytest==7.2.0
pytest-timeout==2.0.1
pytest-timeout==2.1.0
pytest-asyncio>=0.20.2
tox==3.27.1
tox-docker==3.1.0
invoke==1.7.3
pytest-cov>=4.0.0
vulture>=2.3.0
Expand Down
109 changes: 109 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---

version: "3.8"

services:

redis:
image: redis/redis-stack-server:edge
container_name: redis-standalone
ports:
- 6379:6379
environment:
- "REDIS_ARGS=--enable-debug-command yes --enable-module-command yes"
profiles:
- standalone
- sentinel
- replica
- all

replica:
image: redis/redis-stack-server:edge
container_name: redis-replica
depends_on:
- redis
environment:
- "REDIS_ARGS=--replicaof redis 6379"
ports:
- 6380:6379
profiles:
- replica
- all

cluster:
container_name: redis-cluster
build:
context: .
dockerfile: dockers/Dockerfile.cluster
ports:
- 16379:16379
- 16380:16380
- 16381:16381
- 16382:16382
- 16383:16383
- 16384:16384
volumes:
- "./dockers/cluster.redis.conf:/redis.conf:ro"
profiles:
- cluster
- all

stunnel:
image: redisfab/stunnel:latest
depends_on:
- redis
ports:
- 6666:6666
profiles:
- all
- standalone
- ssl
volumes:
- "./dockers/stunnel/conf:/etc/stunnel/conf.d:ro"
- "./dockers/stunnel/keys:/etc/stunnel/keys:ro"

sentinel:
image: redis/redis-stack-server:edge
container_name: redis-sentinel
depends_on:
- redis
environment:
- "REDIS_ARGS=--port 26379"
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26379"
ports:
- 26379:26379
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all

sentinel2:
image: redis/redis-stack-server:edge
container_name: redis-sentinel2
depends_on:
- redis
environment:
- "REDIS_ARGS=--port 26380"
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26380"
ports:
- 26380:26380
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all

sentinel3:
image: redis/redis-stack-server:edge
container_name: redis-sentinel3
depends_on:
- redis
entrypoint: "/opt/redis-stack/bin/redis-sentinel /redis.conf --port 26381"
ports:
- 26381:26381
volumes:
- "./dockers/sentinel.conf:/redis.conf"
profiles:
- sentinel
- all
4 changes: 0 additions & 4 deletions docker/base/Dockerfile

This file was deleted.

11 changes: 0 additions & 11 deletions docker/base/Dockerfile.cluster

This file was deleted.

9 changes: 0 additions & 9 deletions docker/base/Dockerfile.cluster4

This file was deleted.

9 changes: 0 additions & 9 deletions docker/base/Dockerfile.cluster5

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.redis4

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.redis5

This file was deleted.

12 changes: 0 additions & 12 deletions docker/base/Dockerfile.redismod_cluster

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel4

This file was deleted.

4 changes: 0 additions & 4 deletions docker/base/Dockerfile.sentinel5

This file was deleted.

11 changes: 0 additions & 11 deletions docker/base/Dockerfile.stunnel

This file was deleted.

Loading
Loading