Refactor, add tests and support for pixel data interface v2

.coveragerc

@@ -1,8 +1,4 @@
 [run]
 omit =
     pylibjpeg/tests/*
-    pylibjpeg/scripts/*
     pylibjpeg/tools/*
-    pylibjpeg-data/*
-    pylibjpeg-libjpeg/*
-    pydicom/*

.github/workflows/release-deploy.yml

@@ -35,7 +35,18 @@ jobs:
         path: ./dist
 
     - name: Publish package to PyPi
-      uses: pypa/gh-action-pypi-publish@master
-      with:
-        user: __token__
-        password: ${{ secrets.PYPI_PASSWORD }}
+      environment:
+        name: pypi
+        url: https://pypi.org/project/pylibjpeg/
+      permissions:
+        id-token: write
+
+      steps:
+      - name: Download the wheels
+        uses: actions/download-artifact@v4
+        with:
+          path: dist/
+          merge-multiple: true
+
+      - name: Publish package to PyPi
+        uses: pypa/gh-action-pypi-publish@release/v1

.gitignore

@@ -31,6 +31,7 @@ __pycache__
 .pytest_cache
 *.egg-info
 build
+env*/
 
 # Docs build
 docs/_build/*
@@ -45,15 +46,14 @@ doc/reference/generated/*
 # PyCharm IDE files
 *.idea*
 
-
-
 # jupyter notebooks
 *.ipynb
 .ipynb_checkpoints/*
 tests/test_pixel.py
 
 # mypy
-pydicom/.mypy_cache/*
+.mypy_cache/
+.ruff_cache/
 
 # vscode
 .vscode/*

README.md

@@ -1,11 +1,14 @@
-[![codecov](https://codecov.io/gh/pydicom/pylibjpeg/branch/master/graph/badge.svg)](https://codecov.io/gh/pydicom/pylibjpeg)
-[![Build Status](https://github.com/pydicom/pylibjpeg/workflows/build/badge.svg)](https://github.com/pydicom/pylibjpeg/actions?query=workflow%3Abuild)
-[![PyPI version](https://badge.fury.io/py/pylibjpeg.svg)](https://badge.fury.io/py/pylibjpeg)
-[![Python versions](https://img.shields.io/pypi/pyversions/pylibjpeg.svg)](https://img.shields.io/pypi/pyversions/pylibjpeg.svg)
+<p align="center">
+<a href="https://github.com/pydicom/pylibjpeg/actions?query=workflow%3Aunit-tests"><img alt="Build status" src="https://github.com/pydicom/pylibjpeg/workflows/unit-tests/badge.svg"></a>
+<a href="https://codecov.io/gh/pydicom/pylibjpeg"><img alt="Test coverage" src="https://codecov.io/gh/pydicom/pylibjpeg/branch/main/graph/badge.svg"></a>
+<a href="https://pypi.org/project/pylibjpeg/"><img alt="PyPI versions" src="https://badge.fury.io/py/pylibjpeg.svg"></a>
+<a href="https://www.python.org/"><img alt="Python versions" src="https://img.shields.io/pypi/pyversions/pylibjpeg.svg"></a>
+<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
+</p>
 
 ## pylibjpeg
 
-A Python 3.10+ framework for decoding JPEG images and decoding/encoding RLE datasets, with a focus on providing support for [pydicom](https://github.com/pydicom/pydicom).
+A Python 3.8+ framework for decoding JPEG images and decoding/encoding RLE datasets, with a focus on providing support for [pydicom](https://github.com/pydicom/pydicom).
 
 
 ### Installation
@@ -42,26 +45,29 @@ python -m pip install pylibjpeg
 One or more plugins are required before *pylibjpeg* is able to handle JPEG images or RLE datasets. To handle a given format or DICOM Transfer Syntax
 you first have to install the corresponding package:
 
-#### Supported Formats
-|Format                   |Decode?|Encode?|Plugin                 |Based on     |
-|---                      |------ |---    |---                    |---          |
-|JPEG, JPEG-LS and JPEG XT|Yes    |No     |[pylibjpeg-libjpeg][1] |[libjpeg][2] |
-|JPEG 2000                |Yes    |No     |[pylibjpeg-openjpeg][3]|[openjpeg][4]|
-|RLE Lossless (PackBits)  |Yes    |Yes    |[pylibjpeg-rle][5]     |-            |
-
-#### DICOM Transfer Syntax
-
-|UID                   | Description                                    | Plugin                |
-|---                   |---                                             |----                   |
-|1.2.840.10008.1.2.4.50|JPEG Baseline (Process 1)                       |[pylibjpeg-libjpeg][1] |
-|1.2.840.10008.1.2.4.51|JPEG Extended (Process 2 and 4)                 |[pylibjpeg-libjpeg][1] |
-|1.2.840.10008.1.2.4.57|JPEG Lossless, Non-Hierarchical (Process 14)    |[pylibjpeg-libjpeg][1] |
-|1.2.840.10008.1.2.4.70|JPEG Lossless, Non-Hierarchical, First-Order Prediction</br>(Process 14, Selection Value 1) | [pylibjpeg-libjpeg][1]|
-|1.2.840.10008.1.2.4.80|JPEG-LS Lossless                                |[pylibjpeg-libjpeg][1] |
-|1.2.840.10008.1.2.4.81|JPEG-LS Lossy (Near-Lossless) Image Compression |[pylibjpeg-libjpeg][1] |
-|1.2.840.10008.1.2.4.90|JPEG 2000 Image Compression (Lossless Only)     |[pylibjpeg-openjpeg][4]|
-|1.2.840.10008.1.2.4.91|JPEG 2000 Image Compression                     |[pylibjpeg-openjpeg][4]|
-|1.2.840.10008.1.2.5   |RLE Lossless                                    |[pylibjpeg-rle][5]     |
+#### Supported Image Formats
+|Format                   |Decode?|Encode?|Plugin                 | License |Based on     |
+|---                      |------ |---    |---                    |---      |---          |
+|JPEG, JPEG-LS and JPEG XT|Yes    |No     |[pylibjpeg-libjpeg][1] | GPLv3   |[libjpeg][2] |
+|JPEG 2000                |Yes    |No     |[pylibjpeg-openjpeg][3]| MIT     |[openjpeg][4]|
+|RLE Lossless (PackBits)  |Yes    |Yes    |[pylibjpeg-rle][5]     | MIT     |-            |
+
+#### Supported DICOM Transfer Syntaxes
+
+|UID                    | Description                                    | Plugin                |
+|---                    |---                                             |----                   |
+|1.2.840.10008.1.2.4.50 |JPEG Baseline (Process 1)                       |[pylibjpeg-libjpeg][1] |
+|1.2.840.10008.1.2.4.51 |JPEG Extended (Process 2 and 4)                 |[pylibjpeg-libjpeg][1] |
+|1.2.840.10008.1.2.4.57 |JPEG Lossless, Non-Hierarchical (Process 14)    |[pylibjpeg-libjpeg][1] |
+|1.2.840.10008.1.2.4.70 |JPEG Lossless, Non-Hierarchical, First-Order Prediction</br>(Process 14, Selection Value 1) | [pylibjpeg-libjpeg][1]|
+|1.2.840.10008.1.2.4.80 |JPEG-LS Lossless                                |[pylibjpeg-libjpeg][1] |
+|1.2.840.10008.1.2.4.81 |JPEG-LS Lossy (Near-Lossless) Image Compression |[pylibjpeg-libjpeg][1] |
+|1.2.840.10008.1.2.4.90 |JPEG 2000 Image Compression (Lossless Only)     |[pylibjpeg-openjpeg][3]|
+|1.2.840.10008.1.2.4.91 |JPEG 2000 Image Compression                     |[pylibjpeg-openjpeg][3]|
+|1.2.840.10008.1.2.4.201|High-Throughput JPEG 2000 Image Compression (Lossless Only) |[pylibjpeg-openjpeg][3]|
+|1.2.840.10008.1.2.4.202|High-Throughput JPEG 2000 with RPCL Options Image Compression (Lossless Only) |[pylibjpeg-openjpeg][3]|
+|1.2.840.10008.1.2.4.203|High-Throughput JPEG 2000 Image Compression |[pylibjpeg-openjpeg][3]|
+|1.2.840.10008.1.2.5    |RLE Lossless                                    |[pylibjpeg-rle][5]     |
 
 If you're not sure what the dataset's *Transfer Syntax UID* is, it can be
 determined with:
@@ -103,19 +109,6 @@ ds.decompress("pylibjpeg")
 rle_arr = ds.pixel_array
 ```
 
-For datasets with multiple frames you can reduce your memory usage by
-processing each frame separately using the ``generate_frames()`` generator
-function:
-```python
-from pydicom import dcmread
-from pydicom.data import get_testdata_file
-from pydicom.pixel_data_handlers.pylibjpeg_handler import generate_frames
-
-ds = dcmread(get_testdata_file('color3d_jpeg_baseline.dcm'))
-frames = generate_frames(ds)
-arr = next(frames)
-```
-
 ##### Standalone JPEG decoding
 You can also just use *pylibjpeg* to decode JPEG images to a [numpy ndarray](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.html), provided you have a suitable plugin installed:
 ```python

codecov.yml

@@ -13,8 +13,4 @@ coverage:
 
 ignore:
   - "pylibjpeg/tests"
-  - "pylibjpeg/scripts"
   - "pylibjpeg/tools"
-  - "pylibjpeg-libjpeg"
-  - "pylibjpeg-data"
-  - "pydicom"

docs/plugins.md

@@ -58,21 +58,34 @@ the requirements of the transfer syntax:
 
 ```python
 def my_pixel_data_decoder(
-    src: bytes, ds: Optional[pydicom.dataset.Dataset] = None, **kwargs: Any
-) -> numpy.ndarray:
+    src: bytes,
+    ds: pydicom.dataset.Dataset | None = None,
+    version: int = 1,
+    **kwargs: Any,
+) -> numpy.ndarray | bytearray:
     """Return the encoded *src* as an unshaped numpy ndarray of uint8.
 
-    .. versionchanged:: 1.3
+    .. versionchanged: 1.3
 
         Added requirement to return little-endian ordered data by default.
 
+    .. versionchanged: 2.0
+
+        Added `version` keyword argument and support for returning :class:`bytearray`
+
     Parameters
     ----------
     src : bytes
         A single frame of the encoded *Pixel Data*.
     ds : pydicom.dataset.Dataset, optional
         A dataset containing the group ``0x0028`` elements corresponding to
         the *Pixel Data*. If not used then *kwargs* must be supplied.
+    version : int, optional
+
+      * If ``1`` (default) then either supplying either `ds` or `kwargs` is
+        required and the return type is a :class:`~numpy.ndarray`
+      * If ``2`` then `ds` will be ignored, `kwargs` is required and the return
+        type is :class:`bytearray`
     kwargs : Dict[str, Any]
         A dict containing relevant image pixel module elements:
 
@@ -94,8 +107,10 @@ def my_pixel_data_decoder(
 
     Returns
     -------
-    numpy.ndarray
-        A 1-dimensional ndarray of 'uint8' containing the little-endian ordered decoded pixel data.
+    numpy.ndarray | bytearray
+        Either a 1-dimensional ndarray of 'uint8' or a bytearray containing the
+        little-endian ordered decoded pixel data, depending on the value of
+        `version`.
     """
     # Decoding happens here
 ```
@@ -206,7 +221,7 @@ The pixel data encoding function will be passed two required parameters:
 The function should return the encoded pixel data as `bytes`.
 
 ```python
-def my_pixel_data_encoder(src: bytes, **kwargs) -> bytes:
+def my_pixel_data_encoder(src: bytes, **kwargs: Any) -> bytes:
     """Return `src` as encoded bytes.
 
     Parameters

docs/release_notes/v2.0.0.rst

@@ -7,3 +7,5 @@
 * Switched to a ``pyproject.toml`` based project
 * Removed ``pydicom`` module
 * Supported Python versions are 3.8, 3.9, 3.10, 3.11 and 3.12
+* Added type hints
+* Add support for version 2 of the pixel data interface

pylibjpeg/tests/__init__.py

@@ -1,14 +1,14 @@
 import logging
 import sys
 
-_logger = logging.getLogger(__name__)
+LOGGER = logging.getLogger(__name__)
 
 try:
     import ljdata as _data
 
     globals()["data"] = _data
     # Add to cache - needed for pytest
     sys.modules["pylibjpeg.data"] = _data
-    _logger.debug("pylibjpeg-data module loaded")
+    LOGGER.debug("pylibjpeg-data module loaded")
 except ImportError:
     pass

pylibjpeg/tests/test_decode.py

@@ -1,14 +1,15 @@
 """Tests for standalone decoding."""
 
 from io import BytesIO
+import logging
 import os
 from pathlib import Path
 
 import pytest
 
 from pylibjpeg import decode
 from pylibjpeg.data import JPEG_DIRECTORY
-from pylibjpeg.utils import get_decoders
+from pylibjpeg.utils import get_decoders, get_pixel_data_decoders
 
 
 HAS_DECODERS = bool(get_decoders())
@@ -25,22 +26,22 @@ def test_decode_str(self):
         """Test passing a str to decode."""
         fpath = os.path.join(JPEG_DIRECTORY, "10918", "p1", "A1.JPG")
         assert isinstance(fpath, str)
-        with pytest.raises(RuntimeError, match=r"No decoders are available"):
+        with pytest.raises(RuntimeError, match=r"No JPEG decoders are available"):
             decode(fpath)
 
     def test_decode_pathlike(self):
         """Test passing a pathlike to decode."""
         fpath = os.path.join(JPEG_DIRECTORY, "10918", "p1", "A1.JPG")
         p = Path(fpath)
         assert isinstance(p, os.PathLike)
-        with pytest.raises(RuntimeError, match=r"No decoders are available"):
+        with pytest.raises(RuntimeError, match=r"No JPEG decoders are available"):
             decode(p)
 
     def test_decode_filelike(self):
         """Test passing a filelike to decode."""
         fpath = os.path.join(JPEG_DIRECTORY, "10918", "p1", "A1.JPG")
         with open(fpath, "rb") as f:
-            msg = r"No decoders are available"
+            msg = r"No JPEG decoders are available"
             with pytest.raises(RuntimeError, match=msg):
                 decode(f)
 
@@ -51,13 +52,51 @@ def test_decode_bytes(self):
             data = f.read()
 
         assert isinstance(data, bytes)
-        msg = r"No decoders are available"
+        msg = r"No JPEG decoders are available"
         with pytest.raises(RuntimeError, match=msg):
             decode(data)
 
     def test_unknown_decoder_type(self):
         """Test unknown decoder type."""
-        assert not get_decoders(decoder_type="TEST")
+        msg = "No matching plugin entry point for 'foo'"
+        with pytest.raises(KeyError, match=msg):
+            get_decoders(decoder_type="foo")
+
+    def test_get_decoders(self, caplog):
+        """Tests for get_decoders()"""
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            get_decoders()
+            assert (
+                "No plugins found for entry point 'pylibjpeg.jpeg_decoders'"
+            ) in caplog.text
+
+        caplog.clear()
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            assert get_decoders("JPEG-LS") == {}
+            assert (
+                "No plugins found for entry point 'pylibjpeg.jpeg_ls_decoders'"
+            ) in caplog.text
+
+    def test_get_decoders_raises(self):
+        """Test exception raised if invalid decoder type."""
+        msg = "No matching plugin entry point for 'JPEG XX'"
+        with pytest.raises(KeyError, match=msg):
+            get_decoders("JPEG XX")
+
+    def test_get_pixel_data_decoders(self, caplog):
+        """Tests for get_pixel_data_decoders()"""
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            get_pixel_data_decoders()
+            assert (
+                "No plugins found for entry point 'pylibjpeg.pixel_data_decoders'"
+            ) in caplog.text
+
+        caplog.clear()
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            get_pixel_data_decoders(version=2)
+            assert (
+                "No plugins found for entry point 'pylibjpeg.pixel_data_decoders'"
+            ) in caplog.text
 
 
 @pytest.mark.skipif(not RUN_JPEG, reason="No JPEG decoders available")
@@ -189,12 +228,17 @@ def test_decode_str(self):
         assert isinstance(fpath, str)
         decode(fpath)
 
-    def test_decode_pathlike(self):
+    def test_decode_pathlike(self, caplog):
         """Test passing a pathlike to decode."""
         fpath = os.path.join(self.basedir, "693.j2k")
         p = Path(fpath)
         assert isinstance(p, os.PathLike)
-        decode(p)
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            decode(p)
+            assert (
+                "Found plugin(s) 'openjpeg' for entry point "
+                "'pylibjpeg.jpeg_2000_decoders'"
+            ) in caplog.text
 
     def test_decode_filelike(self):
         """Test passing a filelike to decode."""
@@ -224,9 +268,39 @@ def test_specify_decoder(self):
         fpath = os.path.join(self.basedir, "693.j2k")
         decode(fpath, decoder="openjpeg")
 
-    @pytest.mark.skipif("libjpeg" in get_decoders(), reason="Have libjpeg")
+    @pytest.mark.skipif(RUN_JPEGLS, reason="Have libjpeg")
     def test_specify_unknown_decoder(self):
         """Test specifying an unknown decoder."""
         fpath = os.path.join(self.basedir, "693.j2k")
         with pytest.raises(ValueError, match=r"The 'libjpeg' decoder"):
             decode(fpath, decoder="libjpeg")
+
+    def test_v1_get_pixel_data_decoders(self, caplog):
+        """Test version 1 of get_pixel_data_decoders()"""
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            decoders = get_pixel_data_decoders()
+
+            assert "1.2.840.10008.1.2.4.90" in decoders
+            assert callable(decoders["1.2.840.10008.1.2.4.90"])
+            assert (
+                "Found plugin(s) for entry point 'pylibjpeg.pixel_data_decoders'"
+            ) in caplog.text
+            assert (
+                "Found plugin 'openjpeg' for UID '1.2.840.10008.1.2.4.90'"
+            ) in caplog.text
+
+    def test_v2_get_pixel_data_decoders(self, caplog):
+        """Test version 2 of get_pixel_data_decoders()"""
+        with caplog.at_level(logging.DEBUG, logger="pylibjpeg"):
+            decoders = get_pixel_data_decoders(version=2)
+            assert "1.2.840.10008.1.2.4.90" in decoders
+            assert "openjpeg" in decoders["1.2.840.10008.1.2.4.90"]
+            for plugin in decoders["1.2.840.10008.1.2.4.90"]:
+                assert callable(decoders["1.2.840.10008.1.2.4.90"][plugin])
+                assert (
+                    f"Found plugin '{plugin}' for UID '1.2.840.10008.1.2.4.90'"
+                ) in caplog.text
+
+            assert (
+                "Found plugin(s) for entry point 'pylibjpeg.pixel_data_decoders'"
+            ) in caplog.text