Update the test criteria for test_read_random_region_cpu_memleak (rap…

…idsai#726)

We are seeing intermittent errors in CI/CD

See rapidsai#725 (comment)

>Seeing one test failure in [this CI job]( https://github.com/rapidsai/cucim/actions/runs/8716002781/job/23909243290?pr=725#step:8:495 ):

```python
_ test_read_random_region_cpu_memleak[testimg_tiff_stripe_4096x4096_256_deflate] _
[gw6] linux -- Python 3.9.19 /pyenv/versions/3.9.19/bin/python
python/cucim/tests/performance/clara/test_read_region_memory_usage.py:100: in test_read_random_region_cpu_memleak
    assert mem_usage_history[-1] - mem_usage_history[1] < iteration * 100
E   assert (828035072 - 826982400) < (1000 * 100)
```

I couldn't determine which part of the code is increasing memory usage, but it's clear that there is no memory leak on the C++ side, as memory usage does not continuously increase per iteration.

This patch updates the validation logic to improve reliability:
- Increase the number of iterations by 10 times.
- Ensure that the memory leak per iteration does not exceed 768 bytes.
- Verify whether the increase in memory usage is continuous.

Authors:
  - Gigon Bae (https://github.com/gigony)

Approvers:
  - Gregory Lee (https://github.com/grlee77)
  - https://github.com/jakirkham

URL: rapidsai#726

python/cucim/tests/performance/clara/test_read_region_memory_usage.py

@@ -72,6 +72,7 @@ def test_read_region_cpu_memleak(testimg_tiff_stripe_4096x4096_256):
 
 
 def test_read_random_region_cpu_memleak(testimg_tiff_stripe_4096x4096_256):
+    import gc
     import os
     import random
 
@@ -81,23 +82,39 @@ def test_read_random_region_cpu_memleak(testimg_tiff_stripe_4096x4096_256):
 
     img = open_image_cucim(testimg_tiff_stripe_4096x4096_256)
 
-    iteration = 1000
+    iteration = 10000
     mem_usage_history = [process.memory_info().rss] * iteration
     level_count = img.resolutions["level_count"]
 
+    memory_increment_count = 0
+
     for i in range(iteration):
         location = (
             random.randrange(-2048, 4096 + 2048),
             random.randrange(-2048, 4096 + 2048),
         )
         level = random.randrange(0, level_count)
         _ = img.read_region(location, (256, 256), level)
+        if i == 0 or i == iteration - 1:
+            gc.collect()
         mem_usage_history[i] = process.memory_info().rss
+        if i > 0:
+            if mem_usage_history[i] - mem_usage_history[i - 1] > 0:
+                memory_increment_count += 1
+                print(
+                    f"mem increase (iteration: {i:3d}): "
+                    f"{mem_usage_history[i] - mem_usage_history[i - 1]:4d} "
+                    "bytes"
+                )
 
     print(mem_usage_history)
 
-    # Memory usage difference should be smaller than (iteration) * 100 bytes
-    assert mem_usage_history[-1] - mem_usage_history[1] < iteration * 100
+    # The expected memory usage difference should be smaller than
+    # <iteration> * 256 * 3 bytes
+    # (one line of pixels in the tile image is 256 * 3 bytes)
+    assert mem_usage_history[-1] - mem_usage_history[1] < iteration * 256 * 3
+    # The memory usage increment should be less than 1% of the iteration count.
+    assert memory_increment_count < iteration * 0.01
 
 
 def test_tiff_iterator(testimg_tiff_stripe_4096x4096_256):