Renaming a folder duplicates it

[PVince81]

Steps to reproduce

1. docker run -p 9000:9000 minio/minio server /data{1...12}
2. Create a bucket "nextcloud-dev" in the Mino console
3. Enable versioning on the bucket
4. Setup Nextcloud
5. Enable "files_versions_s3" app
6. Configure S3 as external storage, mounted at "/s3" and using access key "minioadmin" and secret "minioadmin" and path style access
7. In the "s3" storage, create a folder "test"
8. Put some files into "test"
9. Rename "test" to "test_renamed" in the web UI

Expected result

Folder "test" is renamed to "test_renamed" with no leftovers.

Actual result

Folder "test" and its contents are renamed to "test_renamed".
But there's another folder "test" without any contents.

Versions

Nextcloud v21.0.5
files_versions_s3 v0.1.6

Logs

No relevant log entries are added.

Notes

This is similar to nextcloud/server#29142 but behaves differently with versioning enabled.
It is possible that nextcloud/server#29142 is the root cause and maybe this app doesn't handle the error correctly.

Interestingly there is no log entry, unlike nextcloud/server#29142 so perhaps this app is using a different code path to do the copy.

[PVince81]

there was a potential fix on server master that did not get backported: nextcloud/server#24185

however, despite us telling S3 to delete the object, it doesn't delete it 😦
I've tried with both deleteObject($path) and deleteObject($path . '/')...

[PVince81]

even deleting the previous empty folder afterwards in the web UI doesn't work, probably as it's also calling deleteObject()

debugging deeper I do see that the S3 request returned with a 204 No Content, so it seems it's been deleted properly, and yet the object is still there...

I'm wondering if it's the versioning thing that is getting in the way

[PVince81]

even if I create an empty folder in the Minio web console, put something in it, then delete the file.
the file is gone
but if I delete the folder, it doesn't go away.
but also, when I open it, the UI looks different:

and after finding minio/minio#10914 it seems that maybe it's "marked as deleted" in some way but the UI isn't clear about this

so maybe there's a way to force delete over the API...

[PVince81]

I've tried adding explicit deletion of the versions in the batchDelete function on the server:

			do {
				// delete versions of container, if any
				$objects = $connection->listObjectVersions([
					'Bucket' => $this->bucket,
					'Prefix' => $path . '/',
				]);
				if (isset($objects['DeleteMarkers'])) {
					/*
					$versionIds = array_map(function ($object) {
						return $object['VersionID'];
					}, $objects['DeleteMarkers']);
					 */
					/*
					$this->getConnection()->deleteObjects([
						'Bucket' => $this->bucket,
						'Delete' => [
							//'Objects' => $versionIds,
							'Objects' => $objects['DeleteMarkers'],
						]
					]);
					 */

					foreach ($objects['DeleteMarkers'] as $versionObject) {
						$this->getConnection()->deleteObject([
							'Bucket' => $this->bucket,
							'Key' => $versionObject['Key'],
							'VersionId' => $versionObject['VersionId'],
						]);
					}
				}
			} while ($objects['IsTruncated']);

but neither operation raise any error, they just say "all is fine" but nothing happens:

 ▾ $this->result = (Aws\Result [2])
  \
   ▾ $this->result->data = (array [4])
    \
     ⬦ $this->result->data["DeleteMarker"] = (bool) 0
     |
     ⬦ $this->result->data["VersionId"] = (string [36]) `3117810f-6838-470a-8d0a-41dba6faa444`
     |
     ⬦ $this->result->data["RequestCharged"] = (string [0]) ``
     |
     ▾ $this->result->data["@metadata"] = (array [4])
      \
       ⬦ $this->result->data["@metadata"]["statusCode"] = (int) 204
       |
       ⬦ $this->result->data["@metadata"]["effectiveUri"] = (string [128]) `http://localhost:9000/nextcloud-dev/flat3/abdullah-ahmad-SLBjcE5IQxo-unsplash.jpg?versionId=3117810f-6838-470a-8d0a-41dba6faa444`
       |
       ▸ $this->result->data["@metadata"]["headers"] = (array [10])
       |
       ▸ $this->result->data["@metadata"]["transferStats"] = (array [1])
      /
   ⬦ $this->result->monitoringEvents = (array)

the zombie folder still exists

[PVince81]

alright, so deleting the marker is probably wrong as it is stated that the contents of the folder will reappear: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeletingObjects.html

[PVince81]

PR here: nextcloud/server#29165

I had developed this against v21.0.5 first and could see that the folder was properly deleted.
Removing the "DeletedMarker" was not enough, also needed to delete all "Versions"

[PVince81]

solution was reverted as it would lose versions since those are not copied over, see #13

[PVince81]

the new solution is nextcloud/server#29220 where folders marked as deleted are skipped in listing

[PVince81]

fixed through nextcloud/server#29220