Add --checksum param to pipe, put, cp, mirror

[klauspost]

Description

Add checksums to pipe, cp, mirror commands.

Add checksum to uploaded object. Values: MD5, CRC32, CRC32C, SHA1 or SHA256. Requires server trailing headers support like AWS, MinIO and possibly others.

--md5 is hidden, but still applied.

Uploads to buckets with object locking will still use MD5.

Bonus: mc stat shows checksum information.

How to test this PR?

Single/multipart upload (mc cp):

λ mc cp 0.txt myminio/testbucket -checksum=SHA256 -q &&mc stat myminio/testbucket/0.txt -q
`e:/gopath/src/github.com/minio/mc/0.txt` -> `myminio/testbucket/0.txt`
Total: 597.67 KiB, Transferred: 597.67 KiB, Speed: 59.64 MiB/s
Name      : 0.txt
Date      : 2024-09-24 16:30:40 CEST
Size      : 598 KiB
ETag      : 5d9003348d6762e91a212b50460b287e
VersionID : c21d90cb-d302-4707-a53d-d3c695bf8d4f
Type      : file
Checksum  : SHA256:X3pWjnxkqONadmZwlOAAF4+jRwLEZnTa/WGKc+s/M4w=
Encryption: SSE-KMS (arn:aws:kms:my-minio-key)
Metadata  :
  Content-Type: text/plain


λ mc cp ab.txt myminio/testbucket -checksum=SHA256 -q &&mc stat myminio/testbucket/ab.txt -q
`e:/gopath/src/github.com/minio/mc/ab.txt` -> `myminio/testbucket/ab.txt`
Total: 381.47 MiB, Transferred: 381.47 MiB, Speed: 239.29 MiB/s
Name      : ab.txt
Date      : 2024-09-24 16:31:10 CEST
Size      : 382 MiB
ETag      : a4831df44d6a59fff21881ccd7d5e86f-24
VersionID : d5cb7670-267a-4e9d-8a8d-a85fd60943c9
Type      : file
Checksum  : SHA256:oKTfpng60iS2rRaWKo8/Ep3wTOHmam8rQZbhzQI1tF0=-24
Encryption: SSE-KMS (arn:aws:kms:my-minio-key)
Metadata  :
  Content-Type: text/plain

(mc put)

λ mc put ab.txt myminio/testbucket -checksum=SHA256 -q &&mc stat myminio/testbucket/ab.txt -q
`e:/gopath/src/github.com/minio/mc/ab.txt` -> `myminio/testbucket/ab.txt`
Total: 381.47 MiB, Transferred: 381.47 MiB, Speed: 758.18 MiB/s
Name      : ab.txt
Date      : 2024-09-24 16:41:42 CEST
Size      : 382 MiB
ETag      : e5c129e03bc360798e6c862fc971f468-24
VersionID : 9ecadd2a-6846-4338-979a-abdbacd3fe95
Type      : file
Checksum  : SHA256:oKTfpng60iS2rRaWKo8/Ep3wTOHmam8rQZbhzQI1tF0=-24
Encryption: SSE-KMS (arn:aws:kms:my-minio-key)
Metadata  :
  Content-Type: text/plain


λ mc put 0.txt myminio/testbucket -checksum=SHA256 -q &&mc stat myminio/testbucket/0.txt -q
`e:/gopath/src/github.com/minio/mc/0.txt` -> `myminio/testbucket/0.txt`
Total: 597.67 KiB, Transferred: 597.67 KiB, Speed: 75.67 MiB/s
Name      : 0.txt
Date      : 2024-09-24 16:42:00 CEST
Size      : 598 KiB
ETag      : b99f4b15c1bd411e130e2b0c98e7df86
VersionID : c8e6b086-d794-4263-8ccd-c52e5cce1749
Type      : file
Checksum  : SHA256:X3pWjnxkqONadmZwlOAAF4+jRwLEZnTa/WGKc+s/M4w=
Encryption: SSE-KMS (arn:aws:kms:my-minio-key)
Metadata  :
  Content-Type: text/plain

(mc pipe):

λ cat 0.txt | mc pipe myminio/testbucket/0.txt -checksum=SHA256&&mc stat myminio/testbucket/0.txt -q
 552.00 KiB / ? [                                                                                                       =                               ] 2.01 MiB/s Name      : 0.txt
Date      : 2024-09-24 16:47:59 CEST
Size      : 598 KiB
ETag      : 062320249d887571850f2daea61b3e1e-1
VersionID : dfb6d042-248b-44fd-8dc7-21273c916666
Type      : file
Checksum  : SHA256:T4uVbgD8TgAJxPDPpgtktYfyVG3AkxP39l/BbWZFFHA=-1
Encryption: SSE-KMS (arn:aws:kms:my-minio-key)
Metadata  :
  Content-Type: text/plain

Types of changes

• New feature (non-breaking change which adds functionality)