RFD 181: Improving Manta Storage Unit Cost (MinIO) Discussion

[KodyKantor]

This issue represents an opportunity for discussion of RFD 181 Improving Manta Storage Unit Cost (MinIO) while it remains in a pre-published state.

[trentm]

In the process of going to Manta v2 we discussed changing the layout of files on storage nodes. In Manta v1 files are written in a single directory for each user. The Manta v2 proposal is to create more directories based on the object UUID.

Clarifying a possible confusion: "mantav2" is a version of Manta that drops jobs/snaplinks/MPU and adds the Buckets API in addition to the existing "Directory" API. So far the storage file layout change (MANTA-4591) is only proposed for objects created using the Buckets API. I'm not sure if that changes anything for this RFD, however.

[KodyKantor]

So far the storage file layout change (MANTA-4591) is only proposed for objects created using the Buckets API. I'm not sure if that changes anything for this RFD, however.

Okay, that makes sense, thanks for the clarification. I'll modify the RFD to make it clear that there are two file layouts in Manta v2 deployments.

[KodyKantor]

@trentm I discovered today that MinIO supports mostly arbitrary nesting of directories on storage nodes. This is good news! I updated the RFD to explain how we can achieve a very similar layout to MANTA-4591 without any MinIO changes and small changes to the layout that up-stack tools would expect.