Why "The creation and management of RBD images is not within the scope of this component." ?

[xin3liang]

Hi,
I think Ceph-nvmeof Gateway is a very cool project because it supports NVMe-oF.
But why the description says "The creation and management of RBD images is not within the scope of this component."
What are the concerns excluding these management APIs?

As a comparison, in the Ceph iSCSI Gateway case, we can use a consistency client(gwcli) to do all the things, like create an RBD image, map and export it into an iSCSI disk, etc.

IMO, if lacking the creation and management of RBD images, Ceph-nvmeof Gateway won't become a complete Ceph Gateway like Ceph iSCSI Gateway.

On the other hand, considering integration with other client projects, like OpenStack Cinder, and Kubernetes Ceph-csi/Rook, if Ceph-nvmeof Gateway becomes a complete one, on the client side, it will be more easily only using the nvmeof-cli to communicate with the Ceph-nvmeof Gateway without using the rbd tool and librbd, and thus make the client more lightweight.

[oritwas]

The NVMe-oF will create an RBD image if it does not exist in the create namespace command.
It will not provide you, with all the image life cycle capabilities RBD provides. Both gateways provide similar functionality in their CLI with differences resulting from the different protocols.
For a single point of management, you should use Ceph CLI and APIs and not the gateway CLI (we have the APIs and CLI is WIP).

Cinde or CSI interfaces are for provisioning and attaching the volumes, they do not handle the data path like the NVMe-oF gateway. On the contrary, the gateway is all about implementing the NMVe-oF protocol (data path). You cannot compare those components
Rook is a Ceph orchestrator for Kubernetes and uses Ceph CSI for PV provisioning. It will be used to run the new gateway in K8s

I am a big advocate of doing fewer things and doing them well!

[xin3liang]

Hi @oritwas, thanks for the clarification. Which really help me a lot to understand the status better.

The NVMe-oF will create an RBD image if it does not exist in the create namespace command.

The project description and test step 2
"2. In order to start working with the NVMe-oF gateway, we need to create an RBD image first"
of README.md made me feel that Ceph-nvmeof Gateway won't support RBD image create/delete/resize. I should check the code to confirm that.

It will not provide you, with all the image life cycle capabilities RBD provides. Both gateways provide similar functionality in their CLI with differences resulting from the different protocols. For a single point of management, you should use Ceph CLI and APIs and not the gateway CLI (we have the APIs and CLI is WIP).

Cinde or CSI interfaces are for provisioning and attaching the volumes, they do not handle the data path like the NVMe-oF gateway. On the contrary, the gateway is all about implementing the NMVe-oF protocol (data path).

I now understand and agree that the Ceph NVMe-oF Gateway should focus on the data path related things, that is RBD image to NVMe-oF disk transformation like Ceph iSCSI Gateway, and it shouldn't provide the whole life cycle management of RBD image, such as snapshot, clone management, etc.

You cannot compare those components Rook is a Ceph orchestrator for Kubernetes and uses Ceph CSI for PV provisioning. It will be used to run the new gateway in K8s

I don't intend to compare Ceph NVMe-oF Gateway with those things, I am thinking about the integration with Ceph NVMe-oF Gateway.

For Cinder integration,
I check the Ceph iSCSI driver, it is based on the rbd driver and provide other image management capabilities via rbd driver. As long as Ceph NVMe-oF Gateway implements similar APIs as Ceph iSCSI Gateway, it should be OK that Cinder integrates with Ceph NVMe-oF Gateway.

For Ceph-CSI/k8s integration,
how to? And what should be done? Write a gateway client to do similar things as rbd-nbd?
And does the Ceph NVMe-oF Gateway team have any plan for this?
Maybe we can fire another topic to discuss this: #727

I am a big advocate of doing fewer things and doing them well!

Indeed!!

[caroav]

One thing for sure, we need to update the README, I will take care of that.

[xin3liang]

Fired two derived discussions about the Ceph NVMe-oF Gateway integration.
K8s integration: #727
OpenStack integration: #728