> But ceph is fundamentally a distributed object store, shared LUNs with block level multi-writer is fundamentally a tightly coupled solution.

There is Ceph and then there is CephFS, "a POSIX-compliant file system built on top of Ceph’s distributed object store, RADOS":

* https://docs.ceph.com/en/latest/cephfs/

* https://pve.proxmox.com/wiki/Storage:_CephFS