2024-08-29T06:41:14.459Z

<Venky Shankar> @Patrick Donnelly PTAL: <https://github.com/ceph/ceph/pull/58547> (cc @Milind Changire)

2024-08-29T09:47:29.004Z

<Igor Golikov> Hi, I am trying to mount cephfs using FUSE, as described in the docs. It seems that my cluster is runnincluster:
    id:     526d8004-1a6c-4c30-ba58-5b770abdb499
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum a,b,c (age 4m)
    mgr: x(active, since 4m)
    mds: 2/2 daemons up, 1 standby
    osd: 3 osds: 3 up (since 4m), 3 in (since 4m)

  data:
    volumes: 2/2 healthy
    pools:   5 pools, 161 pgs
    objects: 46 objects, 453 KiB
    usage:   2.0 MiB used, 3.0 GiB / 3 GiB avail
    pgs:     161 active+clean

and

2024-08-29T09:47:42.800Z

<Igor Golikov> Hi, I am trying to mount cephfs using FUSE, as described in the docs. It seems that my cluster is running:cluster:
    id:     526d8004-1a6c-4c30-ba58-5b770abdb499
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum a,b,c (age 4m)
    mgr: x(active, since 4m)
    mds: 2/2 daemons up, 1 standby
    osd: 3 osds: 3 up (since 4m), 3 in (since 4m)

  data:
    volumes: 2/2 healthy
    pools:   5 pools, 161 pgs
    objects: 46 objects, 453 KiB
    usage:   2.0 MiB used, 3.0 GiB / 3 GiB avail
    pgs:     161 active+clean

and

2024-08-29T09:49:17.312Z

<Igor Golikov> I created the user
[client.foo]
	key = AQBBQtBm5qZJNRAA/lOZOXFfWFkemKp7UB/SeA==
	caps mds = "allow rw fsname=cephfs"
	caps mon = "allow r fsname=cephfs"
	caps osd = "allow rw tag cephfs data=cephfs"

and volume (cephfs)
name: a, metadata pool: cephfs.a.meta, data pools: [cephfs.a.data ]
name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ]

and now I am trying to run
```sudo ceph-fuse --no-mon-config -n client.foo /mnt/mycephfs -k /etc/ceph/ceph.client.foo.keyring```

2024-08-29T09:49:38.850Z

<Igor Golikov> it just hangs with

sudo ceph-fuse --no-mon-config -n client.foo /mnt/mycephfs -k /etc/ceph/ceph.client.foo.keyring

2024-08-29T09:49:59.359Z

<Igor Golikov> it just hangs with

2024-08-29T12:43:24.832+0300 fb206f9bc020 -1 WARNING: all dangerous and experimental features are enabled.
2024-08-29T12:43:24.832+0300 fb206f9bc020 -1 WARNING: all dangerous and experimental features are enabled.
2024-08-29T12:43:24.834+0300 fb206f9bc020 -1 WARNING: all dangerous and experimental features are enabled.
2024-08-29T12:43:24.836+0300 fb206f9bc020 -1 init, newargv = 0xae5418440740 newargc=15
ceph-fuse[6940]: starting ceph client

2024-08-29T09:50:06.951Z

<Igor Golikov> and doesnt progress from here...

2024-08-29T11:19:42.852Z

<Milind Changire> @Igor Golikov try using `client.admin` instead

2024-08-29T11:20:20.575Z

<Milind Changire> the name of the filesystem is `a` and not `cephfs`

2024-08-29T11:22:54.163Z

<Milind Changire> also, if you are starting up the vstart cluster, you don't need to keep the config in /etc/ceph
its already available under `./build`
but, I guess you are using the docs to set up the cluster
which is bit of an odd thing to do for a vstart cluster 🙂

2024-08-29T11:25:47.261Z

<jcollin> `[jcollin@telecast build]$ cat ceph.conf | grep 'mon host'`
        `mon host =  [v2:127.0.0.1:*40284*,v1:127.0.0.1:40285]`
`[jcollin@telecast build]$ sudo ./bin/ceph-fuse --client_mds_namespace a -m 127.0.0.1:*40284* /mnt/cephfs_mount_point`
`ceph-fuse[108231]: starting ceph client`
`ceph-fuse[108231]: starting fuse`

2024-08-29T11:33:38.561Z

<Milind Changire> @Igor Golikov here's how I build and work with cephfs in a vstart cluster:
```Build (on Fedora 40 with Python 3.12):
$ ./do_cmake.sh -G Ninja -DWITH_CCACHE=ON -DWITH_MANPAGE=OFF -DWITH_RDMA=OFF -DWITH_RBD=OFF -DWITH_OPENLDAP=OFF -DWITH_XFS=OFF -DWITH_KRBD=OFF -DWITH_JAEGER=OFF -DWITH_RADOSGW=OFF -DWITH_LTTNG=OFF -DWITH_BABELTRACE=OFF -DWITH_TESTS=OFF -DWITH_MGR_DASHBOARD_FRONTEND=OFF -DWITH_SYSTEMD=OFF -DWITH_PYTHON3=3.12

$ cd build && ninja -j 8```
```Launch:
$ MDS=1 MON=1 MGR=1 OSD=1 FS=1 NFS=0 RGW=0 ../src/vstart.sh -d -b -n --without-dashboard```
```Mount:
$ sudo echo user_allow_other > /etc/fuse.conf

$ cd build
$ chmod go+r keyring

$ sudo mkdir /mnt/mycephfs
$ sudo chown $USER:$USER /mnt/mycephfs
$ ./bin/ceph-fuse --id admin --client_fs a --conf $(pwd)/ceph.conf /mnt/mycephfs```
```Unmount:
$ umount /mnt/mycephfs

Stop cluster:
$ ../src/stop.sh```

2024-08-29T12:34:56.642Z

<Igor Golikov> thanks @Milind Changire for the detailed guidance! will check it

2024-08-29T14:58:06.162Z

<gregsfortytwo> @Venky Shankar did you own the recording for our CDS session? @Igor Golikov would like to see it

2024-08-29T14:58:41.608Z

<Venky Shankar> Neha owns the recording I believe.

2024-08-29T16:39:33.461Z

<Venky Shankar> <https://drive.google.com/file/d/1zDCn-QZrDPBQ3XOJt4eY4EVmDDcNKqGP/view>

2024-08-29T16:39:54.381Z

<Venky Shankar> The recording links directly in the invite itself 🙂

2024-08-29T16:42:22.854Z

<gregsfortytwo> Ah thanks, I was looking at the etherpad

2024-08-29T16:58:48.495Z

<Alex Handy> Anyone been through the process of rebuilding/recovering a CephFS filesystem that has "Metadata damage detected" and is clearly missing a large number of files/directories?

2024-08-29T17:00:21.871Z

<Alex Handy> I've been through the <https://docs.ceph.com/en/reef/cephfs/disaster-recovery-experts/> doc a few times, but despite having seen many different Ceph clusters I'm not sure it's clear to me which processes/tasks to take

2024-08-29T17:24:46.914Z

<gregsfortytwo> Well it would depend on what happened to the cluster and what damage it reports. If it’s missing directories then presumably some PGs got lost. If it was a random subset, you’d need to use the journal tool to flush out all the valid updates it can find, then do the data scan to update the tree to something valid

2024-08-29T17:25:14.759Z

<gregsfortytwo> If you know which pools were/weren’t impacted, you might be able to cut out some of the specific steps

2024-08-29T17:25:41.370Z

<gregsfortytwo> Hopefully you don’t need to reset any of the tables, but it’s always a possibility

2024-08-29T17:38:47.295Z

<Patrick Donnelly> Need approval for merge: <https://github.com/ceph/ceph/pull/59414>

2024-08-29T17:39:04.141Z

<Patrick Donnelly> Needs approval for merge: <https://github.com/ceph/ceph/pull/59309>