ceph - cephadm - 2024-07-24

Timestamp (UTC)Message
2024-07-24T09:17:54.961Z
<verdurin> I'm running a `ceph orch upgrade` and it says it has started, but I see no signs of progress. The status shows as:
```{
    "target_image": "[quay.io/ceph/ceph:v16.2.15](http://quay.io/ceph/ceph:v16.2.15)",
    "in_progress": true,
    "which": "Upgrading all daemon types on all hosts",
    "services_complete": [],
    "progress": "",
    "message": "",
    "is_paused": false
}```
and `ceph -s` shows:
```  progress:
    Upgrade to [quay.io/ceph/ceph:v16.2.15](http://quay.io/ceph/ceph:v16.2.15) (0s)
      [............................]```
and has done so for about ten minutes now.

`/var/log/ceph/cephadm.log` just shows regular processes, nothing related to the upgrade.

It seems to be stuck on something, just can't see where...
2024-07-24T09:33:32.669Z
<Eugen Block> The active MGR logs upgrade steps and progress. Check the log for any issues. You could pause the upgrade, fail the mgr over to a different one and resume the upgrade. Then check the logs of the new active MGR.
2024-07-24T10:45:04.987Z
<verdurin> It turned out to be a problem with connecting to a specific host.
When I removed that host (which wasn't currently active anyway), it all started working. I guess it was trying to apply a change and failing because it couldn't reach that host.
2024-07-24T10:45:42.539Z
<Eugen Block> Yes, that's plausible.
2024-07-24T10:46:10.584Z
<verdurin> Couldn't see any specific upgrade errors in the logs, so I think it must have been that.
Thanks.
2024-07-24T10:47:07.798Z
<Eugen Block> I would have expected some error pointing to a failing connection to one host. But it's good that you figured it out.
2024-07-24T10:47:42.046Z
<verdurin> Yes, though it wasn't immediately obvious that was related to the upgrade. Suppose it must have been.
2024-07-24T13:30:49.376Z
<Deepika Upadhyay> hello, does anyone know how do we handle rgw ops logs in cephadm, want to have this available in rook as well are there any  <https://github.com/rook/rook/issues/14372> we used this workaround for cephadm
```ceph config set global rgw_ops_log_file_path '/dev/stderr'
ceph config set global rgw_enable_ops_log true```
but is there any better way of having rgw ops logs with cephadm

Any issue? please create an issue here and use the infra label.