ceph - cephadm - 2024-10-03

Timestamp (UTC)Message
2024-10-03T08:21:39.501Z
<verdurin> Doesn't show anything...
2024-10-03T08:37:52.247Z
<verdurin> Does it need to be run from a specific node?
2024-10-03T09:18:52.052Z
<verdurin> Okay, ran it again monitoring the log in the Ceph shell, and saw this after trying `check-host`:
```Traceback (most recent call last):
  File "/lib/python3.6/site-packages/execnet/gateway_bootstrap.py", line 48, in bootstrap_exec
    s = io.read(1)
  File "/lib/python3.6/site-packages/execnet/gateway_base.py", line 402, in read
    raise EOFError("expected %d bytes, got %d" % (numbytes, len(buf)))
EOFError: expected 1 bytes, got 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1532, in _remote_connection
    conn, connr = self.mgr._get_connection(addr)
  File "/usr/share/ceph/mgr/cephadm/module.py", line 1371, in _get_connection
    sudo=True if self.ssh_user != 'root' else False)
  File "/lib/python3.6/site-packages/remoto/backends/__init__.py", line 35, in __init__
    self.gateway = self._make_gateway(hostname)
  File "/lib/python3.6/site-packages/remoto/backends/__init__.py", line 46, in _make_gateway
    self._make_connection_string(hostname)
  File "/lib/python3.6/site-packages/execnet/multi.py", line 134, in makegateway
    gw = gateway_bootstrap.bootstrap(io, spec)
  File "/lib/python3.6/site-packages/execnet/gateway_bootstrap.py", line 102, in bootstrap
    bootstrap_exec(io, spec)
  File "/lib/python3.6/site-packages/execnet/gateway_bootstrap.py", line 53, in bootstrap_exec
    raise HostNotFound(io.remoteaddress)
execnet.gateway_bootstrap.HostNotFound: -F /tmp/cephadm-conf-5s160f4_ -i /tmp/cephadm-identity-vsq_nl16 -o ServerAliveInterval=7 -o ServerAliveCountMax=3 centos@<host ip>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/module.py", line 986, in check_host
    error_ok=True, no_fsid=True)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1334, in _run_cephadm
    with self._remote_connection(host, addr) as tpl:
  File "/lib64/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1566, in _remote_connection
    raise OrchestratorError(msg) from e
orchestrator._interface.OrchestratorError: Failed to connect to <FQDN> (<host ip>).
Please make sure that the host is reachable and accepts connections using the cephadm SSH key```

Any issue? please create an issue here and use the infra label.