ceph - sepia - 2024-09-25

Timestamp (UTC)Message
2024-09-25T11:03:26.907Z
<Guillaume Abrioux> @Adam Kraitman <https://jenkins.ceph.com/job/ceph-pull-requests/143938/>
2024-09-25T11:03:41.348Z
<Guillaume Abrioux> ```OSError: [Errno 28] No space left on device
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/virtualenv/bin/cram", line 7, in 
    sys.exit(cram.main(sys.argv[1:]))
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/virtualenv/lib/python3.10/site-packages/cram/_main.py", line 200, in main
    refout, postout, diff = test()
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/virtualenv/lib/python3.10/site-packages/cram/_cli.py", line 115, in testwrapper
    errfile.close()
OSError: [Errno 28] No space left on device```
2024-09-25T11:50:04.128Z
<Anoop C S> :wq
2024-09-25T13:11:09.904Z
<Naveen Naidu> hey folks, I am trying to ssh into sepia after getting the access, but I am unable to log in. I tried troubleshooting this as mentioned in `Troubleshooting` section of <https://wiki.sepia.ceph.com/doku.php?id=vpnaccess#vpn_client_access>, but I am getting the following error `AUTH: Received control message: AUTH_FAILED`. More logs in ๐Ÿงต
2024-09-25T13:11:20.954Z
<Naveen Naidu> ```โฏ openvpn --config /etc/openvpn/client/sepia.conf --cd /etc/openvpn/client --verb 5
2024-09-25 18:39:49 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
2024-09-25 18:39:49 us=635053 Note: --cipher is not set. OpenVPN versions before 2.5 defaulted to BF-CBC as fallback when cipher negotiation failed in this case. If you need this fallback please add '--data-ciphers-fallback BF-CBC' to your configuration and/or add BF-CBC to --data-ciphers.
2024-09-25 18:39:49 us=635100 Note: '--allow-compression' is not set to 'no', disabling data channel offload.
2024-09-25 18:39:49 us=635119 WARNING: file '/etc/openvpn/client/sepia/secret' is group or others accessible
2024-09-25 18:39:49 us=635197 Current Parameter Settings:
2024-09-25 18:39:49 us=635206   config = '/etc/openvpn/client/sepia.conf'
2024-09-25 18:39:49 us=635213   mode = 0
2024-09-25 18:39:49 us=635220 NOTE: --mute triggered...
2024-09-25 18:39:49 us=635235 290 variation(s) on previous 10 message(s) suppressed by --mute
2024-09-25 18:39:49 us=635247 OpenVPN 2.6.12 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-09-25 18:39:49 us=635262 library versions: OpenSSL 3.2.2 4 Jun 2024, LZO 2.10
2024-09-25 18:39:49 us=635276 DCO version: N/A
2024-09-25 18:39:49 us=636653 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
2024-09-25 18:39:49 us=636671 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
2024-09-25 18:39:49 us=636692 LZO compression initializing
2024-09-25 18:39:49 us=636855 Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
2024-09-25 18:39:49 us=638662 Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1500 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-09-25 18:39:49 us=641138 TCP/UDP: Preserving recently used remote address: [AF_INET]8.43.84.129:1194
2024-09-25 18:39:49 us=641447 Socket Buffers: R=[212992->212992] S=[212992->212992]
2024-09-25 18:39:49 us=641464 UDPv4 link local: (not bound)
2024-09-25 18:39:49 us=641473 UDPv4 link remote: [AF_INET]8.43.84.129:1194
2024-09-25 18:39:49 us=641505 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
WR2024-09-25 18:39:49 us=938309 TLS: Initial packet from [AF_INET]8.43.84.129:1194, sid=145eba24 452e829a
2024-09-25 18:39:49 us=938525 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
WRRWR2024-09-25 18:39:50 us=168445 VERIFY OK: depth=1, O=Redhat, CN=openvpnca-sepia
2024-09-25 18:39:50 us=169439 VERIFY KU OK
2024-09-25 18:39:50 us=169586 Validating certificate extended key usage
2024-09-25 18:39:50 us=169604 NOTE: --mute triggered...
WRWR2024-09-25 18:39:51 us=474815 4 variation(s) on previous 10 message(s) suppressed by --mute
2024-09-25 18:39:51 us=474937 [openvpn-sepia] Peer Connection Initiated with [AF_INET]8.43.84.129:1194
2024-09-25 18:39:51 us=474979 TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1
2024-09-25 18:39:51 us=475128 TLS: tls_multi_process: initial untrusted session promoted to trusted
W2024-09-25 18:39:52 us=724572 SENT CONTROL [openvpn-sepia]: 'PUSH_REQUEST' (status=1)
WRR2024-09-25 18:39:53 us=10396 AUTH: Received control message: AUTH_FAILED
2024-09-25 18:39:53 us=10646 TCP/UDP: Closing socket
2024-09-25 18:39:53 us=10701 SIGTERM[soft,auth-failure] received, process exiting```
2024-09-25T13:12:33.079Z
<Naveen Naidu> Another interesting thing to note is when I tried to cat the `secret` file, I am getting the following result:
```USER@HOST
D1lUFOCf5QBgqSDa+Jb0HCLJrEWuHqTVoe+kYLDqVZYs797p0jbb2CS30TmLd/8M0iJhzpkQUg/qFYWPDcHFlA```
this seems odd, since the `user@host` that I set was different than the one being mentioned here
2024-09-25T13:13:05.450Z
<Naveen Naidu> hey folks, I am trying to ssh into sepia after getting the access, but I am unable to log in. I tried troubleshooting this as mentioned in `Troubleshooting` section of <https://wiki.sepia.ceph.com/doku.php?id=vpnaccess#vpn_client_access>, but I am getting the following error `AUTH: Received control message: AUTH_FAILED`.  More logs in ๐Ÿงต

[Tracker Request](https://tracker.ceph.com/issues/68118?tab=history)
2024-09-25T13:14:52.183Z
<Naveen Naidu> Is there a chance that I messed something up and would need to open a new tracker request with new creds ๐Ÿ˜… ?
2024-09-25T14:15:14.715Z
<Adam Kraitman> Hey @Naveen Naidu please add your comments to the tracker ticket and I will replay there
2024-09-25T14:17:08.349Z
<Shraddha Agrawal> @Naveen Naidu can you confirm that the config is present in the correct directory? It should be `/etc/openvpn/client/sepia/secret` .
2024-09-25T14:29:45.927Z
<Naveen Naidu> @Adam Kraitman thanks, I have updated my comments in the tracker ticket
2024-09-25T14:30:03.465Z
<Naveen Naidu> @Shraddha Agrawal yes, the config is present in the correct directory!
2024-09-25T15:54:11.149Z
<Adam Kraitman> Thanks @Guillaume Abrioux irvingi02 grafana agent was not sending data,  now it's fixed and it will report next time
2024-09-25T15:54:33.276Z
<Guillaume Abrioux> great ๐Ÿ‘

Any issue? please create an issue here and use the infra label.