ceph - sepia - 2024-07-22

Timestamp (UTC)Message
2024-07-22T12:13:14.660Z
<Adam King> seeing
```2024-07-22 12:03:54,034.034 ERROR:teuthology.report:POST to <https://paddles.front.sepia.ceph.com/runs/adking-2024-07-22_12:03:49-orch:cephadm-wip-adk-testing-2024-07-19-0851-distro-default-smithi/jobs/> failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>```
trying to schedule a run. The first two jobs went up fine, and then the rest couldn't schedule
2024-07-22T12:30:03.721Z
<Kyrylo Shatskyy> mira013:
here is what can be observed in loogs:
```[9061020.173808] blk_update_request: critical medium error, dev sda, sector 46350832 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[9061020.193255] EXT4-fs warning (device sda1): htree_dirblock_to_tree:1067: inode #1556817: lblock 0: comm du: error -5 reading directory block
[9061026.132267] sd 0:0:0:1: [sda] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=5s
[9061026.132286] sd 0:0:0:1: [sda] tag#19 Sense Key : Medium Error [current] 
[9061026.132294] sd 0:0:0:1: [sda] tag#19 Add. Sense: Unrecovered read error
[9061026.132302] sd 0:0:0:1: [sda] tag#19 CDB: Read(10) 28 00 02 c3 fc 20 00 00 08 00
[9061026.132306] blk_update_request: critical medium error, dev sda, sector 46398496 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[9061026.151832] EXT4-fs warning (device sda1): htree_dirblock_to_tree:1067: inode #1458890: lblock 0: comm du: error -5 reading directory block
[9061032.040063] sd 0:0:0:1: [sda] tag#24 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=5s
[9061032.040083] sd 0:0:0:1: [sda] tag#24 Sense Key : Medium Error [current] 
[9061032.040091] sd 0:0:0:1: [sda] tag#24 Add. Sense: Unrecovered read error
[9061032.040099] sd 0:0:0:1: [sda] tag#24 CDB: Read(10) 28 00 02 c3 fc 18 00 00 08 00
[9061032.040103] blk_update_request: critical medium error, dev sda, sector 46398488 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[9061032.059744] EXT4-fs warning (device sda1): htree_dirblock_to_tree:1067: inode #1458889: lblock 0: comm du: error -5 reading directory block
ubuntu@mira013:~$ ```
2024-07-22T12:30:31.644Z
<Kyrylo Shatskyy> is the disk going to be dead?
2024-07-22T16:16:16.135Z
<Adam Kraitman> Yes I think I will move the OS the sdb for now until we will replace the disk
2024-07-22T17:10:47.793Z
<Adam King> Whatever this was appears to be resolved now. Was able to schedule a run with no issues

Any issue? please create an issue here and use the infra label.