2024-08-20T02:35:06.402Z | <Sepia OpenShift> [FIRING:1] teuthology (SmithiQueuePaused metrics [teuthology.front.sepia.ceph.com:61764](http://teuthology.front.sepia.ceph.com:61764) teuthology-exporter smithi 1390635 openshift-user-workload-monitoring/user-workload teuthology-exporter warning) | https:///console-openshift-console.apps.os.sepia.ceph.com/monitoring/#/alerts?receiver=%23sepia |
2024-08-20T12:21:53.439Z | <Zac Dover> @Laura Flores, I'm seeing repeated API test failures on this PR: <https://github.com/ceph/ceph/pull/59294> |
2024-08-20T12:22:24.607Z | <Zac Dover> Here's the console output (which I didn't find helpful, but maybe someone smarter will): <https://jenkins.ceph.com/job/ceph-api/80073/console> |
2024-08-20T12:41:43.892Z | <Adam Kraitman> Hey @Guillaume Abrioux let me see what I can do |
2024-08-20T13:39:07.823Z | <Adam Kraitman> Hey @Guillaume Abrioux I think it should now |
2024-08-20T13:39:25.361Z | <Adam Kraitman> Hey @Guillaume Abrioux I think it should work now |
2024-08-20T15:13:51.327Z | <Laura Flores> ```[1752/1788] Building CXX object src/rgw/CMakeFiles/rgw_a.dir/rgw_rest_info.cc.o
ninja: build stopped: interrupted by user.
Build step 'Execute shell' marked build as failure
[PostBuildScript] - [INFO] Executing post build scripts.
[PostBuildScript] - [INFO] Build does not have any of the results [ABORTED]. Did not execute build step #0.
Archiving artifacts
[Checks API] No suitable checks publisher found.
Setting status of 34cb4f89b55a7aceab8761a3d8dd322fc09bd5f2 to FAILURE with url <https://jenkins.ceph.com/job/ceph-api/80073/> and message: 'ceph API tests failed
'
Using context: ceph API tests
Finished: FAILURE```
The failure is definitely vague. I would retrigger it once more, and if the same thing happens, it might be an issue with that node (mira015). Then we can have Dan or Adam Kraitman take a look at it. |
2024-08-20T15:31:05.463Z | <Zac Dover> Thanks, Laura. You are the smarter person foretold by (my own) prophecy. |
2024-08-20T18:25:06.405Z | <Sepia OpenShift> [FIRING:1] teuthology (SmithiQueuePaused metrics [teuthology.front.sepia.ceph.com:61764](http://teuthology.front.sepia.ceph.com:61764) teuthology-exporter smithi 1390635 openshift-user-workload-monitoring/user-workload teuthology-exporter warning) | https:///console-openshift-console.apps.os.sepia.ceph.com/monitoring/#/alerts?receiver=%23sepia |
2024-08-20T19:15:04.871Z | <Casey Bodley> still seeing a lot of these failures |
2024-08-20T19:34:37.861Z | <Dan Mick> sigh. yes, I ended up not knowing how to best proceed |
2024-08-20T19:34:50.413Z | <Dan Mick> I tried to find all files not installed from packages but it was clunky and seemed dangerous |
2024-08-20T19:34:59.338Z | <Dan Mick> and then other fires started |
2024-08-20T19:36:47.479Z | <Casey Bodley> the presence of /usr/lib/x86_64-linux-gnu/cmake/protobuf/protobuf-targets.cmake seems to be the immediate issue breaking cmake |
2024-08-20T19:39:23.145Z | <Dan Mick> I guess I can just blast that until I can spend time to devise a better solution |
2024-08-20T19:39:34.654Z | <Dan Mick> we so need containerized builds |
2024-08-20T20:12:48.528Z | <Dan Mick> as far as I can tell that file isn't present on any builder. as far as I can tell, it wasn't before I started today |
2024-08-20T20:15:29.731Z | <Casey Bodley> <https://jenkins.ceph.com/job/ceph-pull-requests/141739/> from today on braggi09 |
2024-08-20T20:17:09.878Z | <Dan Mick> $ ssh braggi09 ls -l /usr/lib/x86_64-linux-gnu/cmake/protobuf/protobuf-targets.cmake
Warning: Permanently added 'braggi09' (ED25519) to the list of known hosts.
ls: cannot access '/usr/lib/x86_64-linux-gnu/cmake/protobuf/protobuf-targets.cmake': No such file or directory |
2024-08-20T20:17:51.964Z | <Casey Bodley> fun |
2024-08-20T20:18:43.069Z | <Casey Bodley> is there a /usr/lib/x86_64-linux-gnu/cmake/protobuf/protobuf-config.cmake ? |
2024-08-20T20:19:25.183Z | <Casey Bodley> that's listed under 'Call Stack' in the cmake error |
2024-08-20T20:20:01.511Z | <Dan Mick> $ ssh braggi09 ls -l /usr/lib/x86_64-linux-gnu/cmake/protobuf/
total 32
-rw-r--r-- 1 root root 544 Feb 24 20:55 protobuf-config.cmake
-rw-r--r-- 1 root root 2614 Feb 24 20:55 protobuf-config-version.cmake
-rw-r--r-- 1 root root 6073 Feb 24 20:55 protobuf-generate.cmake
-rw-r--r-- 1 root root 5760 Feb 24 20:55 protobuf-module.cmake
-rw-r--r-- 1 root root 281 Feb 24 20:55 protobuf-options.cmake
-rw-r--r-- 1 root root 2349 Feb 24 20:55 protobuf-targets-noconfig.cmake |
2024-08-20T20:22:34.219Z | <Dan Mick> when I get this fog image crap done I'll go back to trying to come up with a safe "remove all unpackaged files" script |
2024-08-20T22:13:46.421Z | <Laura Flores> Hey all, FYI I am upgrading the gibba cluster to the latest squid-release (untagged 19.1.1). |
2024-08-20T22:14:23.441Z | <Laura Flores> Follow more updates in <#C04K9SK5M2S|> |