2024-10-28T15:07:31.387Z

<Rost Khudov> Hello team,
I think I found a problem in `squid` branch of ceph
There are 2 failed RGW bucket notification tests `test_ps_s3_persistent_topic_configs_max_retries` and `test_ps_s3_persistent_topic_configs_ttl`
```NameError: name 'get_stats_persistent_topic' is not defined```
Looks like it was a problem with backport, because I can see function in `main` branch: <https://github.com/ceph/ceph/blob/main/src/test/rgw/bucket_notification/test_bn.py#L562>

2024-10-28T15:07:44.492Z

<Rost Khudov> How do I proceed with it to fix `squid` branch?

2024-10-28T15:52:57.906Z

<Casey Bodley> thanks. this is showing up in [recent squid runs](https://qa-proxy.ceph.com/teuthology/teuthology-2024-10-25_21:40:02-rgw-squid-distro-default-smithi/7967325/teuthology.log):
```2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:======================================================================
2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:ERROR: test pushing http s3 notification on master
2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:----------------------------------------------------------------------
2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:Traceback (most recent call last):
2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.9/site-packages/nose/case.py", line 198, in runTest
2024-10-26T04:39:57.082 INFO:teuthology.orchestra.run.smithi038.stderr:    self.test(*self.arg)
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 1631, in test_ps_s3_notification_push_http_on_master
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:    hostname = get_ip_http()
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:NameError: name 'get_ip_http' is not defined
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:======================================================================
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:ERROR: test pushing cloudevents notification on master
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:----------------------------------------------------------------------
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:Traceback (most recent call last):
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.9/site-packages/nose/case.py", line 198, in runTest
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:    self.test(*self.arg)
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 1715, in test_ps_s3_notification_push_cloudevents_on_master
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:    hostname = get_ip_http()
2024-10-26T04:39:57.083 INFO:teuthology.orchestra.run.smithi038.stderr:NameError: name 'get_ip_http' is not defined
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:======================================================================
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:ERROR: test persistent topic configurations with time_to_live
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:----------------------------------------------------------------------
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:Traceback (most recent call last):
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.9/site-packages/nose/case.py", line 198, in runTest
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:    self.test(*self.arg)
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 3423, in test_ps_s3_persistent_topic_configs_ttl
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:    ps_s3_persistent_topic_configs(persistency_time, config_dict)
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:  File "/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py", line 3373, in ps_s3_persistent_topic_configs
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:    get_stats_persistent_topic(topic_name, number_of_objects)
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:NameError: name 'get_stats_persistent_topic' is not defined
2024-10-26T04:39:57.084 INFO:teuthology.orchestra.run.smithi038.stderr:-------------------- >> begin captured logging << --------------------```

2024-10-28T15:56:53.757Z

<Rost Khudov> are you going to fix this one, right?

2024-10-28T16:14:13.063Z

<Casey Bodley> @Yuval Lifshitz any idea which backport went wrong? should we create a squid tracker for this?

2024-10-28T16:23:29.849Z

<Rost Khudov> and also, looks like there is some problem with 19.2.0 tag as well
```FAIL: test persistent topic configurations with time_to_live
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/ceph-src/src/test/rgw/bucket_notification/test_bn.py", line 3437, in test_ps_s3_persistent_topic_configs_ttl
    ps_s3_persistent_topic_configs(persistency_time, config_dict)
  File "/ceph-src/src/test/rgw/bucket_notification/test_bn.py", line 3383, in ps_s3_persistent_topic_configs
    assert_equal(parsed_result['Topic Stats']['Entries'], 0)
AssertionError: 10 != 0```

2024-10-28T16:31:23.426Z

<Casey Bodley> @Shilpa Jagannath noticed a regression in <https://github.com/ceph/ceph/pull/59303#issuecomment-2434342073>

2024-10-28T17:59:14.797Z

<Yuval Lifshitz> @Casey Bodley should i just go on and fix the issue, or do you want to revert the bad merges, and redo the backports?
btw, looks like these are bugs in the tests (missing functions). how did the backport PRs passed teuthology?

2024-10-28T18:52:17.431Z

<Casey Bodley> no need to revert, you can open a pr to fix directly. just say in the commit message why it isn't a cherry-pick (because it fixes an earlier backport)