Project

General

Profile

Actions

Bug #65308

open

qa: fs was offline but also unexpectedly degraded

Added by Rishabh Dave about 1 month ago. Updated 25 days ago.

Status:
New
Priority:
Normal
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Link to the failure - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625691/.

Description of this job -

fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health}

Failure reason -

"2024-03-27T08:20:00.000142+0000 mon.smithi028 (mon.0) 868 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log 

Health warning for FS being offline is expected since the command ceph fs fail was run before running ceph fs volume rename. But the health warning for FS being degraded isn't expected. This warning is first seen in output of command -

2024-03-27T08:19:56.103 DEBUG:teuthology.orchestra.run.smithi028:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 288ca2e2-ec11-11ee-95d0-87774f69a715 -- bash -c 'ceph fs set foo refuse_client_session true

Code for this is located here - https://github.com/ceph/ceph/blob/main/qa/suites/fs/cephadm/renamevolume/1-rename.yaml#L5

Same failure, but for different job, was also seen in Milind's run as well - https://pulpito.ceph.com/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616441/.

Actions #1

Updated by Rishabh Dave about 1 month ago

  • Subject changed from qa: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log to qa: fs was offline but also unexpectedly degraded
Actions #2

Updated by Rishabh Dave about 1 month ago

  • Description updated (diff)
Actions #3

Updated by Rishabh Dave about 1 month ago

  • Description updated (diff)
Actions #4

Updated by Venky Shankar 25 days ago

  • Category set to Correctness/Safety
  • Assignee set to Kotresh Hiremath Ravishankar
  • Target version set to v20.0.0
Actions

Also available in: Atom PDF