Project

General

Profile

Actions

Bug #40905

closed

rgw: list bucket with delimiter wrongly skip some special keys

Added by Tianshan Qu almost 5 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

list with delimiter will skip subfile with directory + after_delim_s,
but the code wrongly add after_delim_s to next marker regardless it have directory.

some example: we have objects like

bucket/0/
bucket/1001
...
bucket/1999
bucket/1999#
bucket/1999+
bucket/2000

first time list get last key = bucket/1999, and the current code will add "/0xff" after 1999 to be next_marker, which cause the 1999# and 1999+ skipped.


Related issues 4 (0 open4 closed)

Related to rgw - Bug #47487: rgw: ordered bucket listing code clean-upResolvedJ. Eric Ivancich

Actions
Copied to rgw - Backport #41484: mimic: rgw: list bucket with delimiter wrongly skip some special keysResolvedNathan CutlerActions
Copied to rgw - Backport #41485: nautilus: rgw: list bucket with delimiter wrongly skip some special keysResolvedNathan CutlerActions
Copied to rgw - Backport #41486: luminous: rgw: list bucket with delimiter wrongly skip some special keysRejectedActions
Actions #2

Updated by Matt Benjamin almost 5 years ago

  • Status changed from New to Triaged
  • Backport set to nautilus,mimic,luminous

need test case(s)

Actions #3

Updated by Casey Bodley almost 5 years ago

  • Pull request ID set to 29215
Actions #4

Updated by Abhishek Lekshmanan almost 5 years ago

  • Status changed from Triaged to In Progress
Actions #5

Updated by Matt Benjamin almost 5 years ago

  • Status changed from In Progress to 17
Actions #6

Updated by Matt Benjamin almost 5 years ago

@Tianshan Qu,

I've been trying to reproduce on an unpatched branch, using RGW NFS. I do have https://github.com/ceph/ceph/pull/29670, which fixes a bug with NFS-generated markers in a pseudo-directory immediately below /bucket.

I did the following, which intends to match your test case--/folder/tianshan is a bucket:

1211  mount -tnfs lemon:/userx /folder -o sync
1212 cd /folder
1213 ls
1214 mkdir tianshan
1215 cd tianshan/
1216 ls
1217 touch f{1001,1002,1999,1999#,1999+,2000}

(unmount and restart of ganesha, then remount--to ensure that a listing is done)

[root@lemon /]# ls /folder/tianshan/
f1001 f1002 f1999 f1999# f1999+ f2000

I adjusted this slightly by creating files f1000..f1999 as well, so that f1999+ should be next_marker, but did not see it skipped.

I think we need a discrete test case.

Matt

Actions #7

Updated by Matt Benjamin almost 5 years ago

  • Status changed from 17 to Pending Backport
Actions #8

Updated by Tianshan Qu almost 5 years ago

@Matt Li Benjamin

need an extra directory before the f1999+ to trigger the skip logical, such as f0/, otherwise the skip_after_delim will not have value.

Actions #9

Updated by Nathan Cutler almost 5 years ago

  • Copied to Backport #41484: mimic: rgw: list bucket with delimiter wrongly skip some special keys added
Actions #10

Updated by Nathan Cutler almost 5 years ago

  • Copied to Backport #41485: nautilus: rgw: list bucket with delimiter wrongly skip some special keys added
Actions #11

Updated by Nathan Cutler almost 5 years ago

  • Copied to Backport #41486: luminous: rgw: list bucket with delimiter wrongly skip some special keys added
Actions #12

Updated by J. Eric Ivancich over 3 years ago

  • Status changed from Pending Backport to Resolved
Actions #13

Updated by J. Eric Ivancich over 3 years ago

  • Related to Bug #47487: rgw: ordered bucket listing code clean-up added
Actions

Also available in: Atom PDF