Project

General

Profile

Actions

Bug #65794

open

Ceph Reef RGW error response fails to be parsed during awscli create-bucket

Added by Peter Razumovsky 30 days ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
s3 low-hanging-fruit
Backport:
reef squid
Regression:
Yes
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Using Ceph Reef v18.2.2 (with Rook v1.13.5 but it is not important here). Running tests with RGW bucket quota exceed and got the following message from aws cli:

# aws --endpoint-url https://rook-ceph-rgw-openstack-store.rook-ceph.svc:8443/ --ca-bundle /etc/rgwcerts/cacert s3api create-bucket --bucket quota

argument of type 'NoneType' is not iterable

 It seems this is due to RGW starts returning message tag with empty body since Ceph Reef:

2024-05-03 14:48:59,116 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>TooManyBuckets</Code><Message></Message><BucketName>quota</BucketName><RequestId>tx000003a27407b65589525-006634f95a-47cc-openstack-store</RequestId><HostId>47cc-openstack-store-openstack-store</HostId></Error>'

(see full debug log in error-message-reef.log)

When using Ceph Quincy v17.2.7 - nothing unexpected, I got TooManyBuckets error as always:

# aws --endpoint-url https://rook-ceph-rgw-openstack-store.rook-ceph.svc:8443/ --ca-bundle /etc/rgwcerts/cacert s3api create-bucket --bucket quota-bucket 

An error occurred (TooManyBuckets) when calling the CreateBucket operation: Unknow

And here is corresponding RGW response for over-quota bucket creation in Ceph Quincy:

2024-05-03 15:01:06,069 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>TooManyBuckets</Code><BucketName>quota-bucket</BucketName><RequestId>tx000004f549f743c0fcb7f-006634fc32-d524-openstack-store</RequestId><HostId>d524-openstack-store-openstack-store</HostId></Error>'

(full debug log could be found in error-message-quincy.log)

awscli and botocore versions are the same for Ceph Quincy and Ceph Reef tests so the version diff has no effect here.


Files

error-message-reef.log (21.1 KB) error-message-reef.log Peter Razumovsky, 05/03/2024 03:03 PM
error-message-quincy.log (20.2 KB) error-message-quincy.log Peter Razumovsky, 05/03/2024 03:03 PM
proving-example.py (1.46 KB) proving-example.py Peter Razumovsky, 05/07/2024 12:50 PM

Related issues 2 (2 open0 closed)

Related to rgw - Bug #59433: s3 error response missing Message fieldPending BackportCasey Bodley

Actions
Blocks rgw - Backport #59615: quincy: s3 error response missing Message fieldIn ProgressMykola GolubActions
Actions

Also available in: Atom PDF