binarybeachio: presigned PUT for uploads (R2/B2 don't implement PostObject)
== WHY (KEEP THIS — IT'S WHY THE FORK EXISTS) ==
Vanilla Plane's upload flow uses AWS S3 PostObject (presigned POST +
multipart/form-data + signed-policy-document). Cloudflare R2 AND
Backblaze B2 — the two most common self-host S3-compatible backends —
both return HTTP 501 NotImplemented for PostObject. Empirically verified
2026-04-30 against B2 s3.us-west-004.backblazeb2.com from inside Plane's
own prod api container, replicating Plane's exact boto3 call:
PUT against B2: 200 OK
POST against B2: 501 NotImplemented "This API call is not supported."
POST against R2: 501 NotImplemented (failure that started this thread)
The error code is `NotImplemented` (not `SignatureDoesNotMatch` etc),
meaning the server rejects the verb itself — no boto3 config, addressing-
style flag, or signature variant fixes it. Tested both path-style and
virtual-hosted-style URLs against B2; both fail identically for POST.
This patch rewrites the upload flow to use presigned PUT, which is
universally supported (R2, B2, AWS S3 native, MinIO, Wasabi, etc).
== WHAT (FIVE-FILE BACKEND, FIVE-FILE FRONTEND) ==
Backend:
* apps/api/plane/settings/storage.py — S3Storage.generate_presigned_post
now mints a presigned PUT URL via generate_presigned_url(HttpMethod="PUT").
Method name kept for caller compat. Response shape:
{url, method: "PUT", fields: {Content-Type, key}}.
* apps/api/plane/utils/openapi/responses.py — example response updated.
* apps/api/plane/tests/unit/settings/test_storage.py — 2 tests updated to
assert the new boto3 call.
Frontend:
* packages/types/src/file.ts — TFileSignedURLResponse.upload_data adds
optional method?: "PUT" | "POST"; drops AWS POST-form-data fields.
* packages/services/src/file/helper.ts — generateFileUploadPayload now
returns a TFileUploadRequest descriptor (url+method+body+headers) that
dispatches on method. POST branch kept for upstream parity but the
fork backend never emits POST.
* packages/services/src/file/file-upload.service.ts +
apps/web/core/services/file-upload.service.ts — uploadFile signature
changes from (url, FormData, progress?) to (payload, progress?).
* 5 caller sites updated (apps/web/core/services/file.service.ts x3,
issue_attachment.service.ts x1, sites-file.service.ts x1).
== TRADEOFFS ACCEPTED ==
* Lost: signed `content-length-range` enforcement at the storage layer.
Server-side validation in the API view still rejects oversized requests
with 413 before minting the URL, so a determined client could only
over-upload by misreporting size, capped at the bucket's own size limit.
* Different request shape on the wire (PUT with raw binary body vs POST
with multipart form). Externally invisible to users.
== ROLLBACK ==
If this becomes a maintenance nightmare:
git revert <this-commit-sha>
# rebuild + push images, swap compose tags, redeploy
After revert, uploads will only work against backends that implement
PostObject (MinIO, AWS S3 native). R2 and B2 will return 501 again.
== FULL DECISION RECORD ==
binarybeachio repo: docs/features/storage-upload-flow.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
7c21b985d9
commit
9fb1ad44cd
10 changed files with 131 additions and 89 deletions
|
|
@ -63,40 +63,50 @@ class S3Storage(S3Boto3Storage):
|
|||
)
|
||||
|
||||
def generate_presigned_post(self, object_name, file_type, file_size, expiration=None):
|
||||
"""Generate a presigned URL to upload an S3 object"""
|
||||
"""Generate a presigned URL for browser-direct upload.
|
||||
|
||||
BB-PATCH (binarybeachio fork): method name preserved for caller
|
||||
compat, but this now mints a presigned PUT URL — not POST.
|
||||
|
||||
Why: Cloudflare R2 and Backblaze B2 — the two most common self-host
|
||||
S3-compatible backends — do NOT implement S3 PostObject. Both return
|
||||
HTTP 501 NotImplemented for the bucket-form POST verb that vanilla
|
||||
Plane uses. Confirmed empirically against both backends 2026-04-30.
|
||||
Rolling our own backend support isn't tractable; PUT is universally
|
||||
supported (R2, B2, AWS S3, MinIO, Wasabi, etc).
|
||||
|
||||
Tradeoff: we lose signed enforcement of `content-length-range`. Size
|
||||
is still validated server-side at presign time via the `file_size`
|
||||
param (see callers: 413 raised before we get here), so a determined
|
||||
client could only over-upload by misreporting the size — they'd be
|
||||
capped by the bucket's max-file-size at worst.
|
||||
|
||||
See docs/features/storage-upload-flow.md in the binarybeachio repo
|
||||
for the full decision record + rollback procedure (`git revert` this
|
||||
commit and rebuild the images).
|
||||
"""
|
||||
if expiration is None:
|
||||
expiration = self.signed_url_expiration
|
||||
fields = {"Content-Type": file_type}
|
||||
|
||||
conditions = [
|
||||
{"bucket": self.aws_storage_bucket_name},
|
||||
["content-length-range", 1, file_size],
|
||||
{"Content-Type": file_type},
|
||||
]
|
||||
|
||||
# Add condition for the object name (key)
|
||||
if object_name.startswith("${filename}"):
|
||||
conditions.append(["starts-with", "$key", object_name[: -len("${filename}")]])
|
||||
else:
|
||||
fields["key"] = object_name
|
||||
conditions.append({"key": object_name})
|
||||
|
||||
# Generate the presigned POST URL
|
||||
try:
|
||||
# Generate a presigned URL for the S3 object
|
||||
response = self.s3_client.generate_presigned_post(
|
||||
Bucket=self.aws_storage_bucket_name,
|
||||
Key=object_name,
|
||||
Fields=fields,
|
||||
Conditions=conditions,
|
||||
url = self.s3_client.generate_presigned_url(
|
||||
"put_object",
|
||||
Params={
|
||||
"Bucket": self.aws_storage_bucket_name,
|
||||
"Key": object_name,
|
||||
"ContentType": file_type,
|
||||
},
|
||||
ExpiresIn=expiration,
|
||||
HttpMethod="PUT",
|
||||
)
|
||||
# Handle errors
|
||||
except ClientError as e:
|
||||
print(f"Error generating presigned POST URL: {e}")
|
||||
print(f"Error generating presigned PUT URL: {e}")
|
||||
return None
|
||||
|
||||
return response
|
||||
return {
|
||||
"url": url,
|
||||
"method": "PUT",
|
||||
"fields": {"Content-Type": file_type, "key": object_name},
|
||||
}
|
||||
|
||||
def _get_content_disposition(self, disposition, filename=None):
|
||||
"""Helper method to generate Content-Disposition header value"""
|
||||
|
|
|
|||
|
|
@ -63,13 +63,15 @@ class TestS3StorageSignedURLExpiration:
|
|||
)
|
||||
@patch("plane.settings.storage.boto3")
|
||||
def test_generate_presigned_post_uses_default_expiration(self, mock_boto3):
|
||||
"""Test that generate_presigned_post uses the configured default expiration"""
|
||||
"""Test that generate_presigned_post uses the configured default expiration
|
||||
|
||||
BB-PATCH: generate_presigned_post now mints a presigned PUT URL under
|
||||
the hood (R2/B2 don't implement PostObject). Test asserts the
|
||||
underlying generate_presigned_url call rather than generate_presigned_post.
|
||||
"""
|
||||
# Mock the boto3 client and its response
|
||||
mock_s3_client = Mock()
|
||||
mock_s3_client.generate_presigned_post.return_value = {
|
||||
"url": "https://test-url.com",
|
||||
"fields": {},
|
||||
}
|
||||
mock_s3_client.generate_presigned_url.return_value = "https://test-url.com"
|
||||
mock_boto3.client.return_value = mock_s3_client
|
||||
|
||||
# Create S3Storage instance
|
||||
|
|
@ -79,9 +81,10 @@ class TestS3StorageSignedURLExpiration:
|
|||
storage.generate_presigned_post("test-object", "image/png", 1024)
|
||||
|
||||
# Assert that the boto3 method was called with the default expiration (3600)
|
||||
mock_s3_client.generate_presigned_post.assert_called_once()
|
||||
call_kwargs = mock_s3_client.generate_presigned_post.call_args[1]
|
||||
mock_s3_client.generate_presigned_url.assert_called_once()
|
||||
call_kwargs = mock_s3_client.generate_presigned_url.call_args[1]
|
||||
assert call_kwargs["ExpiresIn"] == 3600
|
||||
assert call_kwargs["HttpMethod"] == "PUT"
|
||||
|
||||
@patch.dict(
|
||||
os.environ,
|
||||
|
|
@ -96,13 +99,14 @@ class TestS3StorageSignedURLExpiration:
|
|||
)
|
||||
@patch("plane.settings.storage.boto3")
|
||||
def test_generate_presigned_post_uses_custom_expiration(self, mock_boto3):
|
||||
"""Test that generate_presigned_post uses custom expiration from env variable"""
|
||||
"""Test that generate_presigned_post uses custom expiration from env variable
|
||||
|
||||
BB-PATCH: see test_generate_presigned_post_uses_default_expiration for
|
||||
why this asserts generate_presigned_url instead of generate_presigned_post.
|
||||
"""
|
||||
# Mock the boto3 client and its response
|
||||
mock_s3_client = Mock()
|
||||
mock_s3_client.generate_presigned_post.return_value = {
|
||||
"url": "https://test-url.com",
|
||||
"fields": {},
|
||||
}
|
||||
mock_s3_client.generate_presigned_url.return_value = "https://test-url.com"
|
||||
mock_boto3.client.return_value = mock_s3_client
|
||||
|
||||
# Create S3Storage instance with SIGNED_URL_EXPIRATION=60
|
||||
|
|
@ -112,9 +116,10 @@ class TestS3StorageSignedURLExpiration:
|
|||
storage.generate_presigned_post("test-object", "image/png", 1024)
|
||||
|
||||
# Assert that the boto3 method was called with custom expiration (60)
|
||||
mock_s3_client.generate_presigned_post.assert_called_once()
|
||||
call_kwargs = mock_s3_client.generate_presigned_post.call_args[1]
|
||||
mock_s3_client.generate_presigned_url.assert_called_once()
|
||||
call_kwargs = mock_s3_client.generate_presigned_url.call_args[1]
|
||||
assert call_kwargs["ExpiresIn"] == 60
|
||||
assert call_kwargs["HttpMethod"] == "PUT"
|
||||
|
||||
@patch.dict(
|
||||
os.environ,
|
||||
|
|
|
|||
|
|
@ -415,12 +415,11 @@ GENERIC_ASSET_UPLOAD_SUCCESS_RESPONSE = OpenApiResponse(
|
|||
name="Generic Asset Upload Response",
|
||||
value={
|
||||
"upload_data": {
|
||||
"url": "https://s3.amazonaws.com/bucket-name",
|
||||
"url": "https://s3.amazonaws.com/bucket-name/workspace-id/uuid-filename.pdf?X-Amz-Signature=...",
|
||||
"method": "PUT",
|
||||
"fields": {
|
||||
"Content-Type": "application/pdf",
|
||||
"key": "workspace-id/uuid-filename.pdf",
|
||||
"AWSAccessKeyId": "AKIA...",
|
||||
"policy": "eyJ...",
|
||||
"signature": "abc123...",
|
||||
},
|
||||
},
|
||||
"asset_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue