--- title: "AWS S3 - DreamFactory Documentation" source: "http://wiki.dreamfactory.com/Aws_S3" canonical_url: "http://wiki.dreamfactory.com/Aws_S3" converted_at: "2026-10-22T18:50:42.960Z" format: "markdown" converted_by: "html-to-md-ai" --- []() # Aws S3 From DreamFactory Wiki [Jump to navigation](#mw-head) [Jump to search](#searchInput) **Create a REST API for AWS S3 buckets to manage files with role-based access and rate limiting** ## Contents - [1 AWS S3](#AWS_S3) - [1.1 Use Cases](#Use_Cases) - [1.2 Prerequisites](#Prerequisites) - [1.2.1 Minimum IAM Policy](#Minimum_IAM_Policy) - [1.3 Creating an AWS S3 Service](#Creating_an_AWS_S3_Service) - [1.3.1 Step 1: Navigate to API Generation](#Step_1:_Navigate_to_API_Generation) - [1.3.2 Step 2: Create New Service](#Step_2:_Create_New_Service) - [1.3.3 Step 3: Configure Service Details](#Step_3:_Configure_Service_Details) - [1.3.4 Step 4: Configure AWS Credentials](#Step_4:_Configure_AWS_Credentials) - [1.3.5 Step 5: Save and Test](#Step_5:_Save_and_Test) - [1.4 Configuration Options](#Configuration_Options) - [1.4.1 Required Settings](#Required_Settings) - [1.4.2 Optional Settings](#Optional_Settings) - [1.4.3 S3-Compatible Storage](#S3-Compatible_Storage) - [1.5 API Endpoints](#API_Endpoints) - [1.6 API Examples](#API_Examples) - [1.6.1 List Bucket Contents](#List_Bucket_Contents) - [1.6.2 Upload a File](#Upload_a_File) - [1.6.3 Download a File](#Download_a_File) - [1.6.4 Create a Folder](#Create_a_Folder) - [1.6.5 Delete a File](#Delete_a_File) - [1.6.6 Delete a Folder](#Delete_a_Folder) - [1.6.7 Get File Metadata](#Get_File_Metadata) - [1.7 Pre-Signed URLs](#Pre-Signed_URLs) - [1.8 File Upload Limits](#File_Upload_Limits) - [1.8.1 Server Configuration](#Server_Configuration) - [1.8.2 S3 Limits](#S3_Limits) - [1.9 Common Errors](#Common_Errors) - [1.9.1 Troubleshooting AWS Errors](#Troubleshooting_AWS_Errors) - [1.10 Security Best Practices](#Security_Best_Practices) - [1.10.1 IAM Configuration](#IAM_Configuration) - [1.10.2 Bucket Configuration](#Bucket_Configuration) - [1.10.3 DreamFactory RBAC](#DreamFactory_RBAC) - [1.11 Cost Optimization](#Cost_Optimization) - [1.11.1 Request Pricing](#Request_Pricing) - [1.11.2 Data Transfer](#Data_Transfer) - [1.12 Next Steps](#Next_Steps) # AWS S3 DreamFactory's AWS S3 connector provides a REST-based interface for interacting with S3 objects and buckets. Supporting all standard CRUD operations, you can easily manage your S3 data through a unified API. Because the S3 API is native to DreamFactory, you can integrate S3 actions alongside other API-driven tasks: - Upload a newly registered user avatar to S3 while inserting registration data into a database - Email a website visitor a link to a product PDF after writing their email to your CRM - Create new S3 buckets as part of a DevOps workflow - Apply DreamFactory's role-based access controls, rate limiting, and audit logging --- ## Use Cases - **Application file storage**: User uploads, media assets, documents - **Static asset hosting**: Images, CSS, JavaScript for web applications - **Data lake integration**: Store and retrieve data files for analytics - **Backup storage**: Archive files from other systems - **Multi-tenant file isolation**: Per-customer folders with RBAC --- ## Prerequisites Before configuring the S3 connector, you need: 1. **AWS Account** with S3 access 2. **S3 Bucket** created in your desired region 3. **IAM User or Role** with S3 permissions 4. **Access Key ID and Secret Access Key** for the IAM user ### Minimum IAM Policy Create an IAM policy with these permissions: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::your-bucket-name", "arn:aws:s3:::your-bucket-name/*" ] } ] } ``` --- ## Creating an AWS S3 Service ### Step 1: Navigate to API Generation Log in to your DreamFactory instance using an administrator account and select the **API Generation & Connections** tab. Set your API Type to **File**. ### Step 2: Create New Service Click the purple plus button to create a new file service, then search for and select **AWS S3**. ### Step 3: Configure Service Details Field Description Example Name Service name (lowercase, alphanumeric, used in API URL) `s3files` Label Display name in admin console `AWS S3 Storage` Description Service description `Production file storage on S3` ### Step 4: Configure AWS Credentials Scroll to the **Config** section and enter your AWS credentials: Field Required Description Access Key ID Yes AWS IAM access key Secret Access Key Yes AWS IAM secret key Region Yes AWS region (e.g., `us-east-1`, `eu-west-1`) Bucket Yes S3 bucket name Container No Subdirectory within bucket to use as root ### Step 5: Save and Test Click **Save** to create the service. Navigate to **API Docs** to view the generated endpoints and test operations. --- ## Configuration Options ### Required Settings Field Type Description `key` string AWS Access Key ID `secret` string AWS Secret Access Key `region` string AWS region code `bucket` string S3 bucket name ### Optional Settings Field Type Default Description `container` string - Subdirectory prefix for all operations `endpoint` string - Custom S3-compatible endpoint URL `use_path_style_endpoint` boolean false Use path-style URLs (required for MinIO, LocalStack) `cache_enabled` boolean false Enable response caching `cache_ttl` integer 0 Cache time-to-live in seconds ### S3-Compatible Storage DreamFactory's S3 connector works with S3-compatible services by setting a custom endpoint: Service Endpoint Example MinIO `[http://minio.example.com:9000](http://minio.example.com:9000)` DigitalOcean Spaces `[https://nyc3.digitaloceanspaces.com](https://nyc3.digitaloceanspaces.com)` Backblaze B2 `[https://s3.us-west-002.backblazeb2.com](https://s3.us-west-002.backblazeb2.com)` Wasabi `[https://s3.wasabisys.com](https://s3.wasabisys.com)` Set `use_path_style_endpoint` to `true` for MinIO and similar services. --- ## API Endpoints Method Endpoint Description `GET` `/api/v2/{service_name}/` List bucket root `GET` `/api/v2/{service_name}/{path}/` List folder contents `GET` `/api/v2/{service_name}/{path}` Download file `POST` `/api/v2/{service_name}/` Create folder or upload file `POST` `/api/v2/{service_name}/{path}/` Create subfolder or upload file `PUT` `/api/v2/{service_name}/{path}` Replace file contents `DELETE` `/api/v2/{service_name}/{path}` Delete file or folder --- ## API Examples ### List Bucket Contents ``` curl -X GET "https://example.com/api/v2/s3files/" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" ``` **Response:** ``` { "resource": [ { "path": "images/", "type": "folder", "name": "images", "last_modified": "2026-02-10T14:30:00Z" }, { "path": "document.pdf", "type": "file", "name": "document.pdf", "content_type": "application/pdf", "content_length": 102400, "last_modified": "2026-02-09T10:15:00Z" } ] } ``` ### Upload a File ``` curl -X POST "https://example.com/api/v2/s3files/uploads/report.pdf" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" \ -H "Content-Type: application/pdf" \ --data-binary @report.pdf ``` **Response:** ``` { "name": "report.pdf", "path": "uploads/report.pdf", "content_type": "application/pdf", "content_length": 102400 } ``` ### Download a File ``` curl -X GET "https://example.com/api/v2/s3files/uploads/report.pdf" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" \ -o report.pdf ``` ### Create a Folder S3 doesn't have true folders, but DreamFactory creates a placeholder object: ``` curl -X POST "https://example.com/api/v2/s3files/" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" \ -H "X-Folder-Name: documents" ``` ### Delete a File ``` curl -X DELETE "https://example.com/api/v2/s3files/uploads/old-report.pdf" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" ``` ### Delete a Folder Delete all objects with a prefix: ``` curl -X DELETE "https://example.com/api/v2/s3files/old-folder/?force=true" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" ``` ### Get File Metadata ``` curl -X GET "https://example.com/api/v2/s3files/uploads/report.pdf?include_properties=true" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" ``` **Response:** ``` { "path": "uploads/report.pdf", "name": "report.pdf", "type": "file", "content_type": "application/pdf", "content_length": 102400, "last_modified": "2026-02-10T15:45:00Z" } ``` --- ## Pre-Signed URLs For large file uploads or direct client-to-S3 access, use pre-signed URLs: ``` curl -X GET "https://example.com/api/v2/s3files/uploads/large-file.zip?url=true&expires=3600" \ -H "X-DreamFactory-API-Key: YOUR_API_KEY" ``` **Response:** ``` { "url": "https://bucket.s3.amazonaws.com/uploads/large-file.zip?X-Amz-Algorithm=..." } ``` The `expires` parameter sets the URL validity in seconds (default: 3600). --- ## File Upload Limits Upload size limits are controlled by your web server, PHP configuration, and S3: ### Server Configuration See [Local File Storage - File Upload Limits](/Local_File_Storage#File_Upload_Limits) for Nginx and PHP settings. ### S3 Limits Limit Value Maximum object size 5 TB Maximum single PUT 5 GB Multipart upload threshold 100 MB (recommended) For files larger than 100 MB, consider using pre-signed URLs for direct upload. --- ## Common Errors Error Code Message Cause Solution 400 Bad Request Invalid parameters Check path and request format 401 Unauthorized Invalid API key Verify DreamFactory API key 403 Access Denied AWS permissions issue Check IAM policy and bucket policy 404 Not Found Object does not exist Verify the S3 key exists 409 Conflict Object already exists Use PUT to overwrite 413 Payload Too Large File exceeds limits Increase server limits or use pre-signed URL 503 Service Unavailable S3 unreachable Check network connectivity and AWS status ### Troubleshooting AWS Errors **Access Denied (403)**: 1. Verify IAM user has correct permissions 2. Check bucket policy allows access 3. Confirm region matches bucket location 4. Verify Access Key ID and Secret are correct **Bucket Not Found**: 1. Confirm bucket name spelling 2. Verify bucket exists in specified region 3. Check bucket hasn't been deleted --- ## Security Best Practices ### IAM Configuration 1. **Use dedicated IAM user** for DreamFactory 2. **Minimal permissions** - only grant required actions 3. **Restrict to specific bucket** - don't use wildcard resources 4. **Rotate credentials** regularly ### Bucket Configuration 1. **Block public access** unless specifically required 2. **Enable versioning** for critical data 3. **Enable server-side encryption** (SSE-S3 or SSE-KMS) 4. **Enable access logging** for audit trails ### DreamFactory RBAC Layer DreamFactory's RBAC on top of AWS permissions: Scenario Configuration Read-only access Allow GET only Upload-only Allow POST to specific paths User isolation Use filters to restrict by user ID folder --- ## Cost Optimization ### Request Pricing S3 charges per request. Optimize by: - Enabling caching in DreamFactory - Batching operations where possible - Using LIST sparingly (costs more than GET) ### Data Transfer - **Within region**: Free between S3 and EC2 - **Internet egress**: Charged per GB - **CloudFront**: Can reduce costs for high-traffic reads --- ## Next Steps - **[File Services Overview](/File_Services_Overview)**: Compare all file storage options - **[Local File Storage](/Local_File_Storage)**: Store files on the server - **[Role-Based Access Control](/Security/Role_Based_Access)**: Configure fine-grained permissions Retrieved from "[https://wiki.dreamfactory.com/index.php?title=Aws_S3&oldid=973](https://wiki.dreamfactory.com/index.php?title=Aws_S3&oldid=973)" [Categories](/Special:Categories): - [Aws S3](/Category:Aws_S3) - [File Storage](/Category:File_Storage) - [Rest Api](/Category:Rest_Api) - [API](/Category:API) - [Difficulty Intermediate](/Category:Difficulty_Intermediate)