Copy a file from a URL directly to S3 using boto3 and requests

leave a comment »

Copy a file at inUrl directly to a s3 bucket bucketName. ACL is set to public-read and ContentType is maintained from the from URL.

Your AWS credentials must be pre-set.

import boto3
import requests

#Set up boto3 env vars
os.environ['AWS_ACCESS_KEY_ID'] = ''
os.environ['AWS_SECRET_ACCESS_KEY'] = ''
os.environ['AWS_DEFAULT_REGION'] = 'us-west-2'

inUrl = r'file_to_read'
response = requests.get(inUrl)

s3 = boto3.resource('s3')
s3.Bucket(bucketName).put_object(Key='test/file_path.ext', Body=response.content, ACL='public-read',ContentType=response.headers['Content-Type'])

Written by M Kapoor

February 7, 2017 at 2:09 pm

Posted in python

Tagged with ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: