Uploading To S3 From NodeJs

When it comes to uploading directly to S3 with a signed put url, it can be a little frustrating. AWS will just keep giving you 403 errors without much direction. There are a few things to look out for when uploading from a NodeJS server specifically. A lot of the guides out there are centered around sending File objects from Form requests to S3 from the browser.

This request style (while largely the same) differs in some ways.

Here’s everything you need to know:

Sign The Url With The Correct Content-Type

The first thing to check for when you are getting 403’s with a put request to S3 is your Content-Type headers.

They need to match (character-for-character) the content-type being specified when creating the put url.

This means if you are trying to upload a PDF, you should have ‘application/pdf’. Like this:

const params = { Bucket: BUCKET, Key: fileNameWithExtension, Expires: expiresInSeconds, ContentType: "application/pdf" }
const signedUrl: string = await s3.getSignedUrl('putObject', params)

Or, if you are uploading a CSV, you should have ‘text/csv’. Like this:

const params = { Bucket: BUCKET, Key: fileNameWithExtension, Expires: expiresInSeconds, ContentType: "text/csv" }
const signedUrl: string = await s3.getSignedUrl('putObject', params)

Send The Upload As A Put, Content As A Buffer, And With The Correct Content-Type

Now, to send the data, you need to make sure you set the content-type explicitly again. You also need to make sure you are uploading a buffer.

With our csv example, that could look something like this. I am using got in the example:

await got.put(signedUrl, { body: Buffer.from("text,csv,123"), headers: { "Content-Type": "text/csv" }});

Keep in mind that functions like fs.readFileSync() and many http clients return files as a Buffer already. You will often not have to include Buffer.from() in your code if you are getting data from one of these methods.

Here’s an axios example in case you are using that as your http client:

await axios.put(signedUrl, Buffer.from("text,csv,123"), { headers: { "Content-Type": "text/csv" }});

Hopefully, you find this helpful! :)