S3 bucket with NodeJs

In this blog, we will learn to upload, retrieve, and delete files on the AWS S3 server using the aws-sdk library. It provides limitless virtual storage of the files.
Buckets, objects, and folders in Amazon S3 can be managed by using the AWS Management Console.

First of all, we will create a bucket. A bucket can be said as a folder for your files. Choose a bucket name and region. As this is a simple setup, we will not make any changes in the next step, configure options.

In the next step, We can set permissions for the bucket to access the files. Generally, it’s recommended to mark all the options.

Now, let’s start with code:

Installing aws-sdk:

You need to install aws-sdk by the following command:

npm i aws-sdk --save

Import packages

Import these packages in your code:

const fs = require('fs');
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});

As you can notice, we have also imported fs package, which will be used to write file data in this app. It is better to put AWS access and secret access in the process environment variables, as it is a bad practice to put them on Github.

Now, you have the S3 instance object, and it can access the buckets in your AWS S3 account.

S3 is object-based storage. So, in the AWS console, you can see nested folders (similar to objects); behind the scene, they never get saved like that. Every object has two fields: Key and Value. Key here is simply the name of file, and Value is data which is getting stored.

To upload a file on S3 bucket:

upload function –

const fs = require('fs');
const AWS = require('aws-sdk');

const s3 = new AWS.S3({
    accessKeyId: process.env.AWS_ACCESS_KEY,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});

const fileName = 'contacts.csv';
fs.readFile(fileName, (err, fileData) => {
    if (err) throw err;
    const params = {
        Bucket: 'testBucket', // pass your bucket name
        Key: 'xyz.csv', // file will be saved as testBucket/contacts.csv
        Body: fileData
    };
    s3.upload(params, function(error, data) {
        if (error) console.log(error)
        console.log(`File uploaded successfully at ${data.Location}`)
    });
});

S3 upload method returns error and data in the callback as we see in normal function, where the data field contains the location, bucket, key, and other fields of the uploaded file.

To retrieve the file which is stored in S3 bucket-

getObject function –

var params = {
  Bucket: "testbucket", //bucket name
  Key: "xyz.csv"  /file key
 };
 s3.getObject(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);           // successful response
})

It will give you the file object.

To delete a file stored from S3 bucket-

deleteObject function –

var params = {
  Bucket: "testbucket", //bucket name
  Key: "xyz.csv"  /file key
 };
 s3.deleteObject(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);           // successful response
})

Just use the above function, send the bucket name and file key and your work is done!

To delete multiple objects from a bucket-

deleteObjects function –

 var params = {
  Bucket: "testebucket", 
  Delete: {
   Objects: [
      {
     Key: "objectkey1"
    }, 
      {
     Key: "objectkey2"
    }
   ], 
   Quiet: false
  }
 };
 s3.deleteObjects(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);    
})

There can be multiple file keys in the objects field. It will allow you to delete the multiple files at a time.

And that’s it. Now you have got the basic idea of S3 bucket.