Every kind of SaaS apps, mobile apps have a requirement these days where they want to store a lot of images, user-related files, or maybe some other static file. There are tons of ways to do this. Many developers prefer to use a third-party file system in the cloud to store such things (A.W.S.) but I personally believe, that these services are nothing but just "money crunching machines for small-medium apps"

So, what are the alternatives then?

Well, if you are planning to use MongoDB as a primary database for any of your projects, then you don't have to move to AWS for image storage (unless you have tons of users).

MongoDB has a driver specification to upload and retrieve files from it called GridFS. GridFS allows you to store and retrieve files, MongoDB says that you should only store those files in GridFS spec. that usually exceed the 16MB document limit size, for smaller files you can use the binary format and store it in the same document.

Okay enough talk, let's build a Node.js Image storage system with GridFS and Nodejs

Assumptions: You already have basic knowledge about Node.js, MongoDB, and RESTful API

Install dependencies

For the sake of this project, we will use mongoose which is a really popular MongoDB client library for Node.js. We will also use multer to handle file uploads in Node.js, multer-gridfs-storage will be used to build the GridFS engine for our Mongo Database.

I am not going to cover all the code from scratch, I will only cover the core part, which tells us how to initialize a GridFS engine and store our files there. If you need a complete API tutorial on this. Drop your request in the comments.

Fire up your terminal and execute these commands.

yarn add crypto multer method-override mongoose multer-gridfs-storage

Crypto: It is used to encrypt the filenames on being stored and read from the database.

Multer: We are using this as a middleware to handle static file uploads on our Node.JS server
multer-gridfs-storage: It is a wrapper around multer which helps us to initialize our GridFS storage engine
method-override: It is a middleware to enable the delete operation for files

Let's also write a quick model for our Image storage engine

const mongoose = require('mongoose');
const Schema = new mongoose.Schema;

const ImageSchema = new Schema({
    caption: {
        required: true,
        type: String,
    },
    filename: {
        required: true,
        type: String,
    },
    fileId: {
        required: true,
        type: String,
    },
    createdAt: {
        default: Date.now(),
        type: Date,
    },
});

const Image = mongoose.model('Image', ImageSchema);

module.exports = Image;

Initialize the storage engine

Once we initialize the storage engine, we just have to import and call it using the multer middleware. It is then passed to the respective route/controller method executing the various file storage operations.

//write this in your project's entry file (app.js)

const methodOverride = require('method-override');
const multer = require('multer');
const GridFsStorage = require('multer-gridfs-storage');
const config = require('./config'); // Contains env. and other configs


//Initialize the storage engine
const storage = new GridFsStorage({
    url: config.mongoURI,
    file: (req, file) => {
        return new Promise((resolve, reject) => {
            crypto.randomBytes(16, (err, buf) => {
                if (err) {
                    return reject(err);
                }
                const filename = buf.toString('hex') + path.extname(file.originalname);
                const fileInfo = {
                    filename: filename,
                    bucketName: 'imageUploads'
                };
                resolve(fileInfo);
            });
        });
    }
});

const upload = multer({ storage });

Initialize GridFS Stream

This stream will be used to read the files from the database and, we can also use the same stream to send an image directly to different browsers.

# Add this middleware in your routes file

const url = config.mongoURI;
    const connect = mongoose.createConnection(url, { useNewUrlParser: true, useUnifiedTopology: true });

    let gridFS;

    connect.once('open', () => {
        // initialize stream
        gridFS = new mongoose.mongo.GridFSBucket(connect.db, {
            bucketName: "imageUploads"
        });
    });

Finally, it's now time to upload a file.

Now we can reuse the middleware that we created earlier to upload and process our files to MongoDB

myImageRoute.route("/").post(upload.single("file"), (req, res, next) => {
      let newImage = new Image({
        caption: req.body.caption,
        filename: req.file.filename,
        fileId: req.file.id,
      });

      newImage
        .save()
        .then((image) => {
          res.status(200).json({
            message: 'File uploaded successfuly.'
            image,
          });
        })
        .catch((err) => res.status(500).json(err));
    })
    .catch((err) => res.status(500).json(err));
});

Delete a particular file using its ObjectId

myImageRoute.route("/file/del/:id").post((req, res, next) => {
  gridFS.delete(new mongoose.Types.ObjectId(req.params.id), (err, data) => {
    if (err) {
      return res.status(404).json({ err: err });
    }

    res.status(200).json({
      success: true,
      message: `File successfully deleted`,
    });
  });
});

So, that's it. We created a very basic image storage engine using Node.JS, MongoDB, and GridFS. You can learn more about GridFS and extend the functionalities.

If you liked this article, drop comments and tell me what should I write next.

I can be your pair programming bro if you need help in one of your projects.

Eccentric's blog