Seamless Media Storage: Integrating Azure Blob Storage and MongoDB with Spring Boot
Rate this tutorial
From social media to streaming services, many applications require a mixture of different types of data. If you are designing an application that requires storing images or videos, a good idea is to store your media using a service specially designed to handle large objects of unstructured data.
Your MongoDB database is not the best place to store your media files directly. The maximum BSON document size is 16MB. This helps ensure that a single document cannot use an excessive amount of RAM or, during transmission, an excessive amount of bandwidth. This provides an obstacle as this limit can easily be surpassed by images or videos.
MongoDB provides GridFS as a solution to this problem. MongoDB GridFS is a specification for storing and retrieving large files that exceed the BSON-document size limit and works by dividing the file into chunks and storing each chunk as a separate document. In a second collection, it stores the metadata for these files, including what chunks each file is composed of. While this may work for some use cases, oftentimes, it is a good idea to use a service dedicated to storing large media files and linking to that in your MongoDB document. Azure Blob (Binary Large Objects) Storage is optimized for storing massive amounts of unstructured data and designed for use cases such as serving images directly to a browser, streaming video and audio, etc. Unstructured data is data that doesn't conform to a specific data model or format, such as binary data (how we store our media files).
In this tutorial, we are going to build a Java API with Spring Boot that allows you to upload your files, along with any metadata you wish to store. When you upload your file, such as an image or video, it will upload to Azure Blob Storage. It will store the metadata, along with a link to where the file is stored, in your MongoDB database. This way, you get all the benefits of MongoDB databases while taking advantage of how Azure Blob Storage deals with these large files.
Whether you're an experienced developer or just starting out, this guide is designed to equip you with the knowledge to build a media management back end that's not only scalable but also optimized for performance.
Let's dive into the prerequisites you'll need to follow this tutorial and then, step by step, we will build a system that is resilient, efficient, and ready to tackle the demands of a modern media-driven application.
- Java 11 or higher
- Maven or Gradle, but this tutorial will reference Maven
- A MongoDB cluster deployed and configured; if you need help, check out our MongoDB Atlas tutorial on how to get started
There are a couple of different ways you can set up your Azure storage, but we will use the Microsoft Azure Portal. Sign in with your Azure account and it will take you to the home page. At the top of the page, search "Storage accounts."
Select “Storage Accounts.” “Storage accounts (classic)” does not have all the features necessary for this tutorial.
This will take you to the page where you will create a storage account (or use an existing one). We are going to configure this storage account to be open and accessible to anyone who knows the URL. This is fine for this demo, but for production, you should configure this to only allow access from specific people/networks. Learn more about how to do this in the Microsoft Azure documentation.
Select the subscription and resource group you wish to use, and give your storage account a name. The region, performance, and redundancy settings are depending on your plans with this application, but the lowest tiers have all the features we need.
In networking, select to enable public access from all networks. This might not be desirable for production but for following along with this tutorial, it allows us to bypass configuring rules for network access.
For everything else, we can accept the default settings. Once your storage account is created, we’re going to navigate to the resource. You can do this by clicking “Go to resource,” or return to the home page and it will be listed under your resources.
The next step is to set up a container. A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs. On the left blade, select the containers tab, and click the plus container option. A menu will come up where you name your container (and configure access level if you don't want the default, private). Now, let's launch our container!
In order to connect your application to Azure Storage, create your
Shared Access Signature (SAS). SAS allows you to have granular control over how your client can access the data. Select “Shared access signature” from the left blade menu and configure it to allow the services and resource types you wish to allow. For this tutorial, select “Object” under the allowed resource types. Object is for blob-level APIs, allowing operations on individual blobs, like uploading, downloading, or deleting an image. The rest of the settings you can leave as the default configuration. If you would like to learn more about what configurations are best suited for your application, check out Microsoft’s documentation. Once you have configured it to your desired settings, click “Generate SAS and connection string.” Your SAS will be generated below this button.
Now that we have our Azure Storage account and MongoDB cluster set up, let's set up our API. What we're going to need is a way to upload files to our Blob Storage account and track the metadata for this file in a collection in our MongoDB database.
To set up our project, let's use the Spring Initializr. This will generate our pom.xml file, which will contain our dependencies for our project.
For this project, you want to select the options in the screenshot below and create a JAR:
- Project: Maven
- Language: Java
- Dependencies: Spring Web, Spring Data MongoDB, and Azure Storage
- Generate: JAR
Open up the Maven project in the IDE of your choice and let's write some code!
We are going to need some strings and keys to connect to Azure Blob Storage and our MongoDB database. To get your MongoDB connection string, log into MongoDB Atlas and click “Connect.” If you need help, check out our guide in the docs.
Add this to your
application.properties file. Here we will also define the name of the database and the port we wish to run our application on. Don’t worry if the database or collection we use does not exist yet. The moment we send our data to it, MongoDB will create it.
Next, we need to add our Azure Blob Storage information. We are going to add the shared access signature and connection string we generated earlier and the name for our container.
Now that we have everything we need to get connected to our blob and database, we need to create an endpoint to upload an image stored locally and track the image metadata. First, we're going to create a model for our image metadata. Let's create a package in our project called
com.example.azureblob.model and add our class ImageMetadata.java.Tip: Most popular IDEs will have shortcuts to generate getters and setters.
At the start of our document, we use the annotation
@Document to let Spring Boot know what collection in our MongoDB database this data will be added to. MongoDB will create this collection if it doesn't already exist. The @Id annotation indicates what will be used as the _id. If you don't assign an _id, MongoDB will automatically generate one for each new document.We added a couple of fields that describe our image being uploaded, including the URL it will be stored at. This URL will be generated when we insert our image into our blob storage.
The next step is to use this model and interact with our data in the database. In Spring Boot, we use a repository interface. This has all the CRUD (create, read, update, delete) operations we will need. Let's create a package called
com.example.azureblob.repository and add our interface ImageMetadataRepository.java.Our repository will extend MongoRepository. With MongoRepository, you don’t need to provide implementation code for many basic CRUD methods for your MongoDB database, such as save, findById, findAll, delete, etc. Spring Data MongoDB automatically generates the necessary implementation based on the method names and conventions.
Now that we have the repository set up, it's time to set up our service layer. This acts as the intermediate between our repository (data access layer) and our controller (REST endpoints) and contains the applications business logic. We'll create another package
com.example.azureblob.service and add our class ImageMetadataService.java.Here we have a couple of our methods set up for finding our documents in the database and saving our metadata. Our
uploadImageWithCaption method contains the integration with Azure Blob Storage. Here you can see we create a BlobServiceClient to interact with Azure Blob Storage. After it succeeds in uploading the image, it gets the URL of the uploaded blob. It then stores this, along with our other metadata for the image, in our MongoDB database.Our last step is to set up a controller to establish our endpoints for the application. In a Spring Boot application, controllers handle requests, process data, and produce responses, making it possible to expose APIs and build web applications. Create a package
com.example.azureblob.service and add the class ImageMetadataController.java.Here we're able to retrieve all our metadata documents or search by
_id, and we are able to upload our documents.This should be everything you need to upload your files and store the metadata in MongoDB. Let's test it out! You can use your favorite tool for testing APIs but I'll be using a cURL command.
Now, let's check how that looks in our database and Azure storage. If we look in our collection in MongoDB, we can see our metadata, including the URL to the image. Here we just have a few fields, but depending on your application, you might want to store information like when this document was created, the filetype of the data being stored, or even the size of the file.
If we want to see what this looks like in our blob, we can go back to our Azure portal and check our container. And here is our image!
As demonstrated in this tutorial, integrating Azure Blob Storage with a Java API powered by Spring Boot provides an optimal solution for storing large media files such as images and videos. Azure Blob Storage, Microsoft's cloud-based object storage solution, adeptly manages vast amounts of unstructured data. When coupled with MongoDB's efficient database handling, users benefit from a powerful tool that facilitates the uploading of files and storage of pertinent metadata.
Through this tutorial, we've walked you through setting up a MongoDB Atlas cluster and configuring Azure Storage. We demonstrated how to construct a Java API to seamlessly interact with both platforms. Furthermore, we detailed the essential steps, from the groundwork in the Spring Initializr to finalizing endpoints in our controller.
Whether you're aiming to build a sophisticated media platform, a content management system, or any application involving the storage of large media files, this tutorial gives you the steps to get started. By harnessing the combined prowess of Azure Blob Storage and MongoDB with a Spring Boot Java API, developers can build scalable, efficient, and resilient applications for the modern web.
Want to learn more about what you can do with Microsoft Azure and MongoDB? Check out some of our articles on Developer Center, such as How to Use Azure Functions with MongoDB Atlas in Java.