👋 Introduction

From past three years, I have been using Amazon S3, Backblaze B2 and Cloudflare R2 to store my files and data. I thought, everything is smoothly working based on my understanding. But one day, while listening AWS CLI, I realized: I didn’t really get it at all.

This blog is about how my confusion with folders and objects turned into one of my most valuable lessons about object storage.

🌐 Object Storage at a Glance

Amazon, Backblaze and Cloudflare are providing the object storage services. Everything is an object. All objects are stored into the bucket. All objects are identified by their key. For example: Image is an object. Key will be object name sb-logo.png or pictures/sb-logo.png There is no folder concept. Just objects with keys that may look like folder path.

🔍 My Initial Confusion

I usually using tools like Filezilla, S3 Browser to manipulate my objects. They are one more level intelligent on how to display the objects in their UI. If my object is pictures/sb-logo.png, it will show the sb-logo.png object under the pictures directory. But internally it is only one object. I accepted this until I listed my objects by using AWS S3 CLI. That’s when my confusion started.

🧪 The Experiment That Changed My Mind

The followings are my actual objects which I uploaded into S3:

            
Kandhasamy/Images/website/author-logo.png
Kandhasamy/Images/website/logo.png
Kandhasamy/website/required.zip
            
          

But AWS S3 CLI listed the followings:

          
Kandhasamy/
Kandhasamy/Images/
Kandhasamy/Images//website/
Kandhasamy/Images/website/author-logo.png
Kandhasamy/Images/website/logo.png
Kandhasamy/website/required.zip
            
          

I shocked here why folders are listing as objects 😲. I decided to identify the issue and tried the followings:

I created a folder called foo in Filezilla and uploaded an image logo.png. I listed the object through AWS S3 CLI.

            
foo/
foo/logo.png
            
          

When creating a folder in Filezilla, the client tool internally uploading this fake folder as an object.

To confirm this, I uploaded the image file directly without creating a folder in Filezilla. Then I listed the objects

            
Kandhasamy/Images/mine.png
            
          

CLI showed exactly that object. There is no fake folders. Then the following points are verified carefully:

• S3 don’t care about folders.
• Some intelligent client tools are creating a folder as an invalid objects.
• Tool like MinIO is smart. To avoid this confusion, they don’t let us to create a folder without uploading a file.

✅ Takeaway for Developers

If you are working on S3, B2 or R2, I would like to follow the followings:

• Learn core concepts first
• Don’t rely only on client tool like Filezilla or S3 Browser - they are convenient but can cause misconceptions.
• Always verify using the AWS CLI or SDK’s

At the end of the day, everything is just an object with a key. That’s the beauty and simplicity of object storage.

Happy Coding!! ☕.