Скачать или смотреть How to Efficiently Retrieve Sub Folder Names from an S3 Bucket Using Boto3

Learn how to get only the subfolder names from an S3 bucket with Python's `Boto3` library, including a step-by-step guide and sample code.
---
This video is based on the question https://stackoverflow.com/q/67613719/ asked by the user 'ahkam' ( https://stackoverflow.com/u/12198876/ ) and on the answer https://stackoverflow.com/a/67614224/ provided by the user 'John Rotenstein' ( https://stackoverflow.com/u/174777/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to get only the sub folder names from a S3 bucket

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Efficiently Retrieve Sub Folder Names from an S3 Bucket Using Boto3

When working with Amazon S3 (Simple Storage Service) for storing and managing data, you might find yourself needing to access only the subfolder names within a particular bucket. This task can become quite complex when you have multiple layers of folders and need to filter based on specific criteria. In this guide, we’ll explore how to extract just the subfolder names from an S3 bucket using Python's Boto3 library.

Understanding the Problem

Imagine you have an S3 bucket called test-bucket that contains several subfolders, as shown below:

[[See Video to Reveal this Text or Code Snippet]]

In your scenario, you specifically want to loop through the photos folder and get the names of the subfolders that are older than two years, like 2018 and possibly others. The existing approach you’ve been using may not capture all folders effectively.

Solution Overview

The solution lies in using the list_objects_v2() method from the Boto3 library, along with the Delimiter parameter. By setting a delimiter, you can tell S3 to group the results, which will allow you to easily retrieve the subfolder names. Here’s how you can do that:

Step-by-Step Code Example

Set up the Environment: Ensure that you have the Boto3 library installed and that your AWS credentials are configured correctly.

Access the S3 Bucket: Use the boto3 client to interact with your S3 bucket.

Use List Objects with Delimiter: When you call list_objects_v2(), you can pass a delimiter that will return only the subfolder names.

Here’s a simple code snippet to demonstrate this:

[[See Video to Reveal this Text or Code Snippet]]

Breakdown of the Code

Initialize S3 Client: The boto3.client('s3') creates a new client for interacting with S3.

Listing Objects: By calling list_objects_v2() with the parameters Bucket, Delimiter, and Prefix, you ensure that only subfolders are listed.

Looping Through Results: The code checks if there are CommonPrefixes in the result and prints each prefix, which represents the subfolder names.

Conclusion

Using the Boto3 library to interact with your Amazon S3 buckets makes it easy to perform various operations, such as retrieving subfolder names. By utilizing the Delimiter parameter, you can efficiently filter your results and focus only on the relevant folder structure without dealing with the complications of individual file listings.

Now that you have this step-by-step guide, you can implement it in your projects and easily manage your data stored in S3 buckets. Feel free to expand this approach further to include additional filters or to handle specific use cases. Happy coding!