Table of Contents
- Calculating an S3 ETag using Python
- Determine the partsize/chunksize
- Verifying a local file
For an introduction on S3 ETags and how they are calculated see our first post All about AWS S3 ETags.
The following will detail how to calculate the S3 ETag for a local file. We’ve used Python, however the logic can be applied elsewhere if desired.
Calculating an S3 ETag using Python
Given a file and a partsize/chunksize you can easily calculate the S3 ETag for a local file.
def calc_etag(inputfile, partsize):
Determine the partsize/chunksize
However if you need to determine the partsize/chunksize you can make an educated guess using the following information:
- The size of the file/object
- The number of parts used in the multipart upload
def factor_of_1MB(filesize, num_parts):
Make an educated guess
Assuming the file/object has the following properties:
- Size: 6543277389 bytes (6.1 GB)
- Etag: d41d8cd98f00b204e9800998ecf8427e-38
The calculated part size would be 173015040 bytes (165 MB)
Verifying a local file
A quick way to put this all together into a command line script