More customization of transfer via config

[kahkeng]

I would like to be able to customize various aspects of the transfer (buffer size, number of retries per part or total, etc) which are currently hardcoded as constants in TransferClient.h or UploadFileRequest.cpp.

There's a TransferClientConfiguration struct currently used for specifying the upload buffer count, but that's only used inside TransferClient. If re-using this struct for the above configurable parameters is desirable, what would be the best way to make this available to UploadFileRequest from TransferClient? Would passing this config in via the constructors of UploadFileRequest be reasonable? If so, I could submit a pull request for this.

[kahkeng]

Any thoughts about this? The issue for me is that there is a hard limit of 50GB for uploads now because of the fixed 5MB part size and the 10,000 part limit for multipart uploads. I would like to adjust this when using the transfer client. Thanks!

[JonathanHenson]

Since TransferClientConfiguration is a struct, feel free to pass it around, just make sure you don't take a reference to it. Instead copy params out that you want into the object that wants the values.

I think the customization you want to do is an excellent idea.

[kahkeng]

Sorry haven't had time to work on this but I will try to submit this when I can.

[JonathanHenson]

This code base has been deleted with our GA release. Using the new transfer manager interface, you can now change the part sizes, specifiy putobject, create multipart upload, and upload part templates for the transfer manager to use. Also the metadata can be passed to the UploadFile function.