Thursday, 5 September 2013

How can I batch doesObjectExist() requests to Amazon S3?

How can I batch doesObjectExist() requests to Amazon S3?

I need to check whether a set of keys exist in S3, for each of a large
number of items. (Each set of keys relates to one of the large number of
items).
I am using the PHP SDK (v2)
Currently I am calling $client->doesObjectExist(BUCKET, $key) for each of
the keys, which is a bottleneck (the round-trip time to S3 for each call).
I would prefer to do something like $client->doesObjectExist(BUCKET,
$batch) where $batch = array($key1, $key2 ... $keyn), and for the client
to check all of those keys then come back with an array of responses (or
some other similar structure).
I have come across a few references to a "batch api" which sounds
promising, but nothing concrete. I'm guessing that this might have been
present only in the v1 SDK.

No comments:

Post a Comment