Commit 8162bb1e authored by Elouan Martinet's avatar Elouan Martinet
Browse files

[storages] Skip already consumed keys in upload postcommit event

parent 5fea5d55bd6f
Pipeline #36568 failed with stage
in 1 minute and 34 seconds
......@@ -186,7 +186,11 @@ class S3AddFileOp(DataOperationMixIn, LateOperation):
containercls = list
def postcommit_event(self):
consumed_keys = set()
for storage, key, eid, attr in self.get_data():
if key in consumed_keys:
continue
consumed_keys.add(key)
suffixed_key = storage.suffixed_key(key)
storage.s3cnx.copy_object(
Bucket=storage.bucket,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment