Empty AWS S3 bucket of arbitrary cardinality with NodeJS & TypeScript

My removeObjects function has me stummped.The function is suppose to syncronoulsy get a list of objects in an S3 bucket then asyncronously removes the objects. Repeat if the list was truncated, until the there are no more objects to remove. (AWS doesn't provide the total count of objects in the bucket and listObjects pages the results.)

What am I doing wrong / why doesn't my function work? The solution should exploit single thread and async nature of JS. For the bounty I am hoping for an answer specific to the module. The git repo is public if you want to see the entire module.

export function removeObjects(params: IS3NukeRequest): Promise<S3.Types.DeleteObjectsOutput> {    const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];   let isMore;    do {     listObjectsSync(params)       .then((objectList: S3.Types.ListObjectsV2Output) => {         isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;         requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());       })       .catch((err: Error) => { Promise.reject(err); });   } while (isMore);    return Promise.all(requests); }  export async function listObjectsSync(params: IS3NukeRequest): Promise<S3.Types.ListObjectsV2Output> {   try {     return await params.Client.listObjectsV2(s3nukeRequest2listObjectsRequest(params)).promise();   } catch (err) {     return Promise.reject(err);   } } 

Thanks.

1 Answers

Answers 1

The thing is that listObjectsSync function returns a Promise, so you need to treat it as an async function and can't just use a loop with it. What you need to do is to create a chain of promises while your isMore is true, I've done it using a recursive approach (I'm not pro in TS, so please check the code before using it). I also haven't tried the code live, but logically it should work :)

 const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];   function recursive(recursiveParams) {     return listObjectsSync(recursiveParams).then((objectList: S3.Types.ListObjectsV2Output) => {         let isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;                       requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());         if (isMore) {           //do we need to change params here?           return recursive(recursiveParams)         }         //this is not necessary, just to indicate that we get out of the loop         return true;           });   }    return recursive(params).then(() => {      //we will have all requests here      return Promise.all(requests);   }); 

If You Enjoyed This, Take 5 Seconds To Share It

• Facebook
• Twitter
• Google+
• StumbleUpon
• Digg
• Delicious
• LinkedIn
• Reddit
• Technorati