Stop making a new S3 bucket for every integration tests run #127
Conversation
| @@ -12,6 +12,8 @@ provider: | |||
| DD_API_KEY: ${env:DD_API_KEY} | |||
| lambdaHashingVersion: 20201221 | |||
| timeout: 15 | |||
| deploymentBucket: | |||
| name: integration-tests-deployment-bucket | |||
There was a problem hiding this comment.
Can you link to docs for deploymentBucket? I don't see where it sets up the 1 day lifecycle policy by default.
There was a problem hiding this comment.
The lifecycle policy is a setting in AWS. I hardcoded this bucket name in the test, and then configured the bucket in AWS to delete all objects older than one day.
Does that answer your question or do you still want to see docs for something?
There was a problem hiding this comment.
Nah, we're good. I misunderstood you, I thought deploymentBucket was setting the lifecycle policy automatically. It looks like it automatically prunes artifacts once it has more than 5 artifacts, which is pretty good in and of itself. https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml/
There was a problem hiding this comment.
Just a quick question about the deploymentBucket. It looks like a good change regardless. Doing something like this in your run_integration_tests.sh will help tear down tests if they're cancelled midway through as well.
63 # trap remove_stack EXIT should go immediately after we create the stack
64 remove_stack() {
65 if [ "$KEEP_STACK" = true ]; then
66 return
67 fi
68 echo "About to remove!"
69 echo serverless remove -- java-layer-version "$tracing_layer_version"
70 serverless remove --java-layer-version "$tracing_layer_version"
71 }
72
73 trap remove_stack EXIT
|
Thanks @agocs, |
What does this PR do?
Updates the integration tests to use a single shared S3 bucket for deployment rather than creating a new one every time.
Motivation
Sometimes, the CloudFormation stack created by integration tests might not be deleted at the end of the test run (e.g. if the tests are cancelled). When this happens, we cannot easily remove the CloudFormation stack after the test run, because the stack contains an S3 bucket that contains an Object. We have to empty the S3 bucket before we can manually delete the stack.
By having one shared bucket for all integration test runs, we no longer have to worry about this issue when removing the stack. This bucket also has a lifecycle policy set that will remove objects after one day, so they will never accumulate if they are left behind.
Testing Guidelines
I ran the integration tests.
Additional Notes
It is still possible that we might end up with extraneous CloudFormation stacks, but they will be easier to remove and we can deal with that issue later if it becomes a problem.
Types of Changes
Check all that apply