Recently we discussed how to design a serverless batch process that notifies data consumers using S3 bucket notifications. However, when using Terraform to deploy notifications there are a few things we must take into account.
- How Terraform deploys S3 bucket notifications
- Who else is using the same S3 bucket
How Terraform deploys S3 bucket notifications
When using Terraform to deploy your S3 bucket notifications, Terraform treats the selected S3 bucket as its own resource. Therefore, Terraform will remove any existing notifications on the bucket that it did not create. This situation also holds true when multiple Terraform projects create notifications for the same bucket. When each project deploys their changes, they will overwrite the existing S3 notifications. Understanding this is critical to ensuring you do not have overwrite contention with other projects or applications.
Who else is using the same S3 bucket
Answering the following questions will give you a better understanding of your bucket:
- Who owns the bucket? Is this a shared bucket or do you own the bucket?
- Does more than one application or Terraform project use the bucket?
If the bucket you are using is a shared bucket, then you must identify all users. Doing so will make it easier for you to implement your solution for deploying S3 bucket notifications with Terraform.
Implementing our solution
In order to prevent overwrite contention, you should have a common Terraform project that is solely responsible for deploying S3 bucket notifications. For shared or common buckets, you should notify all stakeholders of the project and transfer all existing notifications to the newly created project. For buckets that you own, you can simply create a Terraform project to house and deploy your events.
Conclusion
When using Terraform to deploy S3 bucket notifications, you should create a notification specific project for each bucket. This will ensure the coexistence of all created notifications. If you have further questions regarding S3 bucket notifications, please do not hesitate to contact us.
