non-Qovery managed AWS RDS Cluster (its in AWS managed VPC)
Communication between Qovery managed Application and non-Qovery managed RDS is enabled via VPC Peering.
Preview Environment Requirement
We want to enable Preview-Environments feature on our Production-Environment. In Preview-Environment, Qovery will create a new set of Applications on a Pull Request but it won’t clone the RDS as that is being managed by AWS. Similarly on merging of a Pull Request, Qovery will delete the Preview-Environment and the Applications but it won’t delete the RDS managed by AWS. We wanted to check how (and if) we can utilize Qovery’s customScripts/Webhooks or other features to implement this custom behavior where RDS cluster can be created from a snapshot on PR creation and deleted on PR merge.
Your production environment is managed by Qovery. But the RDS instance is managed by you and connected via the AWS VPC Peering to your production environment.
Each time that a Pull Request is opened, then a new Preview Environment is created with an RDS instance on a dedicated VPC. AND a snapshot coming from the production RDS instance is restored.
Each time that a Pull Request is deleted, then the associated Preview Environment and RDS instance are cleaned up.
Hi Romaric
I really appreciate the architecture diagram. One small update, our idea is to have only one dedicated production VPC where all the RDS clusters would be created. That way we won’t need to enable Peering between Qovery managed VPC and RDS VPC for all the preview environments.
So the only change in the diagram would be that all the RDS Clusters would be in the same AWS managed VPC.
Please let me know if this makes sense. Really appreciate the detailed context and diagram. Thank you
Important note: we will provide better support for integrating Qovery with external services. This is a temporary solution that would work fine and give you the full flexibility you need.
Create a blueprint environment
Turn off the Preview Environment feature for your production environment.
Clone your production environment and give a name blueprint to the cloned environment.
Turn on the Preview Environment feature for your blueprint environment.
The idea of the blueprint environment is to act as a root environment for every new Preview Environment.
The idea is to create an app that will react on the first creation and the deletion of an environment.
For the creation of the app, we’ll need to execute a Terraform file that will create the Preview Environment RDS Instance. While for the deletion of the app, we’ll execute a Terraform file that will delete the Preview Environment RDS instance.
Handling creation and deletion
import os
import signal
import sys
from flask import Flask
PROJECT_ID = os.getenv('PROJECT_ID')
ENVIRONMENT_ID = os.getenv('ENVIRONMENT_ID')
def create_app():
app = Flask(__name__)
# 1. Exec your Terraform command here.
# 1.a Create RDS instance and get the endpoint of the instance + the credentials.
# 2. Inject RDS instance endpoint and credentials into Qovery Env environment variables
return app
app = create_app()
def delete_rds_instance(_signo, _stack_frame):
print("Deleting RDS instance...")
# retrieve the credentials from the RDS instance
# delete the RDS instance with Terraform
sys.exit(0)
if __name__ == '__main__':
signal.signal(signal.SIGTERM, delete_rds_instance)
app.run(host="0.0.0.0", port=5555)
Here my example is with Python but you can use the language that fits best you
AWS RDS instance credentials
To get the AWS RDS instance credentials into your app, you have two options here:
Not recommended: You can hardcode the credentials of your RDS instance into your application
Recommended: You can generate the credentials and inject them as Qovery Secret via the Qovery API. I can provide an example if you want
Lifecycle App needs to start first
Since Lifecycle App creates all the resources required by your apps to properly starts, I’d recommend to force it to start first. You can take a look at this thread.
Let me know if you get the idea or you need any clarification
Hi Romaric
Thank you for the detailed workflow. Couple of quick questions -
Just for clarity, so signal.SIGTERM is used here to trigger the deletion of RDS when the Preview Environment gets deleted ,thereby running the delete_rds_instance function?
For the Preview environment’s App to access the new RDS, we need to do following?
When RDS will be created it will have a new endpoint and configs (e.g username, password, db)
We can use Qovery API to update the environment variables of Preview Environment with new RDS credentials so that Application connects with the newly created RDS?
Hi Romaric
Sounds good. This has been super helpful. I really appreciate the detailed and precise assistance.
Yes, I’ve been using Python and Terraform for this task also. As Qovery has the python API client so I guess updating the environment variables should also be smooth and simple.
Hi @shaider , FYI Lifecycle Job is available now! It’s the perfect feature to support this use case. So feel free to use it. I will provide some examples in the next few days.
Important note: we will provide better support for integrating Qovery with external services. This is a temporary solution that would work fine and give you the full flexibility you need.
- the function