Cloning non-Qovery managed AWS-RDS cluster in Preview Environments

Hi, here’s our use case.

Production Environment

  • Qovery managed Applications
  • non-Qovery managed AWS RDS Cluster (its in AWS managed VPC)
  • Communication between Qovery managed Application and non-Qovery managed RDS is enabled via VPC Peering.

Preview Environment Requirement

We want to enable Preview-Environments feature on our Production-Environment. In Preview-Environment, Qovery will create a new set of Applications on a Pull Request but it won’t clone the RDS as that is being managed by AWS. Similarly on merging of a Pull Request, Qovery will delete the Preview-Environment and the Applications but it won’t delete the RDS managed by AWS. We wanted to check how (and if) we can utilize Qovery’s customScripts/Webhooks or other features to implement this custom behavior where RDS cluster can be created from a snapshot on PR creation and deleted on PR merge.

Thank you.

Hi @shaider ,

Can you confirm that is the following architecture that you want to put in place with Qovery?

  1. Your production environment is managed by Qovery. But the RDS instance is managed by you and connected via the AWS VPC Peering to your production environment.
  2. Each time that a Pull Request is opened, then a new Preview Environment is created with an RDS instance on a dedicated VPC. AND a snapshot coming from the production RDS instance is restored.
  3. Each time that a Pull Request is deleted, then the associated Preview Environment and RDS instance are cleaned up.

Is it correct?

Hi Romaric
I really appreciate the architecture diagram. One small update, our idea is to have only one dedicated production VPC where all the RDS clusters would be created. That way we won’t need to enable Peering between Qovery managed VPC and RDS VPC for all the preview environments.

So the only change in the diagram would be that all the RDS Clusters would be in the same AWS managed VPC.

Please let me know if this makes sense. Really appreciate the detailed context and diagram. Thank you

Super clear :ok_hand:

Can you confirm then it will be like this?

Hi Romaric
Yes that’s exactly what we are targeting. Really appreciate the detailed diagram. Thank you.

1 Like

Here is what I would suggest.

:warning: Important note: we will provide better support for integrating Qovery with external services. This is a temporary solution that would work fine and give you the full flexibility you need.

Create a blueprint environment

  1. Turn off the Preview Environment feature for your production environment.
  2. Clone your production environment and give a name blueprint to the cloned environment.
  3. Turn on the Preview Environment feature for your blueprint environment.

The idea of the blueprint environment is to act as a root environment for every new Preview Environment.

Add lifecycle app

To manage the creation of the external RDS instance, we will add an application into our blueprint environment named lifecycle. This one will:

  1. Create the RDS instance and load the latest production RDS snapshot when the Preview Environment is created.
  2. Delete the RDS instance when the Preview Environment is deleted.

Lifecycle App code example

The idea is to create an app that will react on the first creation and the deletion of an environment.

For the creation of the app, we’ll need to execute a Terraform file that will create the Preview Environment RDS Instance. While for the deletion of the app, we’ll execute a Terraform file that will delete the Preview Environment RDS instance.

Handling creation and deletion

import os
import signal
import sys

from flask import Flask

PROJECT_ID = os.getenv('PROJECT_ID')

def create_app():
    app = Flask(__name__)

    # 1. Exec your Terraform command here.
    # 1.a Create RDS instance and get the endpoint of the instance + the credentials.
    # 2. Inject RDS instance endpoint and credentials into Qovery Env environment variables

    return app

app = create_app()

def delete_rds_instance(_signo, _stack_frame):
    print("Deleting RDS instance...")

    # retrieve the credentials from the RDS instance
    # delete the RDS instance with Terraform


if __name__ == '__main__':
    signal.signal(signal.SIGTERM, delete_rds_instance)"", port=5555)

Here my example is with Python but you can use the language that fits best you

AWS RDS instance credentials

To get the AWS RDS instance credentials into your app, you have two options here:

  1. Not recommended: You can hardcode the credentials of your RDS instance into your application
  2. Recommended: You can generate the credentials and inject them as Qovery Secret via the Qovery API. I can provide an example if you want

Lifecycle App needs to start first

Since Lifecycle App creates all the resources required by your apps to properly starts, I’d recommend to force it to start first. You can take a look at this thread.

Let me know if you get the idea or you need any clarification

Hi Romaric
Thank you for the detailed workflow. Couple of quick questions -

Just for clarity, so signal.SIGTERM is used here to trigger the deletion of RDS when the Preview Environment gets deleted ,thereby running the delete_rds_instance function?

For the Preview environment’s App to access the new RDS, we need to do following?

  • When RDS will be created it will have a new endpoint and configs (e.g username, password, db)
  • We can use Qovery API to update the environment variables of Preview Environment with new RDS credentials so that Application connects with the newly created RDS?
1 Like

Absolutely :100: - the function delete_rds_instance(..) is called when a SIG TERM is sent to the container/pod (on deletion).

Yes, exactly! Then, it will be transparent for your application.

Out of topic:

  1. Are you familiar with Python? I can provide the equivalent in another language that you are more comfortable with if you want.
  2. Qovery provides API clients for Python / Golang / Javascript and Typescript.

Hi Romaric
Sounds good. This has been super helpful. I really appreciate the detailed and precise assistance.

Yes, I’ve been using Python and Terraform for this task also. As Qovery has the python API client so I guess updating the environment variables should also be smooth and simple.

1 Like