Resource Automation & Auto-Scaling Options

[ChrisBolman1 (Chris Bolman)]

Is there any way in Qovery to set a rule to restart a pod if it reaches a certain CPU % threshold?

We have certain computationally intensive processes that run our app, and since that job may only run in a single pod (or perhaps two), it doesn’t trigger auto-scaling because overall CPU remains low (below the 60% threshold) across all our instances.

It would be very helpful if we could auto-refresh those high CPU pods after a certain period of time to make sure the application has sufficient resources. We will do other optimizations on our end, but this would be quite helpful short-term if something like this exists or there are similar options. thanks

[rophilogene (Romaric Philogene)]

Hi @ChrisBolman1 ,

Thank you for explaining why you would need to restart a pod if it reaches a certain CPU threshold.

Short answer is

yes

Long answer

Qovery does not support this out of the box, however you can do something which is:

1. Create a toolbox project with an environment.
2. Create a cronjob with a script that will poll the Qovery API every X minutes.
   2.a We have an WebSocket API endpoint to poll the usage metrics from an app
   2.b This is the API endpoint to restart an application.
3. Restart your service if you reach your defined threshold.

Does that make sense?

[ChrisBolman1 (Chris Bolman)]

Thanks @rophilogene - setting something like this up in practice: is there documentation on the Websockets API to pull usage?

Reading REST API docs, are containers the same thing as pods? it seems like if there’s a way to get pods by application, I could then health and CPU check each one, but I don’t see a query that allows that. Currently can only GET container by ID correct so this won’t work? https://api.qovery.com/container/{containerId}

Ideally cron could:

1. GET application by ID
2. GET pods for that application
3. Check each pod
4. POST to https://api.qovery.com/container/{containerId}/restart-service (or equivalent to restart the pod)

But if you let me know how to get the metrics from WebSocket API and I can also parse that usage data and then have cron call https://api.qovery.com/application/{applicationId}/restart

Also, to confirm, the docs say application/{applicationId}/restart is Deprecated - Please use the “Redeploy application” endpoint now. Is it still ok to use or should it call https://api.qovery.com/application/{applicationId}/deploy instead?

[rophilogene (Romaric Philogene)]

@ChrisBolman1

Thanks @rophilogene - setting something like this up in practice: is there documentation on the Websockets API to pull usage?

The open API page we generate does not display properly WebSocket endpoints - so we have this page qovery-openapi-spec/websocket/websocket-openapi.yaml at main · Qovery/qovery-openapi-spec · GitHub

I can provide an example with Python today if needed?

@ChrisBolman1

Reading REST API docs, are containers the same thing as pods? it seems like if there’s a way to get pods by application, I could then health and CPU check each one, but I don’t see a query that allows that. Currently can only GET container by ID correct so this won’t work? https://api.qovery.com/container/{containerId}

Today, we make a clear distinction between applications deployed from a Git Repository (api.qovery.com/application) and a Container Registry (api.qovery.com/container). (Both endpoints will probably be merged this year).

So if your apps are deployed via Git, you need to look at the application resources from our API. Otherwise, it’s container resources.

Note: To list all your applications, you can use the Endpoint api.qovery.com/environment/:envId/application or api.qovery.com/environment/:envId/container

@ChrisBolman1

Also, to confirm, the docs say application/{applicationId}/restart is Deprecated - Please use the “Redeploy application” endpoint now. Is it still ok to use or should it call https://api.qovery.com/application/{applicationId}/deploy instead?

Actually, you should use the new restart endpoint if you want to restart (kill the process SIG 9). The deploy endpoint will not restart your app if it has not changed.

[ChrisBolman1 (Chris Bolman)]

Thanks @rophilogene. Tried connecting to the websocket API using websocat client to see what the response looks like and I’m getting: 401 Unauthorized

I included our Qovery API key as an api_key={{ key }} param in the request.

A python example would be helpful. Also are the application and service params interchangeable? In this case as I understand it our service is an application

[rophilogene (Romaric Philogene)]

@ChrisBolman1

A python example would be helpful. Also are the application and service params interchangeable? In this case as I understand it our service is an application

Yes, a service can be an application, container, database, lifecycle job, or cronjob… Actually all the resources you can put inside an Environment.

You can find a definition here.

@ChrisBolman1

Thanks @rophilogene. Tried connecting to the websocket API using websocat client to see what the response looks like and I’m getting: 401 Unauthorized

I included our Qovery API key as an api_key={{ key }} param in the request.

Ok I’ll provide an example as soon as I can today

[a_carrano (Alessandro Carrano)]

@rophilogene

Ok I’ll provide an example as soon as I can today

Hi @ChrisBolman1 ,

sorry for the late reply. do you still need the example as you requested?