I am currently designing several dockerized workers that I would like to run from time to time, o demand and recurrently, on my Qovery environment.
I would like to be able deploy one or several worker on my Qovery envrionment on a specific external event (thinking to SQS for its lightness…, one queue per job type) to handle the on demande deployments.
Do you have any idea what would be the best way to design this ? I am thinking to create a small service, that would be up anytime in my Qovery environment, listening to SQS queue and triggering job deployment with arguments thanks to Qovery API.
Hi @qvdp ,
I like what you are trying to achieve. I have a couple of questions to understand your need better.
- You want to expose (HTTP?) a service that will handle some requests.
- This service will deploy and execute a Worker. How many workers would you like to deploy?
- Once the execution is done, must the Worker be stopped or dropped?
Hi @rophilogene ,
I was considering using n8n at first time to achieve this, but I don’t like the “no code” bias as we can not maintained workflow in a git repository, for tests, releasing, and also workflow are executed in same environments, there is no containerization.
Concerning your demands:
- The service could expose an HTTP API yes, at least for the health checks, but only internally to my namespace, but the service would focus only on worker management
- About the number of workers, to be increased in time as we gain customers, but atm. I have identified 2 type of workers:
- first one is for assets resizing, only on demand, difficult to estimate but two/three time a day, depending on workload of our customers and not during the weekend
- second one is for data scraping, on demand two/three time a week and recurring each night if customer choose to.
- Once the execution is done, a worker can be safely dropped.
Thank you for you time
Hi @qvdp , it’s clear to me. What you want to achieve is perfectly doable via Qovery in a programmatic or non-programmatic way. We provide API and clients you can use to provide services with the appropriate configuration.
You can take a look at this documentation.
If you need some examples of using the API, you can take a look at the following resources:
Let me know if it helps and if you have any other questions.