Local cluster/qovery installation partially failed

Hey :wave:

I tried to install Qovery locally by following the instructions of this blog post: Experience Qovery on Your Local Machine

Unfortunately the command qovery demo up fails with (the output is of the second command execution):

qovery demo up
""""""""""""""""""""""""""""""""""""""""""""
Checking and installing dependencies
""""""""""""""""""""""""""""""""""""""""""""
jq already installed
grep already installed
sed already installed
curl already installed
docker already installed
k3d already installed
helm already installed
All dependencies are installed

""""""""""""""""""""""""""""""""""""""""""""
Fetching Qovery values to setup your cluster
""""""""""""""""""""""""""""""""""""""""""""
Helm values written into values.yaml

""""""""""""""""""""""""""""""""""""""""""""
Installing Qovery helm repositories
""""""""""""""""""""""""""""""""""""""""""""
"qovery" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "qovery" chart repository
Update Complete. ⎈Happy Helming!⎈

""""""""""""""""""""""""""""""""""""""""""""
Creating local-demo-peterkuhmann kube cluster
""""""""""""""""""""""""""""""""""""""""""""
INFO[0000] Using the k3d-tools node to gather environment information 
INFO[0000] Starting new tools node...                   
INFO[0000] Starting node 'k3d-local-demo-peterkuhmann-tools' 
INFO[0001] Starting cluster 'local-demo-peterkuhmann'   
INFO[0001] All servers already running.                 
INFO[0001] All agents already running.                  
INFO[0001] All helpers already running.                 
INFO[0001] Started cluster 'local-demo-peterkuhmann'    

""""""""""""""""""""""""""""""""""""""""""""
Installing Qovery helm charts
""""""""""""""""""""""""""""""""""""""""""""
+ helm upgrade --install --create-namespace -n qovery -f values.yaml --wait --atomic qovery qovery/qovery
Error: UPGRADE FAILED: an error occurred while rolling back the release. original upgrade error: pre-upgrade hooks failed: 1 error occurred:
	* timed out waiting for the condition

: release qovery failed: context deadline exceeded
ERRO[2024-05-31T12:00:52+02:00] Error executing the command exit status 1 

The cluster appears in the web UI, but I service deployment is infinitely pending:

There are four audit log entries (see attached image):

Best
Peter

Hey @hedgehogs-mind :wave:,

What’s your OS? And your OS version?

Can you also run the command: k3d cluster list?

Ps: why did you run twice qovery demo up?

Thank you.

@rophilogene

  • MacOS 14.3.1 with an M2 chip
  • Darwin Kernel Version 23.3.0
  • Docker Engine v24.0.2

The first execution failed with the same error. So I tried it another time.

Could it be related to not enough RAM? I now increased the Docker Engine RAM and wanted to start from scratch, but qovery demo up hangs somewhere. :thinking:

Thx :pray:

Got it!

Yes, it’s absolutely possible. How much CPU and RAM did you allocate?

Could you allocate at least 6 GB and 3 vCPU to the Docker runtime?

Before I changed the settings, the engine had 8 GB of RAM and 5vCPU.

I now allocated 16 GB of RAM and 7 vCPU. Now qovery demo up worked flawlessly.

Now I get a little farther, but it still fails. :thinking:

qovery demo up      
""""""""""""""""""""""""""""""""""""""""""""
Checking and installing dependencies
""""""""""""""""""""""""""""""""""""""""""""
jq already installed
grep already installed
sed already installed
curl already installed
docker already installed
k3d already installed
helm already installed
All dependencies are installed

""""""""""""""""""""""""""""""""""""""""""""
Fetching Qovery values to setup your cluster
""""""""""""""""""""""""""""""""""""""""""""
Helm values written into values.yaml

""""""""""""""""""""""""""""""""""""""""""""
Installing Qovery helm repositories
""""""""""""""""""""""""""""""""""""""""""""
"qovery" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "qovery" chart repository
Update Complete. ⎈Happy Helming!⎈

""""""""""""""""""""""""""""""""""""""""""""
Creating local-demo-peterkuhmann kube cluster
""""""""""""""""""""""""""""""""""""""""""""
INFO[0000] portmapping '80:80' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy] 
INFO[0000] portmapping '443:443' targets the loadbalancer: defaulting to [servers:*:proxy agents:*:proxy] 
INFO[0000] Prep: Network                                
INFO[0000] Created network 'k3d-local-demo-peterkuhmann' 
INFO[0000] Created image volume k3d-local-demo-peterkuhmann-images 
INFO[0000] Creating node 'qovery-registry.lan'          
INFO[0000] Successfully created registry 'qovery-registry.lan' 
INFO[0000] Starting new tools node...                   
INFO[0000] Starting node 'k3d-local-demo-peterkuhmann-tools' 
INFO[0001] Creating node 'k3d-local-demo-peterkuhmann-server-0' 
INFO[0001] Creating LoadBalancer 'k3d-local-demo-peterkuhmann-serverlb' 
INFO[0001] Using the k3d-tools node to gather environment information 
INFO[0001] Starting new tools node...                   
INFO[0001] Starting node 'k3d-local-demo-peterkuhmann-tools' 
INFO[0002] Starting cluster 'local-demo-peterkuhmann'   
INFO[0002] Starting servers...                          
INFO[0002] Starting node 'k3d-local-demo-peterkuhmann-server-0' 
INFO[0005] All agents already running.                  
INFO[0005] Starting helpers...                          
INFO[0005] Starting node 'qovery-registry.lan'          
INFO[0006] Starting node 'k3d-local-demo-peterkuhmann-serverlb' 
INFO[0012] Injecting records for hostAliases (incl. host.k3d.internal) and for 4 network members into CoreDNS configmap... 
INFO[0015] Cluster 'local-demo-peterkuhmann' created successfully! 
INFO[0015] You can now use it like this:                
kubectl cluster-info

""""""""""""""""""""""""""""""""""""""""""""
Installing Qovery helm charts
""""""""""""""""""""""""""""""""""""""""""""
+ helm upgrade --install --create-namespace -n qovery -f values.yaml --atomic --set services.certificates.cert-manager-configs.enabled=false --set services.certificates.qovery-cert-manager-webhook.enabled=false --set services.qovery.qovery-cluster-agent.enabled=false --set services.qovery.qovery-engine.enabled=false qovery qovery/qovery
Release "qovery" does not exist. Installing it now.
NAME: qovery
LAST DEPLOYED: Fri May 31 17:25:35 2024
NAMESPACE: qovery
STATUS: deployed
REVISION: 1
TEST SUITE: None
+ set -x
+ helm upgrade --install --create-namespace -n qovery -f values.yaml --wait --atomic qovery qovery/qovery
W0531 17:27:17.842942   11349 warnings.go:70] spec.template.spec.containers[0].ports[1]: duplicate port definition with spec.template.spec.containers[0].ports[0]
W0531 17:27:17.914425   11349 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/instance"
W0531 17:27:17.914446   11349 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/managed-by"
W0531 17:27:17.914452   11349 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/name"
W0531 17:27:17.914456   11349 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/version"
W0531 17:27:17.914460   11349 warnings.go:70] unknown field "spec.secretTemplate.helm.sh/chart"
Error: UPGRADE FAILED: an error occurred while rolling back the release. original upgrade error: post-upgrade hooks failed: 1 error occurred:
	* timed out waiting for the condition

: release qovery failed: context deadline exceeded
ERRO[2024-05-31T17:40:27+02:00] Error executing the command exit status 1

I just tried and it works on my machine (sorry I know - it’s easy to say :sweat_smile: )

...
Release "qovery" does not exist. Installing it now.
NAME: qovery
LAST DEPLOYED: Fri May 31 18:37:56 2024
NAMESPACE: qovery
STATUS: deployed
REVISION: 1
TEST SUITE: None
+ set -x
+ helm upgrade --install --create-namespace -n qovery -f values.yaml --wait --atomic qovery qovery/qovery
W0531 18:38:57.173944   69764 warnings.go:70] spec.template.spec.containers[0].ports[1]: duplicate port definition with spec.template.spec.containers[0].ports[0]
W0531 18:38:57.250927   69764 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/instance"
W0531 18:38:57.250944   69764 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/managed-by"
W0531 18:38:57.250946   69764 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/name"
W0531 18:38:57.250948   69764 warnings.go:70] unknown field "spec.secretTemplate.app.kubernetes.io/version"
W0531 18:38:57.250949   69764 warnings.go:70] unknown field "spec.secretTemplate.helm.sh/chart"
Release "qovery" has been upgraded. Happy Helming!
NAME: qovery
LAST DEPLOYED: Fri May 31 18:38:50 2024
NAMESPACE: qovery
STATUS: deployed
REVISION: 2
TEST SUITE: None
+ set +x

""""""""""""""""""""""""""""""""""""""""""""
Configure network
""""""""""""""""""""""""""""""""""""""""""""
+ sudo ifconfig lo0 alias 172.42.0.3/32 up
+ set +x

""""""""""""""""""""""""""""""""""""""""""""
Qovery demo cluster is now installed !!!!
The kubeconfig is correctly set, so you can connect to it directly with kubectl or k9s from your local machine
To delete/stop/start your cluster, use k3d cluster xxxx

Go to https://console.qovery.com to create your first environment on this cluster 'local-demo-evoxmusic'
"""""""""""""""""""""""""""""""""""""""""""

The helm chart installs take some time (2/3 minutes), but it works.

Do you roughly know how long Helm takes until it fails?

@hedgehogs-mind I reduced my RAM and CPU, and indeed, it’s quite slow to install everything. I suspect if there is not enough CPU, then we hit the helm timeout. I’m going to ask the team to augment this just in case.

@hedgehogs-mind if you download the new/latest version of the cli, we have added a flag --debug to the qovery demo up --debug command.

If you can re-run the command and sahre us the debug log file located at /tmp/qovery-demo/qovery-demo.log we should be able to investigate further your issue.

Thank you! Will do that tomorrow. :slight_smile:

Shared it with you as a private message.