404 Not Found for a staging environment

Hi,

One of our staging environments is showing 404 Not Found from NGINX after the deployment.

The deployment does not show an error and the health checks seem to be OK. I have also checked Cloudwatch logs to see if I spot any issues but nothing so far.

I am not sure if this is related to the recent Cluster Update that happened this morning Standard cluster update - 09/19/2024

The environment ID is de02a9a0-69ca-4361-b9a8-77d7acfd942e. Can someone please check?

Thanks,
Simon

Hello @simofacc,

Indeed, just saw the deployment on your cluster failed somehow, just relaunched it right now everything seems to be fine.
Can you confirm?

If you experience the same behavior on another exposed service, redeploying the service should fix the issue.

Sorry for the inconvenience,
Cheers

Hi Benjamin,

I redeployed multiple times before posting. We had an issue with other environments too.

Seems OK right now.

Thanks,
Simon

1 Like

Perfect ! Thanks, let me know if you see anything else fishy.

Hello,

I have the same issue with my staging environment from yesterday

I redeployed multi times but it doesn’t fix the issue.

Console url: https://console.qovery.com/organization/384a8263-f2b0-4694-9793-6eb69ce1a0ce/project/9417c24f-d4ac-4daf-b9c5-51730197025d/environment/2777c5c3-3f7f-4c68-b99e-9aaba0eb1a0a/services/general

Hello @Geoffrey_S,

Just checked, you cluster was somehow deploying for couple days.
We cancelled and trigger an update on the cluster, issue should be fixed within the next 10 minutes.
We have made a patch for this issue, it will be shipped next Monday on our side, so error shouldn’t happen again.

I keep an eye on your cluster deployment in the meantime.

Should be good now @Geoffrey_S, let me know if it’s ok for you.

Cheers

Hello @bchastanier

Everything seems to be working again, thank you.

Hi Benjamin,

we have the same issue, nginx in our dev environment suddenly showing 404 Not Found, I have re-deployed several times but not worked, please help to take a look https://console.qovery.com/organization/b4271b12-477a-4b41-a274-9c9cba8043e8/project/9595f3a3-73f6-45fa-8688-00dc44c4de57/environment/bf73b088-92d2-4134-8c82-34b67f151c4e/services/general

Hey @Ethan_Huang

Unfortunately, it seems update didn;t went through to your cluster because you did the 1.29 upgrade on AWS console directly (not via Qovery). While 1.29 support should be out soon, we cannot properly update your cluster with nginx change as of now, hence your applications facing 404 because wrong ingress classes are set.

You have two options:
1- create a new cluster and migrate your workload on it;
2- wait for us to support 1.29 (couple weeks). In the meantime, you won’t be able to make any changes on your cluster (advanced settings, node types, etc) and we would need to manually patch your cluster to declare the old ingress class.

While the option 1 seems to be the better one, please let me know if you pick the second one.

Cheers