Thanks! Iād like to get both Airbyte and JupyterHub (latest versions of the Helm charts) deployed successfully. Iām trying a non-Karpenter enabled EKS cluster first. I was able to deploy both previously on ARM nodes, Iāll try AMD nodes today.
I wouldnāt make a mistake that basic. All clusters were live and operational when I tried to deploy the latest versions of Airbyte and JupyterHub on them. You can look at the deployment histories of Airbyte and JupyterHub to see the failing deployments. You can also duplicate the problem yourself by deploying Airbyte and JupyterHub onto your own EKS clusters.
I am looking at your issue.
One of the issue is that your cluster size is to small. Your have for example minio pod that canāt start because the cluster has reached its maximun number of nodes.
So you should give some extra room for the cluster to be able to expand, and increase the maximun number of nodes of the cluster.
The second issue is more on our side, where we set a hard timeout of 5 min to download of the depency of the chart, and it seems now that Airbyte is hitting this timeout all the time.
I am going to increase our hard timeout, and keep you in touch.
Hi back we have increased our timeout for the dependency fetch, now the only thing left for you is to increase the cluster max node size. As the minio pod canāt start.
pod didn't trigger scale-up: 1 max node group size reached
Hello, any updates? Should I shut down our Qovery managed cluster if Qovery doesnāt have the bandwidth to debug this? You guys are welcome to re-deploy the cluster if you have the bandwidth to help debug this.
First of all, stick to the version from my tutorial (1.1.x at the time) just to make sure I did have a valid working version.
I did try to deploy it on your EKS cluster with and without Karpenter. The error was the same - related to minio airbyte pod not starting. So I suspect a bug issue with this pod and I have a few ideas that Iāll explore today. (I suspect some metadata still present on the persistent storage for minio and blocking the proper startup of this service).
In the meantime, and for production purpose, Iād suggest using a s3 storage from AWS or equivalent to remove the dependency to minio (which is not recommended for production purpose).
EDIT: I removed the above environments since both are working; will replace with a more permanent one later.
Donāt worry about the minio airbyte pod; weāre not going to use it in production. Iāll also try using an external AWS managed database service with Airbyte:
There seems to be a
ready.go:284: [debug] PersistentVolumeClaim is not bound: z103d13c4-jupyterhub-production-amd/jupyterhub-hub-db-dir
bug that wasnāt there 2 weeks ago. I was able to deploy JupyterHub from the Helm chart successfully using the instructions above two weeks ago.