Help setting up Airbyte and using Kubernetes EKS

Questions and Answers
,
11

The error I see is the following:

io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: helm-z93111b36-airbyte-temporal/172.20.111.207:7233

from helm-z93111b36-airbyte-worker-76b765b7cd-8jstf

Troubleshooting

This is what I’ve done to troubleshoot.

1. No Custom Values File (OK) :white_check_mark:

I removed the whole values file to validate that the base configuration works. And the default conf works.

2. Override Global Database (OK) :white_check_mark:

Then, I overrode the global database configuration to use the external DB available and deployed it with Qovery. (It worked)

postgresql:
  enabled: false

global:
  database:
    type: "external"
    # -- The database host
    host: qovery.env.DATABASE_HOST
    # -- The database port
    port: qovery.env.DATABASE_PORT
    # -- The database name
    database: qovery.env.DATABASE_NAME
    # -- The database user
    user: qovery.env.DATABASE_USER
    # -- The database password
    password: qovery.env.DATABASE_PASSWORD

3. Override Global Database + S3 Bucket (OK) :white_check_mark:

This conf also worked:

postgresql:
  enabled: false

global:
  database:
    type: "external"
    # -- The database host
    host: qovery.env.DATABASE_HOST
    # -- The database port
    port: qovery.env.DATABASE_PORT
    # -- The database name
    database: qovery.env.DATABASE_NAME
    # -- The database user
    user: qovery.env.DATABASE_USER
    # -- The database password
    password: qovery.env.DATABASE_PASSWORD

  storage:
    type: "S3"
    bucket:
      log: <truncated>
      state: <truncated>
      workloadOutput: <truncated>
    s3:
      region: "ap-southeast-2"
      authenticationType: instanceProfile

But I saw a non-critical error from your S3 auth:

Cannot publish to S3: User: arn:aws:sts::[TRUNCATED]:assumed-role/qovery-eks-workers-zXXX/i-XXX is not authorized to perform: kms:GenerateDataKey on resource: arn:aws:kms:ap-southeast-2:...

4. Override Global Database + S3 Bucket + externalDatabase :white_check_mark:

This one also did work

postgresql:
  enabled: false

global:
  database:
    type: "external"
    # -- The database host
    host: qovery.env.DATABASE_HOST
    # -- The database port
    port: qovery.env.DATABASE_PORT
    # -- The database name
    database: qovery.env.DATABASE_NAME
    # -- The database user
    user: qovery.env.DATABASE_USER
    # -- The database password
    password: qovery.env.DATABASE_PASSWORD

  storage:
    type: "S3"
    bucket:
      log: <truncated>
      state: <truncated>
      workloadOutput: <truncated>
    s3:
      region: "ap-southeast-2"
      authenticationType: instanceProfile

externalDatabase:
  jdbcURL: qovery.env.DATABASE_URL

5. Temporal fix? :red_circle:

This configuration with the Temporal fix does not work.

postgresql:
  enabled: false

global:
  database:
    type: "external"
    # -- The database host
    host: qovery.env.DATABASE_HOST
    # -- The database port
    port: qovery.env.DATABASE_PORT
    # -- The database name
    database: qovery.env.DATABASE_NAME
    # -- The database user
    user: qovery.env.DATABASE_USER
    # -- The database password
    password: qovery.env.DATABASE_PASSWORD
  
  # -- Temp fix for temporal issue
  temporal:
    image:
      tag: "1.20.1"
    extraEnv:
      - name: "SQL_TLS" 
        value: "true"
      - name: "SQL_TLS_DISABLE_HOST_VERIFICATION"
        value: "true"
      - name: "SQL_TLS_ENABLED"
        value: "true"
      - name: "SQL_HOST_VERIFICATION" 
        value: "false"

  storage:
    type: "S3"
    bucket:
      log: <truncated>
      state: <truncated>
      workloadOutput: <truncated>
    s3:
      region: "ap-southeast-2"
      authenticationType: instanceProfile

externalDatabase:
  jdbcURL: qovery.env.DATABASE_URL

–

Why do you need to change the Temporal configuration? This seems to be the reason for the failed deployment.