Hi, there is an issue in TF - I will use an example to explain it. Let’s say we have 2 applications - app1 and app2. App2 calls app1 so it has link to app1 in env variables:
{
key = "APP1_URL"
value = module.app1.internal_url
},
Now, if we change anything in app1 (add new env variable for example) and create TF plan, we see many changes in app2 as well, even it’s not actually changed (we see recreating built-in env variables, etc). Is anything can be done to avoid this?
@km1414 ,
Is it possible to share your tf files with us? At least a simplified version, we don’t know of this behavior yet, but we’ll try to replicate it to understand where lies the problem.
Thanks
Charles-Edouard
Here’s a simple example:
terraform {
required_providers {
qovery = {
source = "qovery/qovery"
version = "0.41.0"
}
}
}
provider "qovery" {
token = "..."
}
resource "qovery_environment" "test-environment" {
cluster_id = "..."
project_id = "..."
name = "test"
mode = "DEVELOPMENT"
}
resource "qovery_container" "app1" {
environment_id = qovery_environment.test-environment.id
registry_id = "9a1d6106-f2f3-42fc-951b-19db40e427c2"
name = "app1"
cpu = 100
memory = 64
image_name = "hello-world"
tag = "linux"
healthchecks = {}
environment_variables = [
{
key = "TEST"
value = "TEST"
}
]
}
resource "qovery_container" "app2" {
environment_id = qovery_environment.test-environment.id
registry_id = "9a1d6106-f2f3-42fc-951b-19db40e427c2"
name = "app2"
cpu = 100
memory = 64
image_name = "hello-world"
tag = "linux"
healthchecks = {}
environment_variables = [
{
key = "APP1_HOST"
value = qovery_container.app1.internal_host
}
]
}
Now, if I change env variable in app1, here’s what I get in the plan:
Terraform will perform the following actions:
# qovery_container.app1 will be updated in-place
~ resource "qovery_container" "app1" {
~ advanced_settings_json = jsonencode({}) -> (known after apply)
~ built_in_environment_variables = [
- {
- description = "Cloud provider region" -> null
- id = "2be8e8c5-2f3b-45c6-b5c2-f41a51053d1b" -> null
- key = "QOVERY_CLOUD_PROVIDER_REGION" -> null
- value = "pl-waw-1" -> null
},
- {
- description = "Container internal Unique ID" -> null
- id = "c7fa92f0-ad2f-4b7c-aae7-89d1ac47b03b" -> null
- key = "QOVERY_CONTAINER_ID" -> null
- value = "85ba8578-1c93-43ed-8643-3914526c4b95" -> null
},
- {
- description = "Container name" -> null
- id = "3c0e1cd9-c61e-407a-a903-b9c6d69a7227" -> null
- key = "QOVERY_CONTAINER_Z85BA8578_NAME" -> null
- value = "app1" -> null
},
- {
- description = "Container tag" -> null
- id = "11f3bb10-3b03-4f3d-bb59-25722bed0667" -> null
- key = "QOVERY_CONTAINER_Z85BA8578_TAG" -> null
- value = "linux" -> null
},
- {
- description = "Deployment ID" -> null
- id = "4016285a-7d47-4710-a5bf-c68d8644b19c" -> null
- key = "QOVERY_DEPLOYMENT_ID" -> null
# (1 unchanged attribute hidden)
},
- {
- description = "Environment internal Unique ID" -> null
- id = "1b5a1e54-7f5c-488e-8e95-6c47d3723695" -> null
- key = "QOVERY_ENVIRONMENT_ID" -> null
- value = "189fcc09-d03f-444c-b6a5-6eeb3944be1e" -> null
},
- {
- description = "Environment name" -> null
- id = "601551a7-e2cf-4f11-a97b-19dfad65ca10" -> null
- key = "QOVERY_CONTAINER_Z85BA8578_ENVIRONMENT_NAME" -> null
- value = "test" -> null
},
- {
- description = "Environment name" -> null
- id = "bd10c2e4-92fb-4c9a-82e2-8efa23a0a4f1" -> null
- key = "QOVERY_ENVIRONMENT_NAME" -> null
- value = "test" -> null
},
- {
- description = "Environment type" -> null
- id = "7dcda189-3ab4-407d-824f-50b94b3db1f4" -> null
- key = "QOVERY_ENVIRONMENT_TYPE" -> null
- value = "DEVELOPMENT" -> null
},
- {
- description = "Internal hostname of the container" -> null
- id = "090c8389-ac76-477b-9be3-80b61e93e5d3" -> null
- key = "QOVERY_CONTAINER_Z85BA8578_HOST_INTERNAL" -> null
- value = "app-z85ba8578-app" -> null
},
- {
- description = "Internal hostname of the container" -> null
- id = "a78d71bf-642b-44b5-a388-d6cbb835819a" -> null
- key = "QOVERY_CONTAINER_ZC45DDA70_HOST_INTERNAL" -> null
- value = "app-zc45dda70-app" -> null
},
- {
- description = "Kubernetes cluster name" -> null
- id = "e5b12655-8b61-44bb-ba77-d3e8e216004c" -> null
- key = "QOVERY_KUBERNETES_CLUSTER_NAME" -> null
- value = "qovery-ze5fe17b2" -> null
},
- {
- description = "Kubernetes namespace name associated with this environment" -> null
- id = "bb81fb5e-a6d6-4c57-8e0f-6edfbc15e455" -> null
- key = "QOVERY_KUBERNETES_NAMESPACE_NAME" -> null
- value = "z189fcc09-test" -> null
},
- {
- description = "Project internal Unique ID" -> null
- id = "62230115-c180-4df1-9b6d-480e3df97c38" -> null
- key = "QOVERY_PROJECT_ID" -> null
- value = "e5894769-84d5-46af-a9d0-1fd3727737c9" -> null
},
] -> (known after apply)
~ deployment_stage_id = "40a5ceb7-69f8-41a2-ac95-cf7d4c1b1e61" -> (known after apply)
~ environment_variables = [
- {
- id = "574fe42d-053f-40b7-9b57-dbbe25df50c3" -> null
- key = "TEST" -> null
- value = "TEST" -> null
},
+ {
+ id = (known after apply)
+ key = "TEST"
+ value = "TEST2"
},
]
+ external_host = (known after apply)
~ icon_uri = "app://qovery-console/container" -> (known after apply)
id = "85ba8578-1c93-43ed-8643-3914526c4b95"
~ internal_host = "app-z85ba8578-app" -> (known after apply)
name = "app1"
# (12 unchanged attributes hidden)
}
# qovery_container.app2 will be updated in-place
~ resource "qovery_container" "app2" {
~ advanced_settings_json = jsonencode({}) -> (known after apply)
~ built_in_environment_variables = [
- {
- description = "Cloud provider region" -> null
- id = "2be8e8c5-2f3b-45c6-b5c2-f41a51053d1b" -> null
- key = "QOVERY_CLOUD_PROVIDER_REGION" -> null
- value = "pl-waw-1" -> null
},
- {
- description = "Container internal Unique ID" -> null
- id = "517f2cc8-2b9a-4c94-9ca7-1f226d92a06e" -> null
- key = "QOVERY_CONTAINER_ID" -> null
- value = "c45dda70-14d6-4a82-9af3-df343ab24196" -> null
},
- {
- description = "Container name" -> null
- id = "f0d8f788-0591-4058-98ea-2d9207a1efff" -> null
- key = "QOVERY_CONTAINER_ZC45DDA70_NAME" -> null
- value = "app2" -> null
},
- {
- description = "Container tag" -> null
- id = "8a548669-7963-4de4-8ef7-34c204a19bb9" -> null
- key = "QOVERY_CONTAINER_ZC45DDA70_TAG" -> null
- value = "linux" -> null
},
- {
- description = "Deployment ID" -> null
- id = "4016285a-7d47-4710-a5bf-c68d8644b19c" -> null
- key = "QOVERY_DEPLOYMENT_ID" -> null
# (1 unchanged attribute hidden)
},
- {
- description = "Environment internal Unique ID" -> null
- id = "1b5a1e54-7f5c-488e-8e95-6c47d3723695" -> null
- key = "QOVERY_ENVIRONMENT_ID" -> null
- value = "189fcc09-d03f-444c-b6a5-6eeb3944be1e" -> null
},
- {
- description = "Environment name" -> null
- id = "152d7921-9d5d-4998-a736-23460b5dfa23" -> null
- key = "QOVERY_CONTAINER_ZC45DDA70_ENVIRONMENT_NAME" -> null
- value = "test" -> null
},
- {
- description = "Environment name" -> null
- id = "bd10c2e4-92fb-4c9a-82e2-8efa23a0a4f1" -> null
- key = "QOVERY_ENVIRONMENT_NAME" -> null
- value = "test" -> null
},
- {
- description = "Environment type" -> null
- id = "7dcda189-3ab4-407d-824f-50b94b3db1f4" -> null
- key = "QOVERY_ENVIRONMENT_TYPE" -> null
- value = "DEVELOPMENT" -> null
},
- {
- description = "Internal hostname of the container" -> null
- id = "090c8389-ac76-477b-9be3-80b61e93e5d3" -> null
- key = "QOVERY_CONTAINER_Z85BA8578_HOST_INTERNAL" -> null
- value = "app-z85ba8578-app" -> null
},
- {
- description = "Internal hostname of the container" -> null
- id = "a78d71bf-642b-44b5-a388-d6cbb835819a" -> null
- key = "QOVERY_CONTAINER_ZC45DDA70_HOST_INTERNAL" -> null
- value = "app-zc45dda70-app" -> null
},
- {
- description = "Kubernetes cluster name" -> null
- id = "e5b12655-8b61-44bb-ba77-d3e8e216004c" -> null
- key = "QOVERY_KUBERNETES_CLUSTER_NAME" -> null
- value = "qovery-ze5fe17b2" -> null
},
- {
- description = "Kubernetes namespace name associated with this environment" -> null
- id = "bb81fb5e-a6d6-4c57-8e0f-6edfbc15e455" -> null
- key = "QOVERY_KUBERNETES_NAMESPACE_NAME" -> null
- value = "z189fcc09-test" -> null
},
- {
- description = "Project internal Unique ID" -> null
- id = "62230115-c180-4df1-9b6d-480e3df97c38" -> null
- key = "QOVERY_PROJECT_ID" -> null
- value = "e5894769-84d5-46af-a9d0-1fd3727737c9" -> null
},
] -> (known after apply)
~ deployment_stage_id = "40a5ceb7-69f8-41a2-ac95-cf7d4c1b1e61" -> (known after apply)
~ environment_variables = [
- {
- id = "72fb42ea-956e-4556-85ba-abdbd6c2e9da" -> null
- key = "APP1_HOST" -> null
- value = "app-z85ba8578-app" -> null
},
+ {
+ id = (known after apply)
+ key = "APP1_HOST"
+ value = (known after apply)
},
]
+ external_host = (known after apply)
~ icon_uri = "app://qovery-console/container" -> (known after apply)
id = "c45dda70-14d6-4a82-9af3-df343ab24196"
~ internal_host = "app-zc45dda70-app" -> (known after apply)
name = "app2"
# (12 unchanged attributes hidden)
}
Plan: 0 to add, 2 to change, 0 to destroy.
Thank you,
We will try this and get back to you.
Regards,
Charles-Edouard
Hello @km1414 ,
I checked with the team and I think this should answer your questions:
Please let me know if this solved your problem.
regarding your example, it looks like this:
resource "qovery_container" "app1" {
environment_id = qovery_environment.test-environment.id
registry_id = qovery_container_registry.my_container_registry.id
name = "app1"
cpu = 100
memory = 64
image_name = "hello-world"
tag = "linux"
healthchecks = {}
environment_variables = [
{
key = "TEST"
value = "TEST"
}
]
lifecycle {
ignore_changes = [built_in_environment_variables]
}
}
and the plan looks like this:
Terraform will perform the following actions:
# qovery_container.app1 will be updated in-place
~ resource "qovery_container" "app1" {
~ advanced_settings_json = jsonencode({}) -> (known after apply)
~ deployment_stage_id = "1e71c972-68c0-4be0-bea7-cb18ba01fba4" -> (known after apply)
~ environment_variables = [
- {
- id = "xxx" -> null
- key = "TEST" -> null
- value = "TEST2" -> null
},
+ {
+ id = (known after apply)
+ key = "TEST"
+ value = "TEST"
},
]
+ external_host = (known after apply)
~ icon_uri = "app://qovery-console/container" -> (known after apply)
id = "xxx"
~ internal_host = "xxx" -> (known after apply)
name = "app1"
# (13 unchanged attributes hidden)
}
Regards,
Charles-Edouard
Thanks you for your answer. We have some experience with “ignore_changes”, but in this case our question is rather about app2. If we make any changes in app1, it also causes changes in app2. Adding “ignore_changes” on built_in_environment_variables helps to see less of them, but app2 is still impacted even it has no actual changes. Can this be fixed somehow? For the reference here’s the plan output with “ignore_changes” added:
Terraform will perform the following actions:
# qovery_container.app1 will be updated in-place
~ resource "qovery_container" "app1" {
~ advanced_settings_json = jsonencode({}) -> (known after apply)
~ deployment_stage_id = "ce153735-dbdc-4e3c-8dd6-0875723b97a6" -> (known after apply)
~ environment_variables = [
- {
- id = "bc1d1e28-f248-409b-a97d-9d471a3e87e5" -> null
- key = "TEST" -> null
- value = "TEST" -> null
},
+ {
+ id = (known after apply)
+ key = "TEST"
+ value = "TEST1"
},
]
+ external_host = (known after apply)
~ icon_uri = "app://qovery-console/container" -> (known after apply)
id = "7b8abeba-40ff-48ea-8748-d4565bb77cff"
~ internal_host = "app-z7b8abeba-app" -> (known after apply)
name = "app1"
# (13 unchanged attributes hidden)
}
# qovery_container.app2 will be updated in-place
~ resource "qovery_container" "app2" {
~ advanced_settings_json = jsonencode({}) -> (known after apply)
~ deployment_stage_id = "ce153735-dbdc-4e3c-8dd6-0875723b97a6" -> (known after apply)
~ environment_variables = [
- {
- id = "7ce7f864-c55b-4a70-b395-8b12018dd2cf" -> null
- key = "APP1_HOST" -> null
- value = "app-z7b8abeba-app" -> null
},
+ {
+ id = (known after apply)
+ key = "APP1_HOST"
+ value = (known after apply)
},
]
+ external_host = (known after apply)
~ icon_uri = "app://qovery-console/container" -> (known after apply)
id = "0d5da4b3-88c3-47bc-8c7e-246ffa90a677"
~ internal_host = "app-z0d5da4b3-app" -> (known after apply)
name = "app2"
# (13 unchanged attributes hidden)
}
Plan: 0 to add, 2 to change, 0 to destroy.
Hello @km1414 ,
We checked with the team and this is a normal behavior. To explain:
value = qovery_container.app1.internal_hostWe created a task in our backlog to improve our Terraform provider. I don’t have an ETA on this yet.
I hope this is helpful,
regards,
Charles-Edouard
Hi @ce_gagnaire,
Yes, it’s normal behaviour when you depend on a value that’s “(known after apply)”. However, internal_host cannot change and Qovery TF provider could be improved to correctly report that. This way, other resources depending on internal_host value will no longer have to be updated and reduce noise in TF planning.