gitlabci should ignore !reference services

[rarkins]

Discussed in #33056

^{Originally posted by Wurstnase December 12, 2024}

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

GitLab (renovate/renovate:39@sha256:68af0e4f4375dcedda551b64d6270b79090edad5fdb23c939001bd657c3c78ab)

Please tell us more about your question or problem

Renovate try to parse an image where no image is.

When you have something like this in a gitlab-ci.yml:

my-job:
  services: !reference [.postgres, services]
  script:
    - echo foo

renovate try to find an image .postgres and will fail. It becomes worse, when the bot is behind a proxy and docker.io is not allowed.

Please check this minimal: minimal

Logs (if relevant)

The runner is in a private network and most internet traffic is forbidden.

Most important part is:

"dockerRepository": "library/services"
...
"dockerRepository": "library/.postgres"

Error without a proxy

DEBUG: HEAD https://index.docker.io/v2/library/services/manifests/latest = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=401 retryCount=0, duration=94) (repository=test-repo)
DEBUG: Unauthorized docker lookup (repository=test-repo)
       "registryHost": "https://index.docker.io",
       "dockerRepository": "library/services"
DEBUG: Response code 401 (Unauthorized) (repository=test-repo)
       "err": {
         "name": "HTTPError",
         "code": "ERR_NON_2XX_3XX_RESPONSE",
         "timings": {
           "start": 1733953012628,
           "socket": 1733953012629,
           "lookup": 1733953012629,
           "connect": 1733953012629,
           "secureConnect": 1733953012629,
           "upload": 1733953012629,
           "response": 1733953012722,
           "end": 1733953012722,
           "phases": {
             "wait": 1,
             "dns": 0,
             "tcp": 0,
             "tls": 0,
             "request": 0,
             "firstByte": 93,
             "download": 0,
             "total": 94
           }
         },
         "message": "Response code 401 (Unauthorized)",
         "stack": "HTTPError: Response code 401 (Unauthorized)\n    at Request.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at processTicksAndRejections (node:internal/process/task_queues:105:5)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "authorization": "***********",
             "accept": "application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/library/services/manifests/latest",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "HEAD",
           "http2": false
         },
         "response": {
           "statusCode": 401,
           "statusMessage": "Unauthorized",
           "body": "",
           "headers": {
             "content-type": "application/json",
             "docker-distribution-api-version": "registry/2.0",
             "www-authenticate": "Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/services:pull\",error=\"insufficient_scope\"",
             "date": "Wed, 11 Dec 2024 21:36:52 GMT",
             "content-length": "159",
             "strict-transport-security": "max-age=31536000",
             "docker-ratelimit-source": "81.200.196.232"
           },
           "httpVersion": "1.1",
           "retryCount": 0
         }
       }
DEBUG: Could not determine new digest for update. (repository=test-repo)
       "packageName": "services",
       "currentValue": undefined,
       "datasource": "docker",
       "newValue": undefined,
       "bucket": undefined
DEBUG: HEAD https://index.docker.io/v2/library/.postgres/manifests/latest = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=307) (repository=test-repo)
DEBUG: Docker Manifest is unknown (repository=test-repo)
       "err": {
         "name": "HTTPError",
         "code": "ERR_NON_2XX_3XX_RESPONSE",
         "timings": {
           "start": 1733953012633,
           "socket": 1733953012634,
           "lookup": 1733953012634,
           "connect": 1733953012742,
           "secureConnect": 1733953012840,
           "upload": 1733953012840,
           "response": 1733953012940,
           "end": 1733953012940,
           "phases": {
             "wait": 1,
             "dns": 0,
             "tcp": 108,
             "tls": 98,
             "request": 0,
             "firstByte": 100,
             "download": 0,
             "total": 307
           }
         },
         "message": "Response code 404 (Not Found)",
         "stack": "HTTPError: Response code 404 (Not Found)\n    at Request.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at processTicksAndRejections (node:internal/process/task_queues:105:5)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "authorization": "***********",
             "accept": "application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/library/.postgres/manifests/latest",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "HEAD",
           "http2": false
         },
         "response": {
           "statusCode": 404,
           "statusMessage": "Not Found",
           "body": "",
           "headers": {
             "content-type": "text/plain; charset=utf-8",
             "docker-distribution-api-version": "registry/2.0",
             "x-content-type-options": "nosniff",
             "date": "Wed, 11 Dec 2024 21:36:52 GMT",
             "content-length": "19",
             "strict-transport-security": "max-age=31536000",
             "docker-ratelimit-source": "81.200.196.234"
           },
           "httpVersion": "1.1",
           "retryCount": 0
         }
       },
       "registryHost": "https://index.docker.io",
       "dockerRepository": "library/.postgres",
       "tag": "latest"
DEBUG: Could not determine new digest for update. (repository=test-repo)
       "packageName": ".postgres",
       "currentValue": undefined,
       "datasource": "docker",
       "newValue": undefined,
       "bucket": undefined

Behind a proxy it was very hard to find. Maybe there is a chance to improve error messages behind a HTTP proxy, which normally has no https.

Logs behind the proxy

DEBUG: GET https://index.docker.io/v2/ = (code=EPROTO, statusCode=-1 retryCount=0, duration=74) (repository=iot-platform/gitlab-ci-cd/renovate/test-provisioner)
 WARN: Host error (repository=test-repo)
       "hostType": undefined,
       "packageName": undefined,
       "err": {
         "name": "RequestError",
         "code": "EPROTO",
         "timings": {
           "start": 1733953091988,
           "socket": 1733953092057,
           "lookup": 1733953092057,
           "connect": 1733953092057,
           "upload": 1733953092060,
           "error": 1733953092062,
           "phases": {"wait": 69, "dns": 0, "tcp": 0, "request": 3, "total": 74}
         },
         "message": "write EPROTO C00C199B9A7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:\n",
         "stack": "RequestError: write EPROTO C00C199B9A7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:\n\n    at ClientRequest.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/got/dist/source/core/index.js:970:111)\n    at Object.onceWrapper (node:events:633:26)\n    at ClientRequest.emit (node:events:530:35)\n    at ClientRequest.emit (node:domain:489:12)\n    at ClientRequest.origin.emit (/usr/local/renovate/node_modules/.pnpm/@[email protected]/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)\n    at emitErrorEvent (node:_http_client:103:11)\n    at TLSSocket.socketErrorListener (node:_http_client:506:5)\n    at TLSSocket.emit (node:events:530:35)\n    at TLSSocket.emit (node:domain:489:12)\n    at emitErrorNT (node:internal/streams/destroy:170:8)\n    at emitErrorCloseNT (node:internal/streams/destroy:129:3)\n    at processTicksAndRejections (node:internal/process/task_queues:90:21)\n    at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:87:19)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://index.docker.io/v2/",
           "hostType": "docker",
           "username": "",
           "password": "",
           "method": "GET",
           "http2": false
         }
       }

everything in the !reference should be ignored. It is a special construct for gitlab.

docs.gitlab.com/ee/ci/yaml/yaml_optimization.html#reference-tags

[viceice]

this seems to be a regerssion, because some time ago we simply removed those references.

[oxdev03]

Could it happen since the switch to the yaml package?

The parsed yaml doesn't include the reference tag. So the rawDocument probably needs to be exposed in parseYaml to check whether the tag exists.