UBI: certificates: refactor patterns for permissions and verification

Closes #2114

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2133



Merged-by: Jason Plum <jplum@gitlab.com>
Approved-by: Clemens Beck <cbeck@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>

Do not cache autodeploy images

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2118



Merged-by: Jason Plum <jplum@gitlab.com>
Approved-by: Dustin Collins <714871-dustinmm80@users.noreply.gitlab.com>
Approved-by: Jason Plum <jplum@gitlab.com>
Co-authored-by: Dmitry Makovey <dmakovey@gitlab.com>

Applying suggestion from Robert M

UBI: centralize all UBI image/version setting to variables

Closes #2124

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2128



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

Remove image path/version  from all `UBI*_IMAGE=` statements.

This is done to centralize to a single point, via `ci_files/variables.yml`

Closes https://gitlab.com/gitlab-org/build/CNG/-/issues/2124

Support internal releases

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2117



Merged-by: Jason Plum <jplum@gitlab.com>
Approved-by: Clemens Beck <cbeck@gitlab.com>
Approved-by: Jason Plum <jplum@gitlab.com>
Reviewed-by: Clemens Beck <cbeck@gitlab.com>
Co-authored-by: Balasankar "Balu" C <balasankar@gitlab.com>

Update dependency Azure/azure-storage-azcopy to v10.27.1

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2105



Merged-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Alec Xu <axu@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

Update dependency mikefarah/yq to v4.44.6

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2137



Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Dustin Collins <714871-dustinmm80@users.noreply.gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

- Updates dependency mikefarah/yq to v4.44.6

Changelog: changed

Update the patterns of the UBI certificates container, in order to
better satisfy `rpm --verify` and resolve permissions challenges that
led to the original issue.

Closes https://gitlab.com/gitlab-org/build/CNG/-/issues/2114

Changelog: changed

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

Update moby/buildkit Docker tag to v0.18.1

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2102



Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Lucas Li <lli@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

- Updates moby/buildkit Docker tag to v0.18.1

Changelog: changed

UBI: Sidekiq: fix layer ordering, consume rails properly

Closes #2115

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2113



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

Update the layer origin for `target` to `gitlab-rails` vs `ubi-micro`

The previous state resulted in a mixed behavior, where "untracked"
packages where present, as ubi-micro was overlayed onto gitlab-rails
instead of simply adding packages onto gitlab-rails. This resulted in
"odd" scanner results, and the problems seen in
https://gitlab.com/gitlab-org/build/CNG/-/issues/2115

UBI: toolbox: fix layer ordering, consume rails properly

Closes #2117 and #2116

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2114



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

Update the layer origin for `target` to `gitlab-rails` vs `ubi-micro`

The previous state resulted in a mixed behavior, where "untracked"
packages where present, as ubi-micro was overlayed onto gitlab-rails
instead of simply adding packages onto gitlab-rails. This resulted in
"odd" scanner results, and the problems seen in

Update dependency kubectl to v1.31.3

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2108



Merged-by: Jason Plum <jplum@gitlab.com>
Approved-by: Lucas Li <lli@gitlab.com>
Approved-by: Jason Plum <jplum@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

Update dependency danger-review to v2.0.0

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2132



Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Clemens Beck <cbeck@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

- Updates dependency danger-review to v2.0.0

Changelog: changed

Update dependency danger-review to v2.0.0

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2131



Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

- Updates danger-review to v2.0.0

Changelog: changed

Enable cosign verify as a gate to pass release

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2091



Merged-by: Jason Plum <jplum@gitlab.com>
Approved-by: Dustin Collins <714871-dustinmm80@users.noreply.gitlab.com>
Approved-by: Jason Plum <jplum@gitlab.com>
Co-authored-by: João Cunha <j.a.cunha@gmail.com>

Update from UBI 9.4 to 9.5

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2095



Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Reviewed-by: Clemens Beck <cbeck@gitlab.com>
Co-authored-by: Clemens Beck <cbeck@gitlab.com>

Update all ubi minimal and micro images from 9.4 to 9.5.

Changelog: changed

Document our usage of cosign from Developer standpoint

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2092



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Clemens Beck <cbeck@gitlab.com>
Approved-by: Evan Read <eread@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Reviewed-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Reviewed-by: Clemens Beck <cbeck@gitlab.com>
Co-authored-by: João Cunha <j.a.cunha@gmail.com>

UBI: base: enforce upgrade packages

Closes #2125

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2115



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

Update dependency gitlab-org/container-registry to v4.14.0-gitlab

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2111



Merged-by: João Pereira <jpereira@gitlab.com>
Approved-by: João Pereira <jpereira@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>

Upgrade to Postgres 16.6 for client libraries

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2109



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

UBI: Workhorse: fix layer ordering, consume base properly

Closes #2117

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2112



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Jason Plum <jplum@gitlab.com>

Update the layer origin for `target` to be `gitlab-base` vs `ubi-micro`

The previous state resulted in a mixed behavior, where "untracked"
packages where present, as `ubi-micro` was overlayed onto `gitlab-base`
instead of simply adding packages onto `gitlab-base`. This resulted in
"odd" scanner results, and the problems seen in
https://gitlab.com/gitlab-org/build/CNG/-/issues/2117

Added --skip-cleanup flag

Closes #2121

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2100



Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Nailia Iskhakova <niskhakova@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Alexandr Ovsyannikov <ovs6868@yandex.ru>

Changelog: added

Bump zoekt version to 0.8.6

See merge request https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/2110



Merged-by: Dmitry Gruzd <dgruzd@gitlab.com>
Approved-by: Dmitry Gruzd <dgruzd@gitlab.com>
Co-authored-by: Ravi Kumar <rkumar@gitlab.com>