Rails tests configure Praefect in front of the tests that exercise
the Rugged direct git access code. As Praefect is now deriving the
filesystem IDs from the names of the virtual storages, the filesystem
id checks fail and thus the test fail. This is not a problem in practice
as one wouldn't use rugged in a real-world setup with Praefect. This
commit worksaround the tests by returning the filesystem ID from the
Gitaly node if a virtual storage has only one Gitaly node configured.
This matches the setup the tests use and thus pass them. The workaround
and the filesystem ID code can be removed in 15.0 once the rugged patches
and NFS support are dropped.

Gitaly storages contain a UUID filesystem ID that is generated by
the Gitaly for each of its storages. The ID is used to determine
which storages can be accessed by Rails directly when rugged patches
are enabled and to see whether two different storages point to the same
directory when doing repository moves.

When repository moves are performed, the worker first checks whether the
repository's destination and source storage are the same. If they are, the
move is not performed. The check is performed by comparing the filesystem
IDs of the storages'. As Praefect is currently routing the server info RPC
to a random Gitaly node, the filesystem ID can differ between calls as each
of the Gitalys have their own ID. This causes the repository moving worker
to occasionally delete repositories from the virtual storage as it receives
two different IDs on sequential calls.

The filesystem ID can identify cases when two storages refer to the same
directory on a Gitaly node as the id is stored in a file in the storage.
This is not really possible with Praefect. The storage's are only identified
by the virtual storage's name. If the name changes, we can't really correlate
the ID between the different names as Praefect would consider them different
storages. Praefect also supports multiple virtual storages so it's not possible
to generate a single ID and use it for all of the virtual storages. Given this,
the approach taken here is to derive a stable filesystem ID from the virtual
storage's name. This guarantees calls to a given virtual storage always return
the same filesystem ID.

Configuring two storages that point to the same filesystem should be considered
an invalid configuration anyway. Historically, there's been cases when that has
been done for plain Gitalys. This is not done for Praefect and wouldn't work as
Praefect wouldn't find the repositories with an alternative virtual storage name.
With that in mind, we don't have to consider the case where two virtual storages
of different names point to the same backing Gitaly storages.

The use cases for the filesystem ID seem to be limited and we may be able to
remove it in the future once the rugged patches are removed.

Changelog: fixed

[ci skip]

[ci skip]

Backport improved replication logic (v14.1)

See merge request gitlab-org/gitaly!3823

When finalizing a transaction, we always schedule replication jobs in
case the primary has returned an error. Given that there are many RPCs
which are expected to return errors in a controlled way, e.g. if a
commit is missing, this causes us to create replication in many contexts
where it's not necessary at all.

Thinking about the issue, what we really care for is not whether an RPC
failed or not. It's that primary and secondary nodes behaved the same.
If both primary and secondaries succeeded, we're good. But if both
failed with the same error, then we're good to as long as all
transactions have been committed: quorum was reached on all votes and
nodes failed in the same way, so we can assume that nodes did indeed
perform the same changes.

This commit thus relaxes the error condition to not schedule replication
jobs anymore in case the primary failed, but to only schedule
replication jobs to any node which has a different error than the
primary. This has both the advantage that we only need to selectively
schedule jobs for disagreeing nodes instead of targeting all
secondaries and it avoids scheduling jobs in many cases where we do hit
errors.

Changelog: performance
(cherry picked from commit 73839029)

[14.1] Only activate Git pack-objects hook if cache is enabled

See merge request gitlab-org/gitaly!3814

[ci skip]

[ci skip]

In https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3301, we
dropped the `upload_pack_gitaly_hooks` feature flag because it was
confusing to have to enable this feature flag on top of the pack objects
cache setting in `config.toml`.

However, we have found that spawning the gitaly-hooks environment can
add significant CPU load due to overhead from transferring data over
gRPC and spawning gitaly-hooks processes.

We now only enable this hook if the pack objects cache is enabled.

Relates to https://gitlab.com/gitlab-org/gitaly/-/issues/3754

Changelog: performance

[ci skip]

[ci skip]

[ci skip]

[ci skip]

Allow parsing of long git commit headers

See merge request gitlab-org/security/gitaly!40

[ci skip]

[ci skip]

`bufio.Scanner` can only handle lines of a certain length. So to enable
parsing of very large git commit headers the parser implementation was
switched to use `bufio.Reader.ReadString`.

Changelog: security

[ci skip]

[ci skip]

[ci skip]

[ci skip]

Update ffi gem to 1.15.3

See merge request gitlab-org/gitaly!3664

Set default Prometheus buckets for Gitalys RPC instrumentation

Closes #3431

See merge request gitlab-org/gitaly!3669

Gitaly doesn't set default buckets for RPC latency instrumentation
which leads to the instrumentation being disabled by default. This
commit adds default buckets to the configuration which is used if
the buckets are not explicilty configured.

Changelog: changed

Add StreamRPC library code

See merge request gitlab-org/gitaly!3601

Support lazy failovers in `praefect dataloss`

See merge request gitlab-org/gitaly!3549

Update read-only repository count metric to account for lazy failover

See merge request gitlab-org/gitaly!3548

Remove feature gitaly_fetch_internal_remote_errors

Closes #3588

See merge request gitlab-org/gitaly!3647

Since FetchInternalRemote has been inlined into ReplicateRepository we
no longer need to make this RPC errors more verbose.

Fix various static lint issues

See merge request gitlab-org/gitaly!3666

Perform failovers lazily

Closes #3207

See merge request gitlab-org/gitaly!3543

Changelog: other

It is not common to use snake case names for packages
and package aliases in Go.
The change renames aliases to a preferred single word name.
We also use 'gitaly' prefix for the project-defined packages
that clashes with standard or 3-rd party package names.

Some functions, types, fields and other variables are not
used. There is no reason to keep them and support.
Some of them became redundant starting from declaration
and some after the code changes.

The done variable is never assigned any value, so
the condition always evaluates into true.

If struct has a list of fields declared the values for
those fields could be assigned during struct instance creation
by providing values in the same order as the fields are declared
or in any order if field names are used to assign the values.
The preferred way is to use a field name assigment as it is less
error prone if you define a new field in the middle of the struct
and as well more readable as you see the list of the initialized
fields.

The current read-only repository count metric describes unavailable
repositories rather than read-only repositories. We have to keep the
name for backwards compatibility as some alerting rules and dashboards
depend on it. To make it possible to migrate to a more accurate metric
later, this commit adds another metric on the side with more accurate
name and description.