Skip to content
Snippets Groups Projects
This project is mirrored from https://gitlab.com/gitlab-org/gitaly.git. Pull mirroring updated .
  1. Dec 20, 2022
    • Patrick Steinhardt's avatar
      git/objectpool: Remove remote configuration after creation · 5dd12b78
      Patrick Steinhardt authored
      When creating object pools we do so by cloning the primary object pool
      member. As git-clone(1) creates remote configuration the gitconfig of
      the pool repository will afterwards contain the `remote.origin` remote.
      We have dropped storing remote configuration in repositories a long time
      ago though and clean up any such config entries during housekeeping.
      
      Explicitly remove the remote after we have created the object pool.
      
      Changelog: fixed
      5dd12b78
    • Patrick Steinhardt's avatar
      objectpool: Stop explicitly removing hooks · 796e9028
      Patrick Steinhardt authored
      When creating repositories Git will install files present in a specific
      templates directory into the newly created repository. This includes
      some stubs like `info/exclude` and `description`, but also a set of
      sample hooks. While these files are tiny on their own, they do add up
      when installed across millions of repositories. We have thus overridden
      `init.templateDir` in bd0f8995 (git: Do not install templates when
      creating repos, 2021-10-28) so that we don't install these templates
      anymore.
      
      One left-over bit from before that time is that we still manually delete
      the `hooks` directory when creating object pools. This is not necessary
      anymore though, so let's drop this logic. Note that we have preexisting
      tests that verify there is indeed no `hooks` directory after creating
      the object pool in `TestCreate/successful`.
      796e9028
    • Patrick Steinhardt's avatar
      Merge branch 'pks-objectpool-atomic-creation' into 'master' · 2bc64b5e
      Patrick Steinhardt authored
      objectpool: Enable atomic creation of object pools
      
      Closes #4385
      
      See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5187
      
      
      
      Merged-by: default avatarPatrick Steinhardt <psteinhardt@gitlab.com>
      Approved-by: default avatarkarthik nayak <knayak@gitlab.com>
      Reviewed-by: default avatarJustin Tobler <jtobler@gitlab.com>
      Reviewed-by: default avatarkarthik nayak <knayak@gitlab.com>
      2bc64b5e
    • Toon Claes's avatar
      Merge branch 'toon-readobject-catfile' into 'master' · 37b4c149
      Toon Claes authored
      localrepo: Use catfile cache in ReadObject
      
      See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5158
      
      
      
      Merged-by: default avatarToon Claes <toon@gitlab.com>
      Approved-by: default avatarWill Chandler <wchandler@gitlab.com>
      37b4c149
    • Patrick Steinhardt's avatar
      objectpool: Enable atomic creation of object pools · e07e5a24
      Patrick Steinhardt authored
      Creating repositories can be a complex task that involves multiple
      steps. We must ensure though that if any of the steps fails that we
      don't leave behind any cruft on disk in the form of a partially
      initialized repository. This is why we have in the past refactored our
      repository-creating RPCs to use a helper that knows to do this in
      multiple steps:
      
          1. The repository gets created in a temporary directory.
      
          2. The temporary repository gets seeded with the data we want to put
             into it.
      
          3. We optionally vote on the resulting repository so that we can be
             sure that any replicas have created the same repository.
      
          4. When we see that everything is fine we move the final repository
             into the target location.
      
      This ensures that we only ever see either fully-initialized repositories
      or no repository at all in the target location after a repo-creating RPC
      finishes.
      
      Object pools don't yet use that machinery though. But there is no other
      reason than object pools frequently being an afterthought in Gitaly. So
      let's use the same helper function to atomically create object pools.
      
      Note that while the helper also supports voting onthe result we don't
      yet wire up that functionality: transactions for `CreateObjectPool()`
      are not enabled in Praefect. We will eventually do this in another
      iteration.
      
      Changelog: fixed
      e07e5a24
    • Patrick Steinhardt's avatar
      git/objectpool: Resolve object pool paths when creating the structure · 73493c52
      Patrick Steinhardt authored
      While we already try to resolve the storage path right now and thus fail
      with an error in case the storage is not known, we don't do this for the
      object pool path yet. While we check whether the object pool path is a
      valid path at a later point anyway, we will require the path before
      doing that check in a subsequent commit. As a consequence, the error
      returned would fail in one of our tests.
      
      Refactor the code to ask for the repository path early on so that we
      don't need to change tests when introducing new logic. The new behaviour
      could be claimed to be better than before as we now return a proper
      error in case the path is misformed. But the previous behaviour was just
      fine, too.
      73493c52
    • Patrick Steinhardt's avatar
      objectpool: Tighten argument checks when creating object pools · c797df49
      Patrick Steinhardt authored
      We don't thoroughly verify arguments passed to the `CreateObjectPool()`
      RPC call. As a result, in some scenarios we fail with not-well-defined
      semantics while already busy creating the repository. This creates
      problems when we refactor the code to create repositories atomically as
      the exact way how we fail would be different.
      
      Add more preliminary checks to verify that arguments are what we expect
      them to look like.
      c797df49
    • Patrick Steinhardt's avatar
      repoutil: Add option to skip initializing repository in Create · 56a25656
      Patrick Steinhardt authored
      The `Create()` helper will make sure to call the repository-seeding
      function with an already initialized Git repository so that we can skip
      the step of having to call git-init(1) at all the callsites. While this
      is generally useful, some of the users of `Create()` need to create the
      target repository by calling git-clone(1), which isn't happy in case the
      target directory exists and is not an empty directory. These callers
      thus manually `os.RemoveAll()` the target directory right now before
      doing the clone.
      
      Provide a new option `WithSkipInit()` that allows callers to skip the
      call to git-init(1). Convert callsites to use the new option.
      56a25656
    • Patrick Steinhardt's avatar
      repoutil: Refactor assembly of options to be more extensible · e16795dd
      Patrick Steinhardt authored
      All the options that can be passed to `Create()` can be represented as
      simple `git.Option`s, which is why we a `CreateOption` is defined as
      appending to an array of `git.Option`s. We're about to add another
      option though that doesn't fit into this model.
      
      Refactor the code to introduce a `createConfig` structure that for now
      only has the array of Git options.
      e16795dd
    • Patrick Steinhardt's avatar
      repoutil: Add tests for `Create()` options · 761379bb
      Patrick Steinhardt authored
      We have no tests that verify the behaviour of any of the existing
      options for the `Create()` functions. Add some to plug this test gap.
      761379bb
    • Patrick Steinhardt's avatar
      repository: Move function to transactionally create repositories · 4642abcf
      Patrick Steinhardt authored
      We're about to add another caller for the `createRepository()` helper
      functions so that we can also transactionally and atomically create
      object pools. Let's thus move the function into its own package
      `repoutil`.
      
      Note that I'm not particularly happy with the resulting package
      structure. But this is service-level functionality, which means that
      moving it into `internal/git` is not an option. So let's not dwell too
      long over it.
      4642abcf
    • Patrick Steinhardt's avatar
      repository: Convert `createRepository()` to not be a receiver function · ce49fb31
      Patrick Steinhardt authored
      We're about to add another user for `createRepository()` that is outside
      of the repository service. As a first step, convert the function to not
      be a receiver function anymore so that we can move it into a different
      package.
      ce49fb31
    • Patrick Steinhardt's avatar
      Merge branch 'pks-helper-errorf-conversion-pt3' into 'master' · 39dba873
      Patrick Steinhardt authored
      helper: Convert `ErrNotFoundf()` and `ErrInvalidArgumentf()`
      
      See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5194
      
      
      
      Merged-by: default avatarPatrick Steinhardt <psteinhardt@gitlab.com>
      Approved-by: default avatarJames Fargher <proglottis@gmail.com>
      39dba873
  2. Dec 19, 2022
  3. Dec 16, 2022
  4. Dec 15, 2022