Class: Toys::Utils::GitCache

Inherits:
Object
  • Object
show all
Defined in:
lib/toys/utils/git_cache.rb,
lib/toys/utils/git_cache.rb,
lib/toys/utils/git_cache.rb,
lib/toys/utils/git_cache.rb,
lib/toys/utils/git_cache.rb

Overview

This object provides cached access to remote git data. Given a remote repository, a path, and a commit, it makes the files available in the local filesystem. Access is cached, so repeated requests for the same commit and path in the same repo do not hit the remote repository again.

Defined Under Namespace

Classes: Error, RefInfo, RepoInfo, SourceInfo

Constant Summary collapse

VERSION =

Version of the git_cache gem

Returns:

  • (String) 
"0.1.2"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_dir: nil) ⇒ GitCache

Access a git cache.

Parameters:

  • cache_dir (String) (defaults to: nil) —

    The path to the cache directory. Defaults to a specific directory in the user's XDG cache. Cache data is stored in a subdirectory named for the cache format version, so that clients using incompatible formats can share a cache directory safely.



23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/toys/utils/git_cache.rb', line 23

def initialize(cache_dir: nil)
  require "digest"
  require "fileutils"
  require "json"
  require "securerandom"
  require "toys/utils/exec"
  @using_default_cache_dir = cache_dir.nil?
  @cache_dir = ::File.expand_path(cache_dir || default_cache_dir)
  @data_dir = ::File.join(@cache_dir, FORMAT_VERSION)
  @exec = ::Toys::Utils::Exec.new(out: :capture, err: :capture)
end

Instance Attribute Details

#cache_dir ⇒ String (readonly)

The cache directory.

Returns:

  • (String) 


40
41
42
 
# File 'lib/toys/utils/git_cache.rb', line 40

def cache_dir
  @cache_dir
end

Instance Method Details

#get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil) ⇒ String Also known as: find

Get the given git-based files from the git cache, loading from the remote repo if necessary.

The resulting files are either copied into a directory you provide in the :into parameter, or populated into a shared source directory if you omit the :into parameter. In the latter case, it is important that you do not modify the returned files or directories, nor add or remove any files from the directories returned, to avoid confusing callers that could be given the same directory. If you need to make any modifications to the returned files, use :into to provide your own private directory.

Parameters:

  • remote (String) —

    The URL of the git repo. Required.

  • path (String) (defaults to: nil) —

    The path to the file or directory within the repo. Optional. Defaults to the entire repo.

  • commit (String) (defaults to: nil) —

    The commit reference, which may be a SHA or any git ref such as a branch or tag. Optional. Defaults to HEAD.

  • into (String) (defaults to: nil) —

    If provided, copies the specified files into the given directory path. If omitted or nil, populates and returns a shared source file or directory.

  • update (boolean, Integer) (defaults to: false) —

    Whether to update non-SHA commit references if they were previously loaded. This is useful, for example, if the commit is HEAD or a branch name. Pass true or false to specify whether to update, or an integer to update if last update was done at least that many seconds ago. Default is false.

  • timestamp (Integer, nil) (defaults to: nil) —

    The timestamp for recording the access time and determining whether a resource is stale. Normally, you should leave this out and it will default to the current time.

Returns:

  • (String) —

    The full path to the cached files. The returned path will correspond to the path given. For example, if you provide the path Gemfile representing a single file in the repository, the returned path will point directly to the cached copy of that file.



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/toys/utils/git_cache.rb', line 78

def get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil)
  path = ::Toys::Utils::GitCache.normalize_path(path)
  commit ||= "HEAD"
  timestamp ||= ::Time.now.to_i
  name = ::Toys::Utils::GitCache.remote_dir_name(remote)
  dir = repo_base_dir_for(name)
  lock_repo(name, remote, timestamp, create: true) do |repo_state|
    ensure_repo(dir, remote)
    sha = ensure_commit(dir, commit, repo_state, update)
    if into
      copy_files(dir, sha, path, repo_state, into)
    else
      ensure_source(dir, sha, path, repo_state)
    end
  end
end

#remotes ⇒ Array<String>

Returns an array of the known remote names.

Returns:

  • (Array<String>) 


101
102
103
104
105
106
107
108
109
110
111
112
 
# File 'lib/toys/utils/git_cache.rb', line 101

def remotes
  result = []
  repos_dir = ::File.join(@data_dir, REPOS_DIR_NAME)
  return result unless ::File.directory?(repos_dir)
  ::Dir.children(repos_dir).each do |name|
    next if name.start_with?(".")
    next unless ::File.file?(::File.join(repos_dir, name, STATE_FILE_NAME))
    remote = lock_repo(name, &:remote)
    result << remote if remote
  end
  result.sort
end

#remove_refs(remote, refs: nil) ⇒ Array<RefInfo>?

Remove records of the given refs (i.e. branches, tags, or HEAD) from the given repository's cache. The next time those refs are requested, they will be pulled from the remote repo.

If you provide the refs: argument, only those refs are removed. Otherwise, all refs are removed.

Parameters:

  • remote (String) —

    The repository

  • refs (Array<String>) (defaults to: nil) —

    The refs to remove. Optional.

Returns:

  • (Array<RefInfo>, nil) —

    The refs actually forgotten, or nil if the given repo is not in the cache.



176
177
178
179
180
181
182
183
184
185
186
187
188
 
# File 'lib/toys/utils/git_cache.rb', line 176

def remove_refs(remote, refs: nil)
  name = ::Toys::Utils::GitCache.remote_dir_name(remote)
  return nil unless ::File.directory?(repo_base_dir_for(name))
  lock_repo(name, remote) do |repo_state|
    results = []
    refs = repo_state.refs if refs.nil? || refs == :all
    Array(refs).each do |ref|
      ref_data = repo_state.delete_ref!(ref)
      results << RefInfo.new(ref, ref_data) if ref_data
    end
    results.sort
  end
end

#remove_repos(remotes) ⇒ Array<String>

Removes caches for the given repos, or all repos if specified.

Removes all cache information for the specified repositories, including local clones and shared source directories. The next time these repositories are requested, they will be reloaded from the remote repository from scratch.

This waits for any in-progress #get calls for these repos to finish. However, be careful not to remove repos whose shared sources are currently in use by other Toys::Utils::GitCache clients.

Parameters:

  • remotes (Array<String>, :all, nil) —

    The remotes to remove. If set to :all or nil, removes all repos.

Returns:

  • (Array<String>) —

    The remotes actually removed.



146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/toys/utils/git_cache.rb', line 146

def remove_repos(remotes)
  remotes = self.remotes if remotes.nil? || remotes == :all
  Array(remotes).map do |remote|
    name = ::Toys::Utils::GitCache.remote_dir_name(remote)
    dir = repo_base_dir_for(name)
    next unless ::File.directory?(dir)
    # Take the lock so we wait for any in-flight operation on this repo.
    # The lock file lives outside the directory being removed, so it keeps
    # excluding later clients after the removal.
    flock_repo(name) do
      next unless ::File.directory?(dir)
      remove_dir(dir)
      remote
    end
  end.compact.sort
end

#remove_sources(remote, commits: nil) ⇒ Array<SourceInfo>?

Removes shared sources for the given cache. The next time a client requests them, the removed sources will be recopied from the repo.

If you provide the commits: argument, only sources associated with those commits are removed. Otherwise, all sources are removed.

Be careful not to remove sources that are currently in use by other Toys::Utils::GitCache clients.

Parameters:

  • remote (String) —

    The repository

  • commits (Array<String>) (defaults to: nil) —

    Remove only the sources for the given commits. Optional.

Returns:

  • (Array<SourceInfo>, nil) —

    The sources actually removed, or nil if the given repo is not in the cache.



206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/toys/utils/git_cache.rb', line 206

def remove_sources(remote, commits: nil)
  name = ::Toys::Utils::GitCache.remote_dir_name(remote)
  dir = repo_base_dir_for(name)
  return nil unless ::File.directory?(dir)
  lock_repo(name, remote) do |repo_state|
    results = []
    commits = nil if commits == :all
    shas = Array(commits).map { |ref| repo_state.lookup_ref(ref) }.compact.uniq if commits
    repo_state.find_sources(shas: shas).each do |(sha, path)|
      data = repo_state.delete_source!(sha, path)
      results << SourceInfo.new(dir, sha, path, data)
    end
    results.map(&:sha).uniq.each do |sha|
      unless repo_state.source_exists?(sha)
        remove_dir(::File.join(dir, sha))
      end
    end
    results.sort
  end
end

#repo_info(remote) ⇒ RepoInfo?

Returns a RepoInfo describing the cache for the given remote, or nil if the given remote has never been cached.

Parameters:

  • remote (String) —

    Remote name for a repo

Returns:



121
122
123
124
125
126
127
128
 
# File 'lib/toys/utils/git_cache.rb', line 121

def repo_info(remote)
  name = ::Toys::Utils::GitCache.remote_dir_name(remote)
  dir = repo_base_dir_for(name)
  return nil unless ::File.directory?(dir)
  lock_repo(name, remote) do |repo_state|
    RepoInfo.new(dir, repo_state.data)
  end
end