Class: Toys::Utils::GitCache

Inherits:
Object
  • Object
show all
Defined in:
lib/toys/utils/git_cache.rb

Overview

This object provides cached access to remote git data. Given a remote repository, a path, and a commit, it makes the files available in the local filesystem. Access is cached, so repeated requests for the same commit and path in the same repo do not hit the remote repository again.

This class is used by the Loader to load tools from git. Tools can also use the :git_cache mixin for direct access to this class.

Defined Under Namespace

Classes: Error, RefInfo, RepoInfo, SourceInfo

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_dir: nil) ⇒ GitCache

Access a git cache.

Parameters:

  • cache_dir (String) (defaults to: nil) —

    The path to the cache directory. Defaults to a specific directory in the user's XDG cache.



314
315
316
317
318
319
320
321
322
 
# File 'lib/toys/utils/git_cache.rb', line 314

def initialize(cache_dir: nil)
  require "digest"
  require "fileutils"
  require "json"
  require "toys/compat"
  require "toys/utils/exec"
  @cache_dir = ::File.expand_path(cache_dir || default_cache_dir)
  @exec = Utils::Exec.new(out: :capture, err: :capture)
end

Instance Attribute Details

#cache_dir ⇒ String (readonly)

The cache directory.

Returns:

  • (String) 


329
330
331
 
# File 'lib/toys/utils/git_cache.rb', line 329

def cache_dir
  @cache_dir
end

Class Method Details

.sources_writable? ⇒ boolean

Returns whether shared source files are writable by default. Normally, shared sources are made read-only to protect them from being modified accidentally since multiple clients may be accessing them. However, you can disable this feature by setting the environment variable TOYS_GIT_CACHE_WRITABLE to any non-empty value. This can be useful in environments that want to clean up temporary directories and are being hindered by read-only files.

Returns:

  • (boolean) 


304
305
306
 
# File 'lib/toys/utils/git_cache.rb', line 304

def self.sources_writable?
  !::ENV["TOYS_GIT_CACHE_WRITABLE"].to_s.empty?
end

Instance Method Details

#get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil) ⇒ String Also known as: find

Get the given git-based files from the git cache, loading from the remote repo if necessary.

The resulting files are either copied into a directory you provide in the :into parameter, or populated into a shared source directory if you omit the :into parameter. In the latter case, it is important that you do not modify the returned files or directories, nor add or remove any files from the directories returned, to avoid confusing callers that could be given the same directory. If you need to make any modifications to the returned files, use :into to provide your own private directory.

Parameters:

  • remote (String) —

    The URL of the git repo. Required.

  • path (String) (defaults to: nil) —

    The path to the file or directory within the repo. Optional. Defaults to the entire repo.

  • commit (String) (defaults to: nil) —

    The commit reference, which may be a SHA or any git ref such as a branch or tag. Optional. Defaults to HEAD.

  • into (String) (defaults to: nil) —

    If provided, copies the specified files into the given directory path. If omitted or nil, populates and returns a shared source file or directory.

  • update (Boolean, Integer) (defaults to: false) —

    Whether to update non-SHA commit references if they were previously loaded. This is useful, for example, if the commit is HEAD or a branch name. Pass true or false to specify whether to update, or an integer to update if last update was done at least that many seconds ago. Default is false.

  • timestamp (Integer, nil) (defaults to: nil) —

    The timestamp for recording the access time and determining whether a resource is stale. Normally, you should leave this out and it will default to the current time.

Returns:

  • (String) —

    The full path to the cached files. The returned path will correspond to the path given. For example, if you provide the path Gemfile representing a single file in the repository, the returned path will point directly to the cached copy of that file.



367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
 
# File 'lib/toys/utils/git_cache.rb', line 367

def get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil)
  path = GitCache.normalize_path(path)
  commit ||= "HEAD"
  timestamp ||= ::Time.now.to_i
  dir = ensure_repo_base_dir(remote)
  lock_repo(dir, remote, timestamp) do |repo_lock|
    ensure_repo(dir, remote)
    sha = ensure_commit(dir, commit, repo_lock, update)
    if into
      copy_files(dir, sha, path, repo_lock, into)
    else
      ensure_source(dir, sha, path, repo_lock)
    end
  end
end

#remotes ⇒ Array<String>

Returns an array of the known remote names.

Returns:

  • (Array<String>) 


389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/toys/utils/git_cache.rb', line 389

def remotes
  result = []
  return result unless ::File.directory?(cache_dir)
  ::Dir.entries(cache_dir).each do |child|
    next if child.start_with?(".")
    dir = ::File.join(cache_dir, child)
    if ::File.file?(::File.join(dir, LOCK_FILE_NAME))
      remote = lock_repo(dir, &:remote)
      result << remote if remote
    end
  end
  result.sort
end

#remove_refs(remote, refs: nil) ⇒ Array<RefInfo>?

Remove records of the given refs (i.e. branches, tags, or HEAD) from the given repository's cache. The next time those refs are requested, they will be pulled from the remote repo.

If you provide the refs: argument, only those refs are removed. Otherwise, all refs are removed.

Parameters:

  • remote (String) —

    The repository

  • refs (Array<String>) (defaults to: nil) —

    The refs to remove. Optional.

Returns:

  • (Array<RefInfo>, nil) —

    The refs actually forgotten, or nil if the given repo is not in the cache.



458
459
460
461
462
463
464
465
466
467
468
469
470
 
# File 'lib/toys/utils/git_cache.rb', line 458

def remove_refs(remote, refs: nil)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  results = []
  lock_repo(dir, remote) do |repo_lock|
    refs = repo_lock.refs if refs.nil? || refs == :all
    Array(refs).each do |ref|
      ref_data = repo_lock.delete_ref!(ref)
      results << RefInfo.new(ref, ref_data) if ref_data
    end
  end
  results.sort
end

#remove_repos(remotes) ⇒ Array<String>

Removes caches for the given repos, or all repos if specified.

Removes all cache information for the specified repositories, including local clones and shared source directories. The next time these repositories are requested, they will be reloaded from the remote repository from scratch.

Be careful not to remove repos that are currently in use by other GitCache clients.

Parameters:

  • remotes (Array<String>, :all, nil) —

    The remotes to remove. If set to :all or nil, removes all repos.

Returns:

  • (Array<String>) —

    The remotes actually removed.



433
434
435
436
437
438
439
440
441
442
443
 
# File 'lib/toys/utils/git_cache.rb', line 433

def remove_repos(remotes)
  remotes = self.remotes if remotes.nil? || remotes == :all
  Array(remotes).map do |remote|
    dir = repo_base_dir_for(remote)
    if ::File.directory?(dir)
      ::FileUtils.chmod_R("u+w", dir, force: true)
      ::FileUtils.rm_rf(dir)
      remote
    end
  end.compact.sort
end

#remove_sources(remote, commits: nil) ⇒ Array<SourceInfo>?

Removes shared sources for the given cache. The next time a client requests them, the removed sources will be recopied from the repo.

If you provide the commits: argument, only sources associated with those commits are removed. Otherwise, all sources are removed.

Be careful not to remove sources that are currently in use by other GitCache clients.

Parameters:

  • remote (String) —

    The repository

  • commits (Array<String>) (defaults to: nil) —

    Remove only the sources for the given commits. Optional.

Returns:

  • (Array<SourceInfo>, nil) —

    The sources actually removed, or nil if the given repo is not in the cache.



488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
 
# File 'lib/toys/utils/git_cache.rb', line 488

def remove_sources(remote, commits: nil)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  results = []
  lock_repo(dir, remote) do |repo_lock|
    commits = nil if commits == :all
    shas = Array(commits).map { |ref| repo_lock.lookup_ref(ref) }.compact.uniq if commits
    repo_lock.find_sources(shas: shas).each do |(sha, path)|
      data = repo_lock.delete_source!(sha, path)
      results << SourceInfo.new(dir, sha, path, data)
    end
    results.map(&:sha).uniq.each do |sha|
      unless repo_lock.source_exists?(sha)
        sha_dir = ::File.join(dir, sha)
        ::FileUtils.chmod_R("u+w", sha_dir, force: true)
        ::FileUtils.rm_rf(sha_dir)
      end
    end
  end
  results.sort
end

#repo_info(remote) ⇒ RepoInfo?

Returns a RepoInfo describing the cache for the given remote, or nil if the given remote has never been cached.

Parameters:

  • remote (String) —

    Remote name for a repo

Returns:



410
411
412
413
414
415
416
 
# File 'lib/toys/utils/git_cache.rb', line 410

def repo_info(remote)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  lock_repo(dir, remote) do |repo_lock|
    RepoInfo.new(dir, repo_lock.data)
  end
end