Class: Toys::Utils::GitCache

Inherits:
Object
  • Object
show all
Defined in:
lib/toys/utils/git_cache.rb

Overview

This object provides cached access to remote git data. Given a remote repository, a path, and a commit, it makes the files availble in the local filesystem. Access is cached, so repeated requests for the same commit and path in the same repo do not hit the remote repository again.

This class is used by the Loader to load tools from git. Tools can also use the :git_cache mixin for direct access to this class.

Defined Under Namespace

Classes: Error, RefInfo, RepoInfo, SourceInfo

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_dir: nil) ⇒ GitCache

Access a git cache.

Parameters:

  • cache_dir (String) (defaults to: nil) —

    The path to the cache directory. Defaults to a specific directory in the user's XDG cache.



298
299
300
301
302
303
304
305
306
 
# File 'lib/toys/utils/git_cache.rb', line 298

def initialize(cache_dir: nil)
  require "digest"
  require "fileutils"
  require "json"
  require "toys/compat"
  require "toys/utils/exec"
  @cache_dir = ::File.expand_path(cache_dir || default_cache_dir)
  @exec = Utils::Exec.new(out: :capture, err: :capture)
end

Instance Attribute Details

#cache_dir ⇒ String (readonly)

The cache directory.

Returns:

  • (String) 


313
314
315
 
# File 'lib/toys/utils/git_cache.rb', line 313

def cache_dir
  @cache_dir
end

Class Method Details

.sources_writable? ⇒ boolean

Returns whether shared source files are writable by default. Normally, shared sources are made read-only to protect them from being modified accidentally since multiple clients may be accessing them. However, you can disable this feature by setting the environment variable TOYS_GIT_CACHE_WRITABLE to any non-empty value. This can be useful in environments that want to clean up temporary directories and are being hindered by read-only files.

Returns:

  • (boolean) 


288
289
290
 
# File 'lib/toys/utils/git_cache.rb', line 288

def self.sources_writable?
  !::ENV["TOYS_GIT_CACHE_WRITABLE"].to_s.empty?
end

Instance Method Details

#get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil) ⇒ String Also known as: find

Get the given git-based files from the git cache, loading from the remote repo if necessary.

The resulting files are either copied into a directory you provide in the :into parameter, or populated into a shared source directory if you omit the :info parameter. In the latter case, it is important that you do not modify the returned files or directories, nor add or remove any files from the directories returned, to avoid confusing callers that could be given the same directory. If you need to make any modifications to the returned files, use :into to provide your own private directory.

Parameters:

  • remote (String) —

    The URL of the git repo. Required.

  • path (String) (defaults to: nil) —

    The path to the file or directory within the repo. Optional. Defaults to the entire repo.

  • commit (String) (defaults to: nil) —

    The commit reference, which may be a SHA or any git ref such as a branch or tag. Optional. Defaults to HEAD.

  • into (String) (defaults to: nil) —

    If provided, copies the specified files into the given directory path. If omitted or nil, populates and returns a shared source file or directory.

  • update (Boolean, Integer) (defaults to: false) —

    Whether to update non-SHA commit references if they were previously loaded. This is useful, for example, if the commit is HEAD or a branch name. Pass true or false to specify whether to update, or an integer to update if last update was done at least that many seconds ago. Default is false.

Returns:

  • (String) —

    The full path to the cached files. The returned path will correspod to the path given. For example, if you provide the path Gemfile representing a single file in the repository, the returned path will point directly to the cached copy of that file.



348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
 
# File 'lib/toys/utils/git_cache.rb', line 348

def get(remote, path: nil, commit: nil, into: nil, update: false, timestamp: nil)
  path = GitCache.normalize_path(path)
  commit ||= "HEAD"
  timestamp ||= ::Time.now.to_i
  dir = ensure_repo_base_dir(remote)
  lock_repo(dir, remote, timestamp) do |repo_lock|
    ensure_repo(dir, remote)
    sha = ensure_commit(dir, commit, repo_lock, update)
    if into
      copy_files(dir, sha, path, repo_lock, into)
    else
      ensure_source(dir, sha, path, repo_lock)
    end
  end
end

#remotes ⇒ Array<String>

Returns an array of the known remote names.

Returns:

  • (Array<String>) 


370
371
372
373
374
375
376
377
378
379
380
381
382
 
# File 'lib/toys/utils/git_cache.rb', line 370

def remotes
  result = []
  return result unless ::File.directory?(cache_dir)
  ::Dir.entries(cache_dir).each do |child|
    next if child.start_with?(".")
    dir = ::File.join(cache_dir, child)
    if ::File.file?(::File.join(dir, LOCK_FILE_NAME))
      remote = lock_repo(dir, &:remote)
      result << remote if remote
    end
  end
  result.sort
end

#remove_refs(remote, refs: nil) ⇒ Array<RefInfo>?

Remove records of the given refs (i.e. branches, tags, or HEAD) from the given repository's cache. The next time those refs are requested, they will be pulled from the remote repo.

If you provide the refs: argument, only those refs are removed. Otherwise, all refs are removed.

Parameters:

  • remote (String) —

    The repository

  • refs (Array<String>) (defaults to: nil) —

    The refs to remove. Optional.

Returns:

  • (Array<RefInfo>, nil) —

    The refs actually forgotten, or nil if the given repo is not in the cache.



438
439
440
441
442
443
444
445
446
447
448
449
450
 
# File 'lib/toys/utils/git_cache.rb', line 438

def remove_refs(remote, refs: nil)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  results = []
  lock_repo(dir, remote) do |repo_lock|
    refs = repo_lock.refs if refs.nil? || refs == :all
    Array(refs).each do |ref|
      ref_data = repo_lock.delete_ref!(ref)
      results << RefInfo.new(ref, ref_data) if ref_data
    end
  end
  results.sort
end

#remove_repos(remotes) ⇒ Array<String>

Removes caches for the given repos, or all repos if specified.

Removes all cache information for the specified repositories, including local clones and shared source directories. The next time these repositories are requested, they will be reloaded from the remote repository from scratch.

Be careful not to remove repos that are currently in use by other GitCache clients.

Parameters:

  • remotes (Array<String>, :all) —

    The remotes to remove.

Returns:

  • (Array<String>) —

    The remotes actually removed.



413
414
415
416
417
418
419
420
421
422
423
 
# File 'lib/toys/utils/git_cache.rb', line 413

def remove_repos(remotes)
  remotes = self.remotes if remotes.nil? || remotes == :all
  Array(remotes).map do |remote|
    dir = repo_base_dir_for(remote)
    if ::File.directory?(dir)
      ::FileUtils.chmod_R("u+w", dir, force: true)
      ::FileUtils.rm_rf(dir)
      remote
    end
  end.compact.sort
end

#remove_sources(remote, commits: nil) ⇒ Array<SourceInfo>?

Removes shared sources for the given cache. The next time a client requests them, the removed sources will be recopied from the repo.

If you provide the commits: argument, only sources associated with those commits are removed. Otherwise, all sources are removed.

Be careful not to remove sources that are currently in use by other GitCache clients.

Parameters:

  • remote (String) —

    The repository

  • commits (Array<String>) (defaults to: nil) —

    Remove only the sources for the given commits. Optional.

Returns:

  • (Array<SourceInfo>, nil) —

    The sources actually removed, or nil if the given repo is not in the cache.



468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
 
# File 'lib/toys/utils/git_cache.rb', line 468

def remove_sources(remote, commits: nil)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  results = []
  lock_repo(dir, remote) do |repo_lock|
    commits = nil if commits == :all
    shas = Array(commits).map { |ref| repo_lock.lookup_ref(ref) }.compact.uniq if commits
    repo_lock.find_sources(shas: shas).each do |(sha, path)|
      data = repo_lock.delete_source!(sha, path)
      results << SourceInfo.new(dir, sha, path, data)
    end
    results.map(&:sha).uniq.each do |sha|
      unless repo_lock.source_exists?(sha)
        sha_dir = ::File.join(dir, sha)
        ::FileUtils.chmod_R("u+w", sha_dir, force: true)
        ::FileUtils.rm_rf(sha_dir)
      end
    end
  end
  results.sort
end

#repo_info(remote) ⇒ RepoInfo?

Returns a RepoInfo describing the cache for the given remote, or nil if the given remote has never been cached.

Parameters:

  • remote (String) —

    Remote name for a repo

Returns:



391
392
393
394
395
396
397
 
# File 'lib/toys/utils/git_cache.rb', line 391

def repo_info(remote)
  dir = repo_base_dir_for(remote)
  return nil unless ::File.directory?(dir)
  lock_repo(dir, remote) do |repo_lock|
    RepoInfo.new(dir, repo_lock.data)
  end
end