LFS Pointers

A Swift library and CLI that allows you to convert a Git repository of large files to Git LFS pointers.

LFSPointers image

4.0.1 is the last release in which the executable is named LFSPointers. All future releases will name it lfs-pointers.

LFS Pointers
Table of Contents

Created by gh-md-toc

It it recommended that you read the Git-LFS Homepage before continuing.

Install Program

Mint

$ mint install LebJe/LFSPointers

Homebrew

$ brew install LebJe/formulae/lfs-pointers

If you would like to install from HEAD then make sure swift --version succeeds (Install Swift otherwise) then run:

$ brew install LebJe/formulae/lfs-pointers --HEAD

From DEB or RPM

DEB

curl -s https://packagecloud.io/install/repositories/LebJe/LFSPointers/script.deb.sh > script.sh
chmod +x script.sh
sudo os=ubuntu dist=focal ./script.sh
sudo apt install lfspointers

RPM

curl -s https://packagecloud.io/install/repositories/LebJe/LFSPointers/script.rpm.sh > script.sh && chmod +x script.sh
sudo os=fedora dist=32 ./script.sh
sudo yum install LFSPointers

Manually

Build Using Docker

If you are on Ubuntu 16.04, 18.04, 20.04, or CentOS 8, you can build LFSPointers using:

Ubuntu 16.04

$ docker run --rm -v $(pwd):/src -w /src swift:xenial swift build -c release --enable-test-discovery --static-swift-stdlib -Xswiftc -static-executable

Ubuntu 18.04

$ docker run --rm -v $(pwd):/src -w /src swift:bionic swift build -c release --enable-test-discovery --static-swift-stdlib -Xswiftc -static-executable

Ubuntu 20.04

$ docker run --rm -v $(pwd):/src -w /src swift:focal swift build -c release --enable-test-discovery --static-swift-stdlib -Xswiftc -static-executable

CentOS 8

$ docker run --rm -v $(pwd):/src -w /src swift:centos8 swift build -c release --enable-test-discovery --static-swift-stdlib -Xswiftc -static-executable

Then run mv .build/release/LFSPointers . to move the binary to your current directory.

Build Without Docker

If you don’t or can’t use Docker, you can Install Swift, then run:

$ swift build -c release

The binary will be located at /path/to/LFSPointers/.build/release/LFSPointers.

Build on Windows

Install Swift, open Windows Powershell, navigate to the directory that contains LFSPointers, and run:

swift build -Xswiftc -sdk -Xswiftc $env:SDKROOT

the executable will be located at C:\path\to\LFSPointers\.build\release\LFSPointers.exe.

From GitHub Release

Simply download the release asset. The binary is statically linked, so there is no need to install additional software.

Setup Shell Completions

ZSH

Oh My ZSH

Create a file called ~/.oh-my-zsh/completions/_LFSPointers, then run:

% LFSPointers --generate-completion-script zsh > ~/.oh-my-zsh/completions/_LFSPointers

Without Oh My ZSH

Add

fpath=(~/.zsh/completion $fpath)
autoload -U compinit
compinit

to your .zshrc, then create ~/.zsh/completion, and run:

% LFSPointers --generate-completion-script zsh > ~/.zsh/completion/_LFSPointers

Bash

Create a directory to store Bash completions, (~/.bash_completions/), and add this to your .bashrc or .bash_profile:

source ~/.bash_completions/LFSPointers.bash

, then run:

$ LFSPointers --generate-completion-script bash > ~/.bash_completions/LFSPointers.bash

Install Library

Swift Package Manager

Add this to the dependencies array in Package.swift:

.package(url: "https://github.com/LebJe/LFSPointers.git", from: “4.0.0”)

. Also add this to the targets array in the aforementioned file:

.product(name: "LFSPointersKit", package: "LFSPointers")

Usage

Library

Import

import LFSPointersKit

File Conversion

To convert a file to a pointer you could write:

let pointer = try LFSPointer(fromFile: URL(fileURLWithPath: "path/to/file"))

The pointer is represented as a Swift struct.

public struct LFSPointer {
    /// The version of the pointer. Example: "https://git-lfs.github.com/spec/v1".
    public let version: String

    /// An SHA 256 hash for the pointer.
    public let oid: String

    /// The size of the converted file.
    public let size: Int

    /// The name of the file.
    public let filename: String

    /// The full path of the file.
    public let filePath: String
    
    /// String representation of this pointer.
    public var stringRep: String
    ...
}

Folder Conversion

To convert a folder of files to pointers, you could write:

let pointers = try LFSPointer.pointers(forDirectory: URL(fileURLWithPath: "path/to/folder"), searchType: .filenames(["foo.java", "bar.js", "baz.py"]))

The search types available are:

// Array of filenames.
.fileNames(["foo.java", "bar.js", "baz.py"])

// Regular expression.
.regex(NSRegularExpression(pattern: "^*.swiftquot;))

// All files.
.all

That function returns an array of LFSPointers.

Writing Pointers

After you generate a pointer, write it to a file using:

let pointer = try LFSPointer(...)
try pointer.write(toFile: URL(fileURLWithPath: "path/to/file"), shouldAppend: false)

Generating JSON

To convert a pointer to JSON:

let pointer = try LFSPointer(...)
try JSONEncoder().encode(pointer)

and to convert an array of LFSPointers:

let pointers = try LFSPointer.pointers(...)
JSONEncoder().encode(pointers)

The JSON for the LFSPointer array will be structured as shown here, and the JSON for the single LFSPointer will be structured as shown here.

Command Line

If you want to use the Docker image, prefix all the following commands with: docker run --rm -v $(pwd):/src -w /src ghcr.io/lebje/lfs-pointers:latest <command>

Let’s imagine you have a directory of large png and jpg files called Project Logos. If you wanted to convert the files with the extension png to LFS pointers, you could run

$ LFSPointers path/to/Project\ Logos path/to/Project\ Logos/*.png

. The first argument is the path to the directory, and the second argument is a regular expression used to search for png files that your shell will convert to a list of filenames.
But wait! It’s not safe to run random programs on your computer! To backup your files just in case something goes wrong, add -b path/to/backup-directory to the previous command, like this:

$ LFSPointers -b path/to/backup-directory path/to/Project\ Logos path/to/Project\ Logos/*.png

If you want to generate JSON output instead, run:

$ LFSPointers --json path/to/Project\ Logos path/to/Project\ Logos/*.png

The JSON will be structured as shown here.

Dependencies

More Information

Run LFSPointers --help.

Tested Platforms

Mac

Tested on MacOS 10.15 and 11, using Swift 5.2 and 5.3.

Linux

Tested on Ubuntu 18.04 (x86_64 and aarch64), also using Swift 5.2 and 5.3.

iOS

This project has not been tested on iOS yet, although, at the time of writing the sample project in Samples/FileToPointer is currently working.

Windows

LFSPointers is currently being built and tested on Windows, but there are a few problems:

Windows paths are converted to UNIX paths: C:\Users\user\file => C:/Users/user/file

JSON Structure for LFSPointer Array

[
    {
        "version": "https://git-lfs.github.com/spec/v1",
        "oid": "10b2cd328e193dd4b81d921dbe91bda74bda704c37bca43f1e15f41fcd20ac2a",
        "size": 1455,
        "filename": "foo.txt",
        "filePath": "/path/to/foo.txt"
    },
    {
        "version": "https://git-lfs.github.com/spec/v1",
        "oid": "601952b2d85214ea602104a4784728ffa6b323b3a6131a124044fa5bfc2f7bf2",
        "size": 1285200,
        "filename": "bar.txt",
        "filePath": "/path/to/bar.txt"
    }
]

JSON Structure for Single LFSPointer

{
    "version": "https://git-lfs.github.com/spec/v1",
    "oid": "10b2cd328e193dd4b81d921dbe91bda74bda704c37bca43f1e15f41fcd20ac2a",
    "size": 1455,
    "filename": "foo.txt",
    "filePath": "/path/to/foo.txt"
}

Install Swift

Download Swift, then follow the instructions for your platform:

Linux (`aarch64`)

More information at swift-arm64.

Ubuntu

$ curl -s https://packagecloud.io/install/repositories/swift-arm/release/script.deb.sh | sudo bash
sudo apt install swiftlang

Contributing

Before committing, please install pre-commit, and swift-format and install the pre-commit hook:

$ brew bundle # install the packages specified in Brewfile
$ pre-commit install

# Commit your changes.

To install pre-commit on other platforms, refer to the documentation.