Accessing private GitHub repositories over HTTP

Published on 2020-05-10
Tagged: git go

This article provides instructions on configuring Git to authenticate with private GitHub repositories. This is helpful for downloading Go modules in a CI environment where HTTP is preferred, credentials can't be stored permanently, and typing a password or tapping a security key is not possible.

This article mainly serves as my own personal reference. I had trouble finding documentation elsewhere, so hopefully others will find this useful, too. There are a lot of ways to do this, so it's not one-size-fits-all.

The plan

We want to download and test a Go module in a private GitHub repository from a Docker container. Our container image may be public, so it can't contain the module itself or our GitHub credentials. Instead, the module name and our credentials will be passed in through environment variables.

We'll use a credential helper to tell Git to use our credentials. A credential helper is a user-specified program that Git can use to fetch and store credentials. gitcredentials has information about configuring helpers. git-credential has information about writing helpers. For this article, we'll use the built-in cache helper, which temporarily stores secrets in memory (15 minutes by default). See git-credential-cache for documentation. You can write your own helpers to read and store secrets from other sources.

Instead of using a password, we'll use a GitHub personal access token as if it were a password. Regular passwords won't work if you have 2FA enabled (you do have it enabled, right?). Tokens can be used for HTTP basic authentication both on GitHub repositories and on the GitHub API. Tokens can also be limited to a small set of privileges. Unfortunately, they can't be limited to read-only access on a personal repository: it's full access or nothing. If you're part of an organization, you can create a service account with read-only access to your organization's repos, then authenticate with that in CI.

Steps

Create a personal access token. Give it repo access (Full control of private repositories).
Run the command below, replacing user with your username or organization name and repo with your repository name:
```
git config --global https://github.com.helper=cache
```
This configures Git to use the cache credential helper for repos at https://github.com.
Run the command below. This assumes the environment variables GITHUB_USERNAME and GITHUB_TOKEN are set to your username and personal access token, respectively.
```
git credential approve <<EOF
protocol=https
host=github.com
username=$GITHUB_USERNAME
token=$GITHUB_TOKEN
EOF
```
This tells Git your credentials. It should be able to access your repository after this point.
When downloading Go modules, make sure to set GOPRIVATE=github.com/user/repo (again replacing user and repo). This tells Go not to attempt to download your module from a proxy and not to verify your module using the checksum database.
Run the command below before running any code you don't control:
```
git credential-cache exit
```
This shuts down the daemon holding credentials in memory. Before this point, processes run by the same user can access cached credentials by communicating with the daemon. So for example, a test in an unreviewed pull request could exfiltrate your token.

Example

Let's put this together with a bash script:

#!/usr/bin/env bash

set -euo pipefail

# Configure Git.
git config --global credential.https://github.com.helper cache
git credential approve <<EOF
protocol=https
host=github.com
username=$USERNAME
password=$PASSWORD
EOF

# Download modules, but don't run anything yet.
mkdir test
cd test
export GOPRIVATE="github.com/$USERNAME/$REPO"
go mod init test
go get -d "github.com/$USERNAME/$REPO@$REV"
go list -test "github.com/$USERNAME/$REPO/..." >/dev/null

# Tell Git to forget about our credentials.
git credential-cache exit

# Run tests.
go test "github.com/$USERNAME/$REPO/..."

This script could be run from a short Dockerfile:

FROM golang:latest
COPY test.sh .
CMD ["./test.sh"]