Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* text=auto eol=lf
46 changes: 46 additions & 0 deletions .github/workflows/check-dist.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Check Dist

on:
pull_request:
push:

permissions:
contents: read

jobs:
check-dist:
name: Check dist/
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20

- uses: pnpm/action-setup@v3
with:
version: 8

- name: Install and Build
run: pnpm i

- name: Validate Dist
id: diff
run: |
if [ "$(git diff --ignore-space-at-eol --text dist/ | wc -l)" -gt "0" ]; then
echo "Detected uncommitted changes after build. See status below:"
git diff --ignore-space-at-eol --text dist/
exit 1
fi

# If `dist/` was different than expected, upload it
- if: ${{ failure() && steps.diff.outcome == 'failure' }}
name: Upload Artifact
id: upload
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
retention-days: 10
37 changes: 17 additions & 20 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,26 @@ jobs:
id: meta
with:
images: test
- name: Cache var-cache-apt
id: cache-var-cache-apt
uses: actions/cache@v3

- name: Cache
uses: actions/cache@v4
id: cache
with:
path: var-cache-apt
key: var-cache-apt-${{ hashFiles('.github/workflows/test/Dockerfile') }}
- name: Cache var-lib-apt
id: cache-var-lib-apt
uses: actions/cache@v3
with:
path: var-lib-apt
key: var-lib-apt-${{ hashFiles('.github/workflows/test/Dockerfile') }}
- name: inject var-cache-apt into docker
uses: ./
with:
cache-source: var-cache-apt
cache-target: /var/cache/apt
- name: inject var-lib-apt into docker
path: |
var-cache-apt
var-lib-apt
key: cache-${{ hashFiles('.github/workflows/test/Dockerfile') }}

- name: inject cache into docker
uses: ./
with:
cache-source: var-lib-apt
cache-target: /var/lib/apt
skip-extraction: ${{ steps.cache-var-lib-apt.outputs.cache-hit }}
skip-extraction: false
cache-map: |
{
"var-cache-apt": "/var/cache/apt",
"var-lib-apt": "/var/lib/apt"
}

- name: Build and push
uses: docker/build-push-action@v5
with:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
.parcel-cache/
96 changes: 68 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# The BuildKit Cache Dance: save `RUN --mount=type=cache` caches on GitHub Actions
# The BuildKit Cache Dance
Save `RUN --mount=type=cache` caches on GitHub Actions or other CI platforms

The BuildKit Cache Dance allows saving [`RUN --mount=type=cache`](https://docs.docker.com/build/guide/mounts/#add-a-cache-mount)
caches on GitHub Actions.
caches on GitHub Actions or other CI platforms by extracting the cache from the previous build and injecting it into the current build.

Use cases:
- apt-get (`/var/cache/apt`, `/var/lib/apt`)
Expand All @@ -14,7 +15,9 @@ This [`reproducible-containers/buildkit-cache-dance`](https://github.com/reprodu
This action be used for "non-reproducible" containers too.

## Examples
### apt-get

### apt-get GitHub Actions

Dockerfile:
```dockerfile
FROM ubuntu:22.04
Expand All @@ -32,43 +35,39 @@ Action:
```yaml
---
name: Build
on: push
on:
push:

jobs:
build:
name: Build
Build:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/metadata-action@v5
id: meta
with:
images: YOUR_IMAGE
- name: Cache var-cache-apt
id: cache-var-cache-apt
uses: actions/cache@v3
with:
path: var-cache-apt
key: var-cache-apt-${{ hashFiles('Dockerfile') }}
- name: Cache var-lib-apt
id: cache-var-lib-apt
images: Build

- name: Cache
uses: actions/cache@v3
id: cache
with:
path: var-lib-apt
key: var-lib-apt-${{ hashFiles('Dockerfile') }}
- name: inject var-cache-apt into docker
uses: reproducible-containers/[email protected]
with:
cache-source: var-cache-apt
cache-target: /var/cache/apt
skip-extraction: ${{ steps.cache-var-cache-apt.outputs.cache-hit }}
- name: inject var-lib-apt into docker
uses: reproducible-containers/[email protected]
path: |
var-cache-apt
var-lib-apt
key: cache-${{ hashFiles('.github/workflows/test/Dockerfile') }}

- name: inject cache into docker
uses: reproducible-containers/[email protected]
with:
cache-source: var-lib-apt
cache-target: /var/lib/apt
skip-extraction: ${{ steps.cache-var-lib-apt.outputs.cache-hit }}
cache-map: |
{
"var-cache-apt": "/var/cache/apt",
"var-lib-apt": "/var/lib/apt"
}
skip-extraction: ${{ steps.cache.outputs.cache-hit }}

- name: Build and push
uses: docker/build-push-action@v5
with:
Expand All @@ -79,12 +78,47 @@ jobs:
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

```

Real-world examples:
- <https://github.com/rootless-containers/slirp4netns/blob/v1.2.2/.github/workflows/release.yaml#L18-L36>
- <https://github.com/containers/fuse-overlayfs/blob/40e0f3c/.github/workflows/release.yaml#L17-L36>

## CLI Usage

In other CI systems, you can run the script directly via `node`:

```shell
curl -LJO https://github.com/reproducible-containers/buildkit-cache-dance/archive/refs/tags/v3.0.0.tar.gz
tar xvf buildkit-cache-dance-3.0.0.tar.gz
```
During injection:

```shell
node ./buildkit-cache-dance-3.0.0/dist/index.js --cache-map '{"var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt"}'
```

After build during extraction:

```shell
node ./buildkit-cache-dance-3.0.0/dist/index.js --extract --cache-map '{"var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt"}'
```

Here are the available options:

```
build-cache-dance [options]
Save 'RUN --mount=type=cache' caches on GitHub Actions or other CI platforms

Options:
--extract Extract the cache from the docker container (extract step). Otherwise, inject the cache (main step)
--cache-map The map of actions source to container destination paths for the cache paths
--scratch-dir Where the action is stores some temporary files for its processing. Default: 'scratch'
--skip-extraction Skip the extraction of the cache from the docker container
--help Show this help
```

## Releases
### v1
v1 follows the original design of [`overmindtech/buildkit-cache-dance`](https://github.com/overmindtech/buildkit-cache-dance/tree/306d31a77191f643c0c4a95083f36c6ddccb4a16).
Expand All @@ -98,6 +132,12 @@ See the [`releases/v1`](https://github.com/reproducible-containers/buildkit-cach
### v2
v2 is composed of the single `reproducible-containers/buildkit-cache-dance` action.

### v3

Rewrote the action in TypeScript and adds support for `cache-map` that gets a string of files that need to be injected as a JSON string. This makes it possible to inject multiple directories in one call and simplifies the usage.

This release also makes it possible to run the script outside GitHub Actions in other CI platforms or locally using command line arguments.

## Acknowledgement
- Thanks to [Alexander Pravdin](https://github.com/speller) for the basic idea in [this comment](https://github.com/moby/buildkit/issues/1512).
- Thanks to the authors of the original [`overmindtech/buildkit-cache-dance`](https://github.com/overmindtech/buildkit-cache-dance).
11 changes: 7 additions & 4 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
name: Inject/Extract Cache
description: "Injects the cached data into the docker build(x|kit) process"
inputs:
cache-map:
required: true
description: "The map of actions source to container destination paths for the cache paths"
cache-source:
default: cache
deprecationMessage: "Use `cache-map` instead"
description: "Where the cache is stored in the calling workspace. Default: `cache`"
cache-target:
default: /root/.cache/go-build
deprecationMessage: "Use `cache-map` instead"
description: "Where the cache is stored in the docker container. Default: `/root/.cache/go-build`"
scratch-dir:
default: scratch
Expand All @@ -15,5 +18,5 @@ inputs:
description: "Skip the extraction of the cache from the docker container"
runs:
using: 'node20'
main: 'entrypoint.js'
post: 'entrypoint.js'
main: 'dist/index.js'
post: 'dist/index.js'
Loading