Cache module dependencies between container builds by kaovilai · Pull Request #8400 · vmware-tanzu/velero

kaovilai · 2024-11-12T21:35:34Z

Cache module dependencies between container builds

Thank you for contributing to Velero!

Please add a summary of your change

Makes subsequent builds faster if only code change and no go.mod changes.

Does your change fix a particular issue?

Fixes #(issue)

Please indicate you've done the following:

Accepted the DCO. Commits without the DCO will delay acceptance.
Created a changelog file (make new-changelog) or comment /kind changelog-not-required on this PR.
Updated the corresponding documentation in site/content/docs/main.

kaovilai · 2024-11-12T21:35:36Z

/kind changelog-not-required

codecov · 2024-11-12T21:43:37Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.38%. Comparing base (9afad9a) to head (6be30f8).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8400   +/-   ##
=======================================
  Coverage   59.38%   59.38%           
=======================================
  Files         370      370           
  Lines       39979    39979           
=======================================
  Hits        23742    23742           
  Misses      14745    14745           
  Partials     1492     1492

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Dockerfile

hack/build-restic.sh

kaovilai · 2024-11-15T17:42:01Z

as of e465159
Making one .go change and rebuild saves about 15s vs main branch 🤷

export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
...
make all-containers  0.87s user 0.90s system 2% cpu 1:15.95 total

Details

export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
[+] Building 0.7s (26/26) FINISHED                                                                                             kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.3s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.3s
 => [auth] paketobuildpacks/run-jammy-tiny:pull token for registry-1.docker.io                                                           0.0s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                            0.0s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 561.56kB                                                                                                    0.3s
 => CACHED [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                        0.0s
 => CACHED [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                      0.0s
 => CACHED [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                          0.0s
 => CACHED [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                    0.0s
 => CACHED [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                       0.0s
 => CACHED [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  0.0s
 => CACHED [stage-4 2/4] COPY --from=velero-builder /output /                                                                            0.0s
 => CACHED [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /outpu  0.0s
 => CACHED [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                     0.0s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git clone   0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git apply   0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         GOARM=$(ec  0.0s
 => CACHED [stage-4 4/4] COPY --from=restic-builder /output /                                                                            0.0s
 => exporting to image                                                                                                                   0.1s
 => => exporting layers                                                                                                                  0.0s
 => => exporting manifest sha256:dcfbf179655194c22da2659e05a68ea402f1d1549ff0a6b1d3ca32a73ea49333                                        0.0s
 => => exporting config sha256:c5dc14d0311b58770a32d62e5206deec2a416c8b7a70bcc7ceebd7f82217c711                                          0.0s
 => => exporting attestation manifest sha256:1647713ba73b9d510d193c64a33df1f0a1d6c66462ed12a3e86adfbd005dc436                            0.0s
 => => exporting manifest list sha256:7037efc2a8a65ed211f09b845f3dbb45008e31e7932c09c49791396521595689                                   0.0s
container: velero/velero:main
[+] Building 0.4s (24/24) FINISHED                                                                                             kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 543.23kB                                                                                                    0.2s
 => [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                        0.0s
 => CACHED [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                      0.0s
 => CACHED [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                          0.0s
 => CACHED [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                    0.0s
 => CACHED [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                       0.0s
 => CACHED [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  0.0s
 => CACHED [stage-4 2/4] COPY --from=velero-builder /output /                                                                            0.0s
 => CACHED [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /outpu  0.0s
 => CACHED [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                     0.0s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [stage-4 4/4] COPY --from=restic-builder /output /                                                                            0.0s
 => exporting to image                                                                                                                   0.1s
 => => exporting layers                                                                                                                  0.0s
 => => exporting manifest sha256:255255475f956eab91df8fcc77f4bb5227b16c8107f694efa287eb955879296a                                        0.0s
 => => exporting config sha256:b963c6d13b12412bd87fba4fedbf7f88a9dfdf10c37eaf5b7e1d05177e6ccefb                                          0.0s
 => => exporting attestation manifest sha256:ff8d442eda7b9fab5480296f8f654beb4b9316c77d9575ab2839ebb236f90d0a                            0.0s
 => => exporting manifest list sha256:bfb0cc9ad1cacf09e849c960eb218f9ce03b8646b841b62cb48c2a1592a13730                                   0.0s
container: velero/velero-restore-helper:main
[+] Building 52.4s (24/24) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => CACHED [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 544.21kB                                                                                                    0.2s
 => [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                               2.9s
 => [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                             2.0s
 => [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                                 1.8s
 => [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                           1.8s
 => [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                              2.0s
 => [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/velero       37.7s
 => [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  4.9s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git clone   0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         git apply   0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero" = "velero" ]; then         GOARM=$(ec  0.0s
 => [stage-4 2/4] COPY --from=velero-builder /output /                                                                                   0.3s
 => [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                            0.2s
 => [stage-4 4/4] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   3.1s
 => => exporting layers                                                                                                                  3.0s
 => => exporting manifest sha256:c823dfc0ba2540e89ab6594ef91c18aec4c1633cd777e21d06591a740e10022a                                        0.0s
 => => exporting config sha256:ce5ecab4a7989ca4e673977cad3a1fcfec4fb9b43631d04f8a6fffe188a709c2                                          0.0s
 => => exporting attestation manifest sha256:b474847335e75a25362468baaadfa89ee505825aa3f68735e4c0275d1f482840                            0.0s
 => => exporting manifest list sha256:772be783cef2ecd2579fcc51200b7d463f663afd21a7f041e95069326ce8eb26                                   0.0s
container: velero/velero:main
[+] Building 22.3s (24/24) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 4.05kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [restic-builder 1/7] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 561.56kB                                                                                                    0.2s
 => CACHED [stage-4 1/4] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [velero-builder-base 2/6] RUN mkdir -p /output/usr/bin                                                                               2.1s
 => [velero-builder-base 3/6] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                             1.7s
 => [velero-builder-base 4/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/                                                 1.6s
 => [velero-builder-base 5/6] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb go mod download                                           2.1s
 => [velero-builder-base 6/6] COPY . /go/src/github.com/vmware-tanzu/velero                                                              6.6s
 => [velero-builder 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/velero-resto  4.7s
 => [velero-builder-helper 1/1] RUN --mount=type=cache,target=/go/pkg/mod,id=vbb GOARM=$( echo "" | cut -c2-) go build -o /output/veler  6.8s
 => [stage-4 2/4] COPY --from=velero-builder /output /                                                                                   0.1s
 => CACHED [restic-builder 2/7] RUN mkdir -p /output/usr/bin && mkdir -p /build/restic                                                   0.0s
 => CACHED [restic-builder 3/7] WORKDIR /build/restic                                                                                    0.0s
 => CACHED [restic-builder 4/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 5/7] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/                                0.0s
 => CACHED [restic-builder 6/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => CACHED [restic-builder 7/7] RUN --mount=type=cache,target=/go/pkg/mod,id=restic if [ "velero-restore-helper" = "velero" ]; then      0.0s
 => [stage-4 3/4] COPY --from=velero-builder-helper /output /                                                                            0.1s
 => [stage-4 4/4] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:90d5e28e7f7af8a3f3f7d7644dd966d158decb4c8234ae2add50bcfd5c78be42                                        0.0s
 => => exporting config sha256:874f8a23969c0475ab6935408fea9df2fb66ff5a1e2464ec20255c59239caef2                                          0.0s
 => => exporting attestation manifest sha256:1e80048ae5da47811786af550b7f51f41ac177b089053f161ee3a75f963fa080                            0.0s
 => => exporting manifest list sha256:a417ed6e4df5a1443c637f51e3e90eff5d0041566437a69a48cd14d91a975a10                                   0.0s
container: velero/velero-restore-helper:main
make all-containers  0.87s user 0.90s system 2% cpu 1:15.95 total

vs main branch

git reset --hard && git checkout upstream/main && export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
...
make all-containers  0.93s user 1.01s system 2% cpu 1:28.55 total

Details

❯ git reset --hard && git checkout upstream/main && export BUILDX_PLATFORMS=linux/arm64; make all-containers && echo "// test cache" >> pkg/constant/constant.go && time make all-containers
HEAD is now at e46515963 Cache module dependencies between container builds
HEAD is now at dacd5eff9 Merge pull request #8380 from sseago/worker-count
[+] Building 70.9s (15/15) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 1.44MB                                                                                                      0.3s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                                  2.7s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   5.1s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   4.5s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu  38.6s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero      59.7s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.2s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s 
 => exporting to image                                                                                                                   3.0s 
 => => exporting layers                                                                                                                  2.9s 
 => => exporting manifest sha256:cc0bcc448812dc8cda723d577dbc4b0a7f1b0b2c959d11bf6138132d0aca781b                                        0.0s 
 => => exporting config sha256:af7e3af8c54a3e2f29090bf0e18f50595cdd6c085e314f81bd0e207a97784bf5                                          0.0s 
 => => exporting attestation manifest sha256:4dabab59e9a03035e65aec199abaf41388bafb8bf1e91492f488f9febb2caa76                            0.0s 
 => => exporting manifest list sha256:69c0f6643eb59d576739e3a27b09a972c4743020b8b4b2bc273d88a432e6a719                                   0.0s 
container: velero/velero:main                                                                                                                 
[+] Building 8.5s (15/15) FINISHED                                                                                             kubernetes:crc 
 => [internal] load build definition from Dockerfile                                                                                     0.0s 
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s 
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.2s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.2s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => [internal] load build context                                                                                                        0.3s
 => => transferring context: 543.35kB                                                                                                    0.3s
 => [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => CACHED [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                            0.0s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => CACHED [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                            0.0s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu/  2.2s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero-rest  6.8s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.1s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:d4a247b9a394bbc7a9a811a0c09c979cb626df084b9a9f412d27af87c390ef3a                                        0.0s
 => => exporting config sha256:a78a5ad0417fd9393119e25fc7d0fadc9072a9ecfb670652c6b112a36c6eb459                                          0.0s
 => => exporting attestation manifest sha256:808c4106e969f75e9defc3e9e758539ea15273d65ae35f7c5823a1e531530916                            0.0s
 => => exporting manifest list sha256:11501d2996a4fb0726d80a88943ee80fbffafeeb4ffcfbc4d4e1b21ba0d68485                                   0.0s
container: velero/velero-restore-helper:main
[+] Building 71.5s (15/15) FINISHED                                                                                            kubernetes:crc
 => [internal] load build definition from Dockerfile                                                                                     0.0s
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.1s
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.1s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 544.33kB                                                                                                    0.2s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   3.5s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   5.5s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu  45.2s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero      61.8s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.2s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s 
 => exporting to image                                                                                                                   3.0s 
 => => exporting layers                                                                                                                  2.9s 
 => => exporting manifest sha256:e42bc864d6742dc5823df75c79d488f2fbe93ddac7425bf718e6876567b83024                                        0.0s 
 => => exporting config sha256:a5a2d96b157fcdb1ad02d4011796e489428609560b1206b0e76fbb9792032547                                          0.0s 
 => => exporting attestation manifest sha256:8dfc981ebcc11a3acc60251044f35757b5797870d9154656153de9d4d20869d4                            0.0s 
 => => exporting manifest list sha256:1be37e0ed679001fcbc553ba112865fba9cb42a6a523202f1066c3b2f7edd09f                                   0.0s 
container: velero/velero:main                                                                                                                 
[+] Building 15.8s (17/17) FINISHED                                                                                            kubernetes:crc 
 => [internal] load build definition from Dockerfile                                                                                     0.0s 
 => => transferring dockerfile: 2.38kB                                                                                                   0.0s 
 => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest                                                        0.3s
 => [internal] load metadata for docker.io/library/golang:1.22-bookworm                                                                  0.3s
 => [auth] library/golang:pull token for registry-1.docker.io                                                                            0.0s
 => [auth] paketobuildpacks/run-jammy-tiny:pull token for registry-1.docker.io                                                           0.0s
 => [internal] load .dockerignore                                                                                                        0.0s
 => => transferring context: 60B                                                                                                         0.0s
 => CACHED [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a6  0.0s
 => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae  0.0s
 => CACHED [restic-builder 1/3] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def654  0.0s
 => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4            0.0s
 => [internal] load build context                                                                                                        0.2s
 => => transferring context: 561.69kB                                                                                                    0.2s
 => CACHED [velero-builder 2/4] WORKDIR /go/src/github.com/vmware-tanzu/velero                                                           0.0s
 => [velero-builder 3/4] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   2.2s
 => [restic-builder 2/3] COPY . /go/src/github.com/vmware-tanzu/velero                                                                   3.8s
 => [velero-builder 4/4] RUN mkdir -p /output/usr/bin &&     export GOARM=$( echo "" | cut -c2-) &&     go build -o /output/velero-res  11.7s
 => [restic-builder 3/3] RUN mkdir -p /output/usr/bin &&     export GOARM=$(echo "" | cut -c2-) &&     /go/src/github.com/vmware-tanzu/  9.4s
 => [stage-2 2/3] COPY --from=velero-builder /output /                                                                                   0.1s 
 => [stage-2 3/3] COPY --from=restic-builder /output /                                                                                   0.1s
 => exporting to image                                                                                                                   0.7s
 => => exporting layers                                                                                                                  0.6s
 => => exporting manifest sha256:added338a5b3f516ca59ff06a053c987abd11d5dfe44565385d6be94737cb591                                        0.0s
 => => exporting config sha256:f26c3643831e7df6379b746ee2429e6a8dcbdf3ec220e830e83999bfc329138c                                          0.0s
 => => exporting attestation manifest sha256:aaa90d2e1055a79d71d33c756e25f2bdddc493ab2ab7d56e901a8df1503ab326                            0.0s
 => => exporting manifest list sha256:4f3bb657a22d2e90e497ec1ed917542b7c045d0d1fdde5f2bc577a247393f092                                   0.0s
container: velero/velero-restore-helper:main
make all-containers  0.93s user 1.01s system 2% cpu 1:28.55 total

Dockerfile

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com> break up velero-builder stage to allow more concurrent layer builds. Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com> Add restic caching if fix_restic_cve.txt changes, only the dep added during patch will require redownload. ``` ❯ make container [+] Building 64.5s (20/20) FINISHED docker-container:colima-multiplat => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 3.24kB 0.0s => [internal] load metadata for docker.io/paketobuildpacks/run-jammy-tiny:latest 0.1s => [internal] load metadata for docker.io/library/golang:1.22-bookworm 0.1s => [internal] load .dockerignore 0.0s => => transferring context: 60B 0.0s => [stage-2 1/3] FROM docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae 0.0s => => resolve docker.io/paketobuildpacks/run-jammy-tiny:latest@sha256:6a3b9424b0616c4c37309b3f3e410f305ad75c484ab74b0a624d22c8a33ae5ae 0.0s => [restic-builder 1/5] FROM docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4 0.0s => => resolve docker.io/library/golang:1.22-bookworm@sha256:475ff60e52faaf037be2e7a1bc2ea5ea4aaa3396274af3def6545124a18b99b4 0.0s => [internal] load build context 0.3s => => transferring context: 546.21kB 0.3s => CACHED [restic-builder 2/5] RUN if [ "velero" = "velero" ]; then mkdir -p /build/restic && cd /build/restic && git clone --single-branch -b v0.15.0 https://github.com/restic/restic.git . && go mod download; fi 0.0s => [restic-builder 3/5] COPY hack/fix_restic_cve.txt /go/src/github.com/vmware-tanzu/velero/hack/ 0.0s => CACHED [velero-builder 2/6] WORKDIR /go/src/github.com/vmware-tanzu/velero 0.0s => CACHED [velero-builder 3/6] COPY go.mod go.sum /go/src/github.com/vmware-tanzu/velero/ 0.0s => CACHED [velero-builder 4/6] RUN go mod download 0.0s => [velero-builder 5/6] COPY . /go/src/github.com/vmware-tanzu/velero 6.5s => [restic-builder 4/5] RUN if [ "velero" = "velero" ]; then mkdir -p /output/usr/bin && cd /build/restic && git apply /go/src/github.com/vmware-tanzu/velero/hack/fix_restic_cve.txt && go mod download; fi 7.2s => [velero-builder 6/6] RUN mkdir -p /output/usr/bin && export GOARM=$( echo "" | cut -c2-) && go build -o /output/velero -ldflags "-X github.com/vmware-tanzu/velero/pkg/buildinfo.Version=main -X github.com/vmware-tanzu/velero/pkg/buildinfo.GitSHA=590f6df48d3fdc 55.5s => [restic-builder 5/5] RUN if [ "velero" = "velero" ]; then cd /build/restic && GOARM=$(echo "" | cut -c2-) go run build.go --goos "linux" --goarch "amd64" --goarm "" -o /output/usr/bin/restic && chmod +x /output/usr/bin/restic; fi && go cle 30.0s => CACHED [stage-2 2/3] COPY --from=velero-builder /output / 0.0s => CACHED [stage-2 3/3] COPY --from=restic-builder /output / 0.0s => exporting to docker image format 0.7s => => exporting layers 0.0s => => exporting manifest sha256:d275186bbf7b7fb08da8fe957037d8600c7b96693fe59bdfb119d977058daaa0 0.0s => => exporting config sha256:5cb88b0eac0b14df3efd549e0ca4df1980501a5f5cea677f0de1f5dd97050141 0.0s => => sending tarball 0.7s => importing to docker 0.0s container: velero/velero:main ``` Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

github-actions bot added kind/changelog-not-required PR does not require a user changelog. Often for docs, website, or build changes Website non-docs changes for the website labels Nov 12, 2024

kaovilai closed this Nov 12, 2024

kaovilai reopened this Nov 12, 2024

kaovilai force-pushed the cache-mod-deps branch from c29a40e to 590f6df Compare November 12, 2024 21:36

kaovilai marked this pull request as ready for review November 12, 2024 21:36

github-actions bot assigned kaovilai Nov 12, 2024

github-actions bot requested review from Lyndon-Li and reasonerjt November 12, 2024 21:37

blackpiglet reviewed Nov 14, 2024

View reviewed changes

Dockerfile Show resolved Hide resolved

kaovilai force-pushed the cache-mod-deps branch 2 times, most recently from 0b3f013 to 4edd799 Compare November 14, 2024 18:35

kaovilai commented Nov 14, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

kaovilai force-pushed the cache-mod-deps branch 2 times, most recently from 2789858 to 8781e95 Compare November 15, 2024 04:18

blackpiglet reviewed Nov 15, 2024

View reviewed changes

hack/build-restic.sh Show resolved Hide resolved

kaovilai force-pushed the cache-mod-deps branch 3 times, most recently from fdf8ba0 to cd4423e Compare November 15, 2024 07:29

This comment was marked as outdated.

Sign in to view

kaovilai force-pushed the cache-mod-deps branch from cd4423e to e465159 Compare November 15, 2024 10:22

reasonerjt removed their request for review November 18, 2024 13:01

Lyndon-Li reviewed Nov 19, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

kaovilai force-pushed the cache-mod-deps branch from e465159 to 19aee84 Compare November 20, 2024 02:11

kaovilai force-pushed the cache-mod-deps branch from 19aee84 to 1051dee Compare November 20, 2024 02:15

kaovilai requested a review from Lyndon-Li November 20, 2024 02:18

kaovilai requested a review from blackpiglet November 20, 2024 02:18

kaovilai removed the Website non-docs changes for the website label Nov 26, 2024

shubham-pampattiwar previously approved these changes Dec 3, 2024

View reviewed changes

Merge branch 'main' into cache-mod-deps

6be30f8

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

kaovilai dismissed shubham-pampattiwar’s stale review via 6be30f8 January 23, 2025 17:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache module dependencies between container builds#8400

Cache module dependencies between container builds#8400
kaovilai wants to merge 2 commits intovmware-tanzu:mainfrom
kaovilai:cache-mod-deps

kaovilai commented Nov 12, 2024 •

edited

Loading

Uh oh!

kaovilai commented Nov 12, 2024

Uh oh!

codecov bot commented Nov 12, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

kaovilai commented Nov 15, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

kaovilai commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Please add a summary of your change

Does your change fix a particular issue?

Please indicate you've done the following:

Uh oh!

kaovilai commented Nov 12, 2024

Uh oh!

codecov bot commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

kaovilai commented Nov 15, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kaovilai commented Nov 12, 2024 •

edited

Loading

codecov bot commented Nov 12, 2024 •

edited

Loading