Skip to content

FileName is a directory appears when downloading artifacts from minio #9908

@zcerhub

Description

@zcerhub

Pre-requisites

  • I have double-checked my configuration
  • I can confirm the issues exists when I tested with :latest
  • I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

Downloading artifacts from minio when retring no longer appears on fileName is a directory.

As you can see from the logs of the init container, the connection reset will be retried. the spec.templates[1].inputs.artifacts[0].s3.key is the directory.From the source code

func loadS3Artifact(s3cli argos3.S3Client, inputArtifact *wfv1.Artifact, path string) (bool, error) {
,we can see that the s3cli.GetFile method would throws a fileName is a directory error if path is an existing folder. This folder is created the first time s3cli.GetDirectory is called.

Version

v3.3.8

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow                  # new type of k8s spec
metadata:
  generateName: devops-test-    # name of the workflow spec
spec:
  entrypoint: devops-cicd-java          # invoke the whalesay template
  templates:
    - name: devops-cicd-java              # name of the template
      dag:
        tasks:
          - name: compilecode
            template: compilecode-task


    - name: compilecode-task
      inputs:
        artifacts:
          - name: devops-codecompile-source-s3
            path: '{{inputs.parameters.work-dir}}'
            s3:
              endpoint: xxxxxx
              bucket: xxxxxx
              insecure: true
              key: '827398648978857985/checkoutcode-task-0ee4a9f6-409b-413f-9ab8-efb24b446cf9/src/code'
              accessKeySecret:
                name: xxxxxx
                key: xxxxxx
              secretKeySecret:
                name: xxxxxx
                key: xxxxx
      container:
        image: nginx

Logs from the workflow controller

workflow controller logs don't matter

Logs from in your workflow's wait container

time="2022-10-25T06:25:52.899Z" level=info msg="Start loading input artifacts..."
time="2022-10-25T06:25:52.899Z" level=info msg="Downloading artifact: codebuildimage-s3"
time="2022-10-25T06:25:52.899Z" level=info msg="S3 Load path: /argo/inputs/artifacts/codebuildimage-s3.tmp, key: xxxxxxx/src/code/outputs"
time="2022-10-25T06:25:52.899Z" level=info msg="Creating minio client using static credentials" endpoint="xxxxxx"
time="2022-10-25T06:25:52.899Z" level=info msg="Getting file from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:52.911Z" level=info msg="Getting directory from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:52.911Z" level=info msg="Listing directory from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs
time="2022-10-25T06:25:53.733Z" level=info msg="Transient error: read tcp 172.40.62.4:51698->10.48.196.248:9000: read: connection reset by peer"
time="2022-10-25T06:25:55.097Z" level=info msg="S3 Load path: /argo/inputs/artifacts/codebuildimage-s3.tmp, key: xxxxxxx/src/code/outputs"
time="2022-10-25T06:25:55.097Z" level=info msg="Creating minio client using static credentials" endpoint="xxxxxx"
time="2022-10-25T06:25:55.097Z" level=info msg="Getting file from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:55.097Z" level=warning msg="Non-transient error: fileName is a directory."
time="2022-10-25T06:25:55.097Z" level=error msg="executor error: artifact codebuildimage-s3 failed to load: failed to get file: fileName is a directory."
time="2022-10-25T06:25:55.097Z" level=info msg="Alloc=6222 TotalAlloc=11034 Sys=18642 NumGC=3 Goroutines=3"
time="2022-10-25T06:25:55.097Z" level=fatal msg="artifact codebuildimage-s3 failed to load: failed to get file: fileName is a directory."

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low priority

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions