Pushing the Docker image using FMP gives access denied

[arun-gupta]

Pushing the Docker image using FMP gives the following error:

services $ mvn fabric8:push -Pdocker
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] services
[INFO] name
[INFO] greeting
[INFO] webapp
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building services 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5.34:push (default-cli) @ services ---
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building name 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5.34:push (default-cli) @ name --
[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/arungupta/name]
0e014839ff57: Preparing   
f14ef5f204ff: Preparing   
85bb608219b7: Preparing   
160c15b8811e: Preparing   
498396e67e5c: Preparing   
2e544e952df4: Waiting     
e07090bc1b2d: Waiting     
c8af1cb1c035: Waiting     
64062a6d7af2: Waiting     
dfb51d70f8af: Waiting     
56939e74679c: Waiting     
d1be66a59bc5: Waiting     
[ERROR] F8> Unable to push 'arungupta/name' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] services ........................................... SUCCESS [  3.287 s]
[INFO] name ............................................... FAILURE [  1.382 s]
[INFO] greeting ........................................... SKIPPED
[INFO] webapp ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.026 s
[INFO] Finished at: 2018-02-13T15:22:44-08:00
[INFO] Final Memory: 35M/415M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5.34:push (default-cli) on project name: Unable to push 'arungupta/name' : denied: requested access to the resource is denied  -> [Help 1]

Pushing the image using Docker CLI works fine.

[ALRubinger]

@arun-gupta Help me with some env config so I can replicate this? Seeing:

$ mvn fabric8:push -Pdocker
...
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5.34:push (default-cli) on project services: Execution default-cli of goal io.fabric8:fabric8-maven-plugin:3.5.34:push failed: No <dockerHost> given, no DOCKER_HOST environment variable, no read/writable '/var/run/docker.sock' or '//./pipe/docker_engine' and no external provider like Docker machine configured -> [Help 1]

[arun-gupta]

Seems like you don't have Docker running in your environment?

Make sure Docker is running using one of the methods explained at https://store.docker.com/search?type=edition&offering=community. This is how FMP identifies the Docker host.

[ALRubinger]

@arun-gupta Nevermind; had just resetup Docker and forgot to add the group and permissions so I don't need to run as sudo. Now replicated as you've got:

[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5.34:push (default-cli) on project name: Unable to push 'arungupta/name' : An image does not exist locally with the tag: arungupta/name  -> [Help 1]

Investigating; thanks.

[ALRubinger]

Charting further progressing and now at the point where I've replicated Arun's same error:

[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/arungupta/name]
b18fc48878d6: Preparing   
f14ef5f204ff: Preparing   
85bb608219b7: Preparing   
160c15b8811e: Preparing   
498396e67e5c: Preparing   
2e544e952df4: Waiting     
e07090bc1b2d: Waiting     
c8af1cb1c035: Waiting     
64062a6d7af2: Waiting     
dfb51d70f8af: Waiting     
56939e74679c: Waiting     
d1be66a59bc5: Waiting     
[ERROR] F8> Unable to push 'arungupta/name' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]

What's required is running the steps 1) and 2) under Docker Images here: https://github.com/aws-samples/aws-compute-options and then running mvn fabric8:push -Pdocker instead of the 3) workaround noted

[ALRubinger]

@arun-gupta So I'm understanding the problem correctly, wouldn't I expect access denied here because I should instead be pushing to alrubinger/name, on Docker Hub from the retagged image in my local repo?

https://stackoverflow.com/questions/41984399/denied-requested-access-to-the-resource-is-denied-docker

I'm currently not seeing the option to specify the namespace in https://maven.fabric8.io/#fabric8:push; does this correctly summarize the root cause, do you believe?

[ALRubinger]

Hm, even when changing docker.repo:

F8> Unable to push 'alrubinger/name' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]

[ALRubinger]

OK, @arun-gupta : I got this passing by first docker login, then running mvn fabric8:push -Pdocker with docker.repo set to my own namespace.

Is this a valid workaround for you to use FMP - assuming you docker login first?

[kenfinnigan]

Thanks for diving into this @ALRubinger, really appreciated

[arun-gupta]

It is absolutely a valid workaround, as long as it works in my setup. Here are the commands issued in my terminal:

services $ mvn -pl greeting fabric8:push -Pdocker
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building greeting 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5.34:push (default-cli) @ greeting ---
[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/arungupta/greeting]
d2cda8e485d1: Preparing   
f14ef5f204ff: Preparing   
85bb608219b7: Preparing   
160c15b8811e: Preparing   
498396e67e5c: Preparing   
2e544e952df4: Waiting     
e07090bc1b2d: Waiting     
c8af1cb1c035: Waiting     
64062a6d7af2: Waiting     
dfb51d70f8af: Waiting     
56939e74679c: Waiting     
d1be66a59bc5: Waiting     
[ERROR] F8> Unable to push 'arungupta/greeting' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.623 s
[INFO] Finished at: 2018-02-15T04:54:07-08:00
[INFO] Final Memory: 34M/327M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5.34:push (default-cli) on project greeting: Unable to push 'arungupta/greeting' : denied: requested access to the resource is denied  -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
services $ mvn -pl greeting fabric8:push -Ddocker.repo=arungupta -Pdocker
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building greeting 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5.34:push (default-cli) @ greeting ---
[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/arungupta/greeting]
d2cda8e485d1: Preparing   
f14ef5f204ff: Preparing   
85bb608219b7: Preparing   
160c15b8811e: Preparing   
498396e67e5c: Preparing   
2e544e952df4: Waiting     
e07090bc1b2d: Waiting     
c8af1cb1c035: Waiting     
64062a6d7af2: Waiting     
dfb51d70f8af: Waiting     
56939e74679c: Waiting     
d1be66a59bc5: Waiting     
[ERROR] F8> Unable to push 'arungupta/greeting' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.863 s
[INFO] Finished at: 2018-02-15T04:54:32-08:00
[INFO] Final Memory: 35M/418M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5.34:push (default-cli) on project greeting: Unable to push 'arungupta/greeting' : denied: requested access to the resource is denied  -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
services $ docker push arungupta/greeting
The push refers to repository [docker.io/arungupta/greeting]
d2cda8e485d1: Pushed 
f14ef5f204ff: Layer already exists 
85bb608219b7: Layer already exists 
160c15b8811e: Layer already exists 
498396e67e5c: Layer already exists 
2e544e952df4: Layer already exists 
e07090bc1b2d: Layer already exists 
c8af1cb1c035: Layer already exists 
64062a6d7af2: Layer already exists 
dfb51d70f8af: Layer already exists 
56939e74679c: Layer already exists 
d1be66a59bc5: Layer already exists 
latest: digest: sha256:1279e63b5696157f813cdc9345f0b111c4a41ff4b3027e486238a9c765c6b5ba size: 2828

As you can see, docker push works fine but fabric8:push, with or without docker.repo does not. And in my pom.xml, docker.repo is already set to arungupta.

What could be the discrepancy in my setup? How should I further debug it?

[ALRubinger]

@arun-gupta I don't see where you issued docker login? Is your manual docker push set up with some config to send along your credentials without having to use login first?

Also this differs from the command I used by passing -pl to the mvn commands.

I can look again on my machine in a few hours, but yeah, dock login before running mvn is what made this start to work for me.

[ALRubinger]

@arun-gupta Yeah, check my traces below. First I am not logged into docker locally:

[alr@tubescreamer ~]$ docker logout 
Removing login credentials for https://index.docker.io/v1/

Then I attempt the same command you've provided me, which fails with familiar "requested access to the resource is denied" error:

✔ ~/business/jboss/git/aws-samples/aws-compute-options/services [alrubinger-fmp L|✚ 1] 
16:34 $ mvn -pl greeting fabric8:push -Pdocker
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building greeting 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5-SNAPSHOT:push (default-cli) @ greeting ---
[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/alrubinger/greeting]
dbbebd36d1c7: Preparing   
f14ef5f204ff: Preparing   
85bb608219b7: Preparing   
160c15b8811e: Preparing   
498396e67e5c: Preparing   
2e544e952df4: Waiting     
e07090bc1b2d: Waiting     
c8af1cb1c035: Waiting     
64062a6d7af2: Waiting     
dfb51d70f8af: Waiting     
56939e74679c: Waiting     
d1be66a59bc5: Waiting     
[ERROR] F8> Unable to push 'alrubinger/greeting' : denied: requested access to the resource is denied  [denied: requested access to the resource is denied ]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.682 s
[INFO] Finished at: 2018-02-15T16:35:00+00:00
[INFO] Final Memory: 34M/392M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:3.5-SNAPSHOT:push (default-cli) on project greeting: Unable to push 'alrubinger/greeting' : denied: requested access to the resource is denied  -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Then I run docker login:

✘-1 ~/business/jboss/git/aws-samples/aws-compute-options/services [alrubinger-fmp L|✚ 1] 
16:35 $ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: alrubinger
Password: 
Login Succeeded

Finally I try the same command again and the pushes go forward:

✔ ~/business/jboss/git/aws-samples/aws-compute-options/services [alrubinger-fmp L|✚ 1] 
16:35 $ mvn -pl greeting fabric8:push -Pdocker
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building greeting 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- fabric8-maven-plugin:3.5-SNAPSHOT:push (default-cli) @ greeting ---
[INFO] F8> Running generator wildfly-swarm
[INFO] F8> wildfly-swarm: Using Docker image fabric8/java-jboss-openjdk8-jdk:1.3 as base / builder
[INFO] F8> The push refers to repository [docker.io/alrubinger/greeting]
dbbebd36d1c7: Preparing   
f14ef5f204ff: Pushed      
85bb608219b7: Pushed      
160c15b8811e: Pushed      
498396e67e5c: Pushed      
2e544e952df4: Pushing     [==============================>                    ]  525.8kB/853kB
e07090bc1b2d: Pushed      
c8af1cb1c035: Pushed      
64062a6d7af2: Pushing     [>                                                  ]  1.078MB/203.1MB
dfb51d70f8af: Pushed      
56939e74679c: Pushing     [=>                                                 ]  593.4kB/28.13MB
d1be66a59bc5: Pushing     [>                                                  ]    555kB/203.5MB

[ALRubinger]

@arun-gupta Note that my environment is Fedora:

[alr@tubescreamer ~]$ uname -a
Linux tubescreamer 4.14.16-300.fc27.x86_64 #1 SMP Wed Jan 31 19:24:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

@kenfinnigan notes that he's on Mac and sees the same issue you do; FMP doesn't seem to be respecting the login creds on that platform. We're looking into it.

[ALRubinger]

So the contents of my:

[alr@tubescreamer ~]$ cat $HOME/.docker/config.json
{
	"auths": {
		"https://index.docker.io/v1/": {
			"auth": "$$$$_REDACTED_BY_ALR_FOR_SECURITY_$$$$"
		}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/17.12.0-ce (linux)"
	}
}

@kenfinnigan version has a credstore variable pointing to osxkeychain, so we're wondering if FMP is not keychain-aware and respecting the login.