Git SSH conflicting with OpenSSH integrated in Windows

[stefano-xy]

• I was not able to find an open or closed issue matching what I'm seeing

Git for Windows installs SSH binaries by default, together with a lot of other linux tools.

Starting from (at least) Windows 10 version 19.09, Microsoft integrated directly a version of OpenSSH client. For example, ssh-agent is a proper Windows Service, that seems to persist the keys somewhere in the user profile. I think this is even installed by default, while the corresponding OpenSSH server is also available for a manual installation. Both can be installed via the Manage Optional Features settings panel.

Get-Command -Name ssh*

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     ssh.exe                                            7.7.2.1    C:\WINDOWS\System32\OpenSSH\ssh.exe
Application     ssh-add.exe                                        7.7.2.1    C:\WINDOWS\System32\OpenSSH\ssh-add.exe
Application     ssh-agent.exe                                      7.7.2.1    C:\WINDOWS\System32\OpenSSH\ssh-agent.exe
Application     ssh-keygen.exe                                     7.7.2.1    C:\WINDOWS\System32\OpenSSH\ssh-keygen.exe
Application     ssh-keyscan.exe                                    7.7.2.1    C:\WINDOWS\System32\OpenSSH\ssh-keyscan.exe

The problem is that the OpenSSH integrated in Windows and what is shipped with Git for Windows don't talk to each other. Keys added to the ssh-agent Windows service are not looked up by git command from Git for Windows when working with ssh:// repositories. Depending on the order of entries in the PATH environment variable, one is used by default instead of another. On my installation, VS Code launches the Windows' ssh and expect keys to be available in the Windows' service. Using start-ssh-agent works, but that agent is not consulted.

The general solution is to configure GIT_SSH, pointing explicitly to the implementation wanted, but this is a manual process users have to discover first, and then do.

To avoid this overlap, I propose that an option is given not to install SSH together with Git for Windows. I can't suggest what its default setting should be, or if it's better to build a package without SSH, or else, but I think you should be aware of the problem and think about a better integration with recent versions of Windows 10.

Additional info: https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview

[dscho]

To avoid this overlap, I propose that an option is given not to install SSH together with Git for Windows.

That's a good idea. You could even give it a try:

1. install Git for Windows' SDK,
2. sdk cd installer,
3. edit install.iss (imitate the way Git LFS is handled, maybe?)
4. build a new installer via sdk build installer
5. verify that that installer works as expected
6. open a PR

[stefano-xy]

I could, but unfortunately I have no time to dedicate to implementing it myself. I think the community of Git for Windows should decide on the general approach, I have no knowledge of what's better, given the fact that you're working even on credential managers and this may overlap. Also an option 0.1% of the users will unclick is probably not the best, a more holistic approach should be thought.

I prefer the Git for Windows core team takes a decision.

[KalleOlaviNiemitalo]

In the past, I have tried configuring Git to use the OpenSSH installed by Windows. It was pretty easy to set up but tended to hang during Git operations. Perhaps that was caused by PowerShell/Win32-OpenSSH#1377. Anyway, I recommend postponing this until Windows upgrades its OpenSSH to a fixed version.

[internetfreak]

I ran into the same issue yesterday as I'm using the windows openSSH with a GnuPG smartcard and wondered that git doesn't pick my key provided by the agent. Configuring git via the environment variable or config variable works but it would be somehow better to tell git somehow to use the included binaries instead of its own (helps against forgetting to configure and wondering why it doesn'r work).
As for the issue linked in the comment above mine, this seems to be fixed, at least for me I did multiple pushes without issues.

[dscho]

I ran into the same issue yesterday as I'm using the windows openSSH with a GnuPG smartcard and wondered that git doesn't pick my key provided by the agent. Configuring git via the environment variable or config variable works but it would be somehow better to tell git somehow to use the included binaries instead of its own (helps against forgetting to configure and wondering why it doesn'r work).

Sure. This would have to be a new option in the installer. Meaning: someone with enough time on their hands needs to do it.

As for the issue linked in the comment above mine, this seems to be fixed, at least for me I did multiple pushes without issues.

Did you experience the issue at any time?

[mpdude]

👋🏼

I am the maintainer of https://github.com/webfactory/ssh-agent, a GitHub Actions (GHA) module to set up the SSH Agent with private keys for use in the GitHub Actions Build/CI/Automation service.

While trying to make this work in the Windows virtual environment as well, I came across the same problem that both sets of the SSH binaries seem to be incompatible or at least not "talk" to each other (webfactory/ssh-agent#63).

While I understand that the needs of such an GHA module are not the basis for your decision here, please take into consideration that the version of SSH provided by Windows seems to have some deficiencies. These include:

• It is not possible to select a particular key by pointing IdentityFile to a file containing the public key part (something, to my knowledge, might also be necessary when keys are on a smartcard?)
• Not loading keys from the agent when IdentitiesOnly=yes is set (ssh-agent is ignored when with "IdentitiesOnly yes" config file setting PowerShell/Win32-OpenSSH#1550)
• Not being able to read passphrases from stdin in automated/scripted situations; workaround is to set the DISPLAY env var to a fake value, so ssh-add forks and runs SSH_ASKPASS
• Tests for a terminal being present by looking for the C:\dev\tty file, which needs to be created as a workaround in some situations (Update _PATH_TTY to point to the console PowerShell/openssh-portable#447).

To me it seems that the version of ssh currently included by this project here (?) is newer and does not have these issues.

[stefano-xy]

The SSH agent provided by Windows has its problem, some of them fixed in upcoming versions, some not yet. I suggest you to report your findings to Microsoft on GitHub if not yet done. Note also the SSH agent is implemented as a Windows Service, so it seems to be quite different than the classic Linux ssh-agent.

Perfectly fine if Git for Windows keeps its own ssh-agent. My suggestion is just to introduce the (non-default?) option to not install it. If you open a command prompt and type ssh it uses the Windows one and looks up keys in the Windows agent anyway.

[mpdude]

report your findings to Microsoft on GitHub

Would that be on https://github.com/PowerShell/Win32-OpenSSH or https://github.com/PowerShell/openssh-portable?

[stefano-xy]

The README of the former forwards to the latter, so https://github.com/PowerShell/openssh-portable.

[mpdude]

But the README of the former also says...

This repo (https://github.com/PowerShell/Win32-OpenSSH) is being maintained to keep track of releases and issues.

(Emphasis mine 😉)