Tong Family

Well, here are some more notes about living with Windows. I do have to say, what pain to get this running, but the path of just using git-bash seems to work with a few caveats that are really annoying, but here are some tips, but I was getting a very obscure error message. It was really hard to figure out what was wrong, but tl;dr two deep bugs:

1. You cannot have certain characters like a colon (:) in a file name, git gets messed up because Windows doesn’t work.
2. There is a deep bug the Windows version of OpenSSH (v7.7) and you have to replace it with the Choco installation of V8.0 which involves much Powershell hackery to start the right SSHD agents. You do not want the native Windows installation
3. PowerShell is a very strange product, it is completely different from the shell sh scripts and derivatives and the cmd.exe of old windows, I did a lot of GGoogling to figure it out.

---

1. If you want to use ssh links like git@github.com:richtong/src instead of https, then you have to do some special magic. The first is that you have to start OpenSSH authentication on the Windows side of the house. So in Powershell side of the house, you need to make sure it is installed and running.
2. Then you have to make sure that the magic shell variable GIT_SSH is set properly and that you have pushed the OpenSSH version of ssh up high enough. Annoyingly, the default puts /usr/bin up way ahead of the Windows path names and there is no easy way to untangle that. It makes sense because most of the time you *do not* want the Windows executable, but the one supplied by MINGW64.

1. If you are debugging export GIT_TRACE=1 so that you can see what commands are being executed.
2. Also for the git submodule, you can add the --progress flag so you can see what is happening
3. The first thing discovered is that there are more restrictions on Windows file names than Linux/Mac ones. Specifially the ext4 and APFS systems vs NTFS.
4. If you are using git lfs, then the first git submodule is doing to take a LOOOONGG time on Windows like it is hung, but when you are running the trace command and progress, you can see that git-lfs gets to update times 100% and then stops.
5. And if you try to stop it and remove it, then some process hangs and you get a resource busy message. I think what is happening is that the git lfs implementation is downloading things but you can’t see it.
6. So a reboot is the only way that I can find to solve this

---

1. Then you if you are using Git LFS, you need to know that Windows NTFS is way more restrictive on file names that Linux/Mac, so that means that if a repo have an offending file name then really bad things happen. For me, this was a hang in git submodule update because git LFS has no idea how to deal with a long file name and gets trapped.

The only solution appears to be that you have to make your git repos safe for Windows and there are lots of rules here:

1. On a Mac, you can’t have a colon (:) in older file system with the new APFS you can use that but not the slash, Linux, you can’t have a forward slash (/), but on Windows, these are also not allowed, <>:"/\|?*
2. There are some reserved names in Windows from the early days which are primarily device names, like /dev/ names in Linux/Mac so CON, PRN, AUX, NUL, COM[1-9] an LPT[1-9]
3. The maximum file name length of a file name is 256 and path length is 32K bytes

As an example, the following name is illegal on Windows but works fine on Linux/Ma

course/deep1/data/Week 4/Deep Neural Network Application: Image Classification

But I think this is actually a problem with git lfs on Windows.

---

The next problem has to do with git lfs. This is a really strange one, but the command is hanging for some reason. Looks like an interaction with the Open SSH client which is noted in Git LFS Issue #4285 where git lfs hangs on clone (I’ve had that) and it hangs on push (Issue #3779). And the core issue is still open in Win32-OpenSSH which does not like stdin to be a null which is true in version 7.x.

I actually tried to make this work with the ssh that is included with MINGW64 and it does not interact properly with the git which is living in windows land. This makes sense because there are really two world here real windows and the Linux system. You can see this by doing a which git and then a which git-lfs and you can see git-lfs lives in MINGW64 while git lives in and it fails in PowerShell because ssh.exe does not like redirection from /dev/null

c:\Windows\System32\OpenSSH\ssh.exe -- 'git@github.com git-lfs-authenticate richtong/docker upload' </dev/null 2>/dev/null

And this finally led me to the right Github issue which has to od with this authentication and it is a bug in the OpenSSH side of the house.

There are few fixes for this. YOu cna actually change the core.sshCommand with beecause cat does not mind /dev/null as input. Also some has said that installing openssh with scoop install openssh fixes this by bumping up openssh. There is also a git-with-openssh there which is interesting and make sure you unset GIT_SSH in your profile and GIT_SSH_COMMAND

git config --global core.sshCommand 'cat - | /C/Windows/System32/OpenSSH/ssh'

There are a sea of OpenSSH out there for Windows, so a choco install OpenSSH installs version 7 but jscoop install openssh does not seem to work it reports an error

The confusing thing to install a newer version of OpenSSH, is that you first have to uninstall the Windows version and then install the Chocolaty version

# In the Powershell and hold your nose on the syntax
Get-Windows Capability -Online | ? Name -like 'OpenSSH*'
Remove-Windows-Capability -Online -Name OpenSSH.Client~~~~0.0.1.0
Remove-Windows-Capability -Online -Name OpenSSH.Serverj~~~~0.0.1.0
# Now to get the later version 8.x from Choco
choco install openssh -parmas ""/SSHServerFeature /KeyBasedAuthenticationFeature"" -y
# Note that this dumps ssh into a different directory 
# so in git-bash you need
# In powershell, this will not be on the path you can see this with
Get-Command ssh
# and you check the path with 
[System.Environment]::GetEnvironmentVariable('Path')
# or the shorter form
$ENV:PATH

Now Choco puts the client in a different dedicated directory and the problem is that, unlike, Linux, the standard directories are always ahead of the user added ones which you do not want, so while Choco does add C:\Windows\Porgram Files\OpenSSH-Win64 it is too far back.

There is no ~/.bash_profile in Windows, so what you have to do is to set the registry (I’m holding my nose as to how verbose and different PowerShell is from Linux)

# echo oldpath=$PATH becomes
$oldpath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHONE\System\CurrentControlSet\Control\Session Manager\Environment" -Name PATH).Path
# or you can use the simpler syntax
$oldpath = $env:path
# this is the same as PATH="/dnew:$PATH"
$newpath = "c:\Windows Program Files\OpenSSH-Win64;$oldpath"
# simpler syntax but this is not permanent
$env:path = "C:\Program Files\OpenSSH-Win64;$env:path"
# Need to set it here to make it work alway and make sure you are in Administrator RunAs mode
Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $env:path

One thing you notice in looking at the Windows path is that it is filled with duplicates. Things are not checking ahead of time and really you want the standard things to be last and not first

($env:path).split(";") > foo.txt
# then use vi or something to clean it up and then this rejoins it
# because Get Content returns a string with spaces and the -join
# tells the string to use semicolon as the delimiter
$env:path = (Get-Content foo.txt) -join ";"
# now commit to registry
Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -value $env:path

Note that this is way down in the session manager, so you will need actually reboot the machine to make it work.

Now that you have this installed, you need to install the daemons

c:\Program Files\OpenSSH-Win64\install-sshd.ps1
Set-Service SSHD -StartupType Automatic
Set-Service SSH-Agent -StartupType Automatic