python3 Support

[Jwink3101]

python2 is EOL Jan 1, 2020.

I am going to see what is involved in updating git-fat but even if I can't it is good to have an issue open to track progress

[Jwink3101]

I am extremely close to having this done but I am having issues with 2.6. It may be that I do not have the best test platform for it but I am still digging in.

With that said, I argue that we really do not need to support 2.6 anymore. I think that is an acceptable break.

Thoughts?

I'll keep trying to fix it in the mean time

[Jwink3101]

Just another followup,

My changes don't break 2.6 per se. I couldn't get it to work with 2.6 even with the current version!

Seems to be a known issue (see #46). I guess it is therefore out of scope here.

And there really is no need to support 2.6!

[jrollins]

Is this project dead? I hope not. I like the simplicity of the model compared to other large file support systems.

[Jwink3101]

I haven't had the motivation to go back and fix the few things.

I developed an alternative with a few more features but I lost steam. I've considered parring it down and releasing it. It's basically like git-fat but uses fewer command chains and does more logic in Python. And the biggest thing is that it lets you keep data on the server only. And support rclone

Are you interested? I haven't found the time to do it but it shouldn't be to hard to get to early beta.

[jrollins]

I'm certainly curious. I just found this:

https://github.com/ciena-blueplanet/git-fat

but it also appears to be defunct :(

[jedbrown]

Hi, sorry to be unresponsive. The simple state is that this project's test suite isn't sufficient for me to be able to quickly review PRs and I'm stretched super thin these days. I'm not using git-fat in any current projects and I feel it's more important to ensure that all former users continue to be supported without surprises than to deliver new features. Yes, Python-3 support is super overdue, and I'd definitely have finished if I were using git-fat.

The best case scenario in my opinion would be for someone else to take over maintainership. Second best would be to solicit contributions that improve testing.

[leggewie]

@jedbrown Thank you for creating git-fat. I strongly believe it has a place.

You say the test suite is insufficient. I'd love to help, but would need some mentoring. You've already approved #95 which apparently improves the test suite. What's holding it back from being merged?

[malaybasu]

Can someone take over this project? Python3 support is urgently needed. I don't know enough python to take over. Otherwise I would have. Git-fat is the simplest way to large files with git.

[Jwink3101]

I had pull request #95 but didn't finish. I kind of lost steam and it didn't feel worth it.

I also started my own project that was heavily inspired by git-fat but worked with rclone. It was pretty much down but I decided to abandon it since there were some edge cases I didn't want to deal with and it was large and complex.

I started a simpler one but also gave up.

The issue is, there is a lot of complexity and keeping up with the latest APIs. Plus, I kept finding that my use case didn't match others.

My Suggestion

I think the best answer is to use git-lfs with a custom transfer agent to git the best of both worlds. You get the support, documentation, etc of git-lfs with the custom storage of git-fat.

And there are a few options including an alpha/beta one I wrote!

• lfs-folderstore: Store data in a folder (probably a shared one). This is what inspired me to write one, Also used a lot
• lfsrclone. This is my tool but it is beta at best. I use it and it works. And rclone is perfect for this. You can access things via SFTP so you could use it where you used to use rsync. And there is no benefit to rsync anyway since files don't change
• git-lfs-rsync-agent: Also seems abandoned but may still work if you are dead-set on rsync.
• git-lfs-swift-transfer-agent: Abandoned but in python so I found it useful!

[malaybasu]

Thank you for your reply. When I tested last time, none of these plugin supported simple ssh. the 'rsync-agent' is simply flawed. It uses tmp directory to transfer the file completely ignoring the $TMPDIR variable. This breaks our server which has small tmp size. Which one has ssh support and a replacement for git-fat? Did not try your tool. But lsf-folderstore is not transparent and requires git configuration. Frankly, there is no replacement for git-fat I could find. git-lfs is simply evil. Also almost all of them will try to reconsitute the entire fat objects. Here are the crucial features I need:

1. Transparent config. A user should be able to use it right away about cloning the repository. git-fat allows the config files to be synced.
2. ssh support with no additional config
3. On demand file download at the individual file object level.
4. Transparent use using git GUIs

[malaybasu]

Has anyone tried using 2to3 to support python3?