Replace pre-commit with prek

[potiuk]

As @ashb noticed there is a new drop-in replacement for pre-commit in town written in rust - with presumably less opinionated choices (and nicer features) than pre-commit. It's not "ready yet" - it does not have everything implemented yet, but looks promising.

https://github.com/j178/prefligit

We might want to take a look at figure out what is missing and replace pre-commit with it when we think it's ready

[potiuk]

Current status - it fails wit:

[jarek:~/code/airflow] main+ 101 ± RUST_BACKTRACE=1 prefligit run   
⠙ Initializing hooks...                                                                                                                                                                                                                                                                                                                                                                       thread 'main' panicked at src/languages/mod.rs:47:18:
not yet implemented
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: prefligit::hook::HookBuilder::combine
   4: prefligit::hook::Project::init_hooks::{{closure}}
   5: prefligit::cli::run::run::run::{{closure}}
   6: prefligit::run::{{closure}}
   7: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
   8: tokio::runtime::runtime::Runtime::block_on
   9: prefligit::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[jarek:~/code/airflow] main+ 101 ± 

[ashb]

Shell completion!

https://github.com/j178/prefligit/blob/921f86411cc2de04dbf94c696d572665bfca467f/src/main.rs#L235

[potiuk]

Shell completion!

https://github.com/j178/prefligit/blob/921f86411cc2de04dbf94c696d572665bfca467f/src/main.rs#L235

🚀

That's what I was sincerely hoping for. This is main reason why we have breeze static-checks, another one are some predefined flags like --last-comit and --only-my-changes which I hope we might be able to contribute to prefligit once we get it working :)

[j178]

Hey, I’m the author of prefligit 👋🏻, just wanted to stop by and share the latest updates:

prefligit now fully supports Python and Node, and we're adding Go support soon! So, it won’t be long before you can run all your airflow hooks through prefligit.

And here are some cool benchmark numbers: for a cold installation of airflow hooks, prefligit is about 10x faster than pre-commit and takes up only a third of the disk space.

That said, prefligit isn’t production-ready yet. I’d love for airflow to be the first big adaptor and would really appreciate any help with testing! Let me know if you’re interested!

[ashb]

@j178 Love it! I'm happy to drive this locally at any point you tell me you think it's good enough for local testing.

[j178]

@ashb Hi! Just wanted to let you know that with the prefligit 0.0.20 release, all languages that Airflow uses are now supported. I think it’s a good time for you to give it a try!

[potiuk]

👀

[potiuk]

Cooo. I am also looking forward to TODO! monorepo support. That would help us to modularise our pre-commits way, way better.

[potiuk]

I tried it with Airflow current .pre-commit config and we still have problem with node pre-commits

When I had no node installed at all on my Linux on system level I got:

[jarek:~/code/airflow] main+ ± prefligit run --all-files
error: Failed to install hook `compile-ui-assets`
  caused by: command `npm install` exited with an error:

[status]
exit status: 127

[stderr]
/usr/bin/env: ‘node’: No such file or directory

I installed npm using regular apt (mint 21.3 cinnamon):

 sudo apt install npm 

Then I get this:

[jarek:~/code/airflow] main+ ± prefligit run --all-files
error: Failed to install hook `lint-markdown`
  caused by: command `npm install` exited with an error:

[status]
exit status: 1

[stderr]
internal/modules/cjs/loader.js:818
throw err;
^
Error: Cannot find module 'node:path'
Require stack:
- /home/jarek/.cache/prefligit/tools/node/22.18.0-Jod/lib/node_modules/npm/lib/cli.js
- /home/jarek/.cache/prefligit/tools/node/22.18.0-Jod/lib/node_modules/npm/bin/npm-cli.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
at Function.Module._load (internal/modules/cjs/loader.js:667:27)
at Module.require (internal/modules/cjs/loader.js:887:19)
at require (internal/modules/cjs/helpers.js:85:18)
at Object.<anonymous> (/home/jarek/.cache/prefligit/tools/node/22.18.0-Jod/lib/node_modules/npm/lib/cli.js:10:18)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Module.require (internal/modules/cjs/loader.js:887:19) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/home/jarek/.cache/prefligit/tools/node/22.18.0-Jod/lib/node_modules/npm/lib/cli.js',
'/home/jarek/.cache/prefligit/tools/node/22.18.0-Jod/lib/node_modules/npm/bin/npm-cli.js'
]
}

I think prefligit should install complete node environment even if you do not have one installed (this is what pre-commit does) - we actually use that feature in our breeze environment, we are using the fact that pre-commit will install the specified node version from scratch under-the-hood when it is used in .pre-commit-config.yml. We have a number of people who do not do UI development, don't have node installed and we do not want them to bother about it when we need to compile assets when locally installing airlfow for development. The nice thing about node setup done by pre-commit is that it just works.

[potiuk]

Also this:

node --version
v12.22.9