Skip to content

WIP: #14 Docker image to run the markdown files #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

WIP: #14 Docker image to run the markdown files #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b950ac9
#14 parallel release process
htimur Nov 30, 2022
d7bfb8a
#14 fixed the typo in outputs
htimur Nov 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 66 additions & 9 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,14 @@ on:
name: Release artifacts

jobs:
release:
name: Compile and release
compile_linux_musl:
name: Compile x86_64-unknown-linux-musl
runs-on: ubuntu-latest
outputs:
ARCHIVE: ${{ steps.linux_musl.outputs.BUILT_ARCHIVE }}
CHECKSUM: ${{ steps.linux_musl.outputs.BUILT_CHECKSUM }}
steps:
- uses: actions/checkout@master
- uses: actions/checkout@v3
- name: Compile target x86_64-unknown-linux-musl
id: linux_musl
uses: rust-build/[email protected]
Expand All @@ -21,6 +24,23 @@ jobs:
ARCHIVE_TYPES: "tar.gz tar.xz tar.zst"
UPLOAD_MODE: none
EXTRA_FILES: "README.md LICENSE"
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: linux_musl
retention-days: 7
path: |
${{ steps.linux_musl.outputs.BUILT_ARCHIVE }}
${{ steps.linux_musl.outputs.BUILT_CHECKSUM }}

compile_apple:
name: Compile x86_64-apple-darwin
runs-on: ubuntu-latest
outputs:
ARCHIVE: ${{ steps.apple.outputs.BUILT_ARCHIVE }}
CHECKSUM: ${{ steps.apple.outputs.BUILT_CHECKSUM }}
steps:
- uses: actions/checkout@v3
- name: Compile target x86_64-apple-darwin
id: apple
uses: rust-build/[email protected]
Expand All @@ -30,6 +50,22 @@ jobs:
RUSTTARGET: "x86_64-apple-darwin"
ARCHIVE_TYPES: zip
UPLOAD_MODE: none
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: apple
retention-days: 7
path: |
${{ steps.apple.outputs.BUILT_ARCHIVE }}
${{ steps.apple.outputs.BUILT_CHECKSUM }}
compile_windows:
name: Compile x86_64-pc-windows-gnu
runs-on: ubuntu-latest
outputs:
ARCHIVE: ${{ steps.windows.outputs.BUILT_ARCHIVE }}
CHECKSUM: ${{ steps.windows.outputs.BUILT_CHECKSUM }}
steps:
- uses: actions/checkout@v3
- name: Compile target x86_64-pc-windows-gnu
id: windows
uses: rust-build/[email protected]
Expand All @@ -39,21 +75,42 @@ jobs:
RUSTTARGET: "x86_64-pc-windows-gnu"
ARCHIVE_TYPES: zip
UPLOAD_MODE: none
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: windows
retention-days: 7
path: |
${{ steps.windows.outputs.BUILT_ARCHIVE }}
${{ steps.windows.outputs.BUILT_CHECKSUM }}
release:
name: Compile x86_64-pc-windows-gnu
runs-on: ubuntu-latest
needs:
- compile_linux_musl
- compile_apple
- compile_windows
steps:
- uses: actions/checkout@v3
- name: "✏️ Generate release changelog"
id: changelog
uses: heinrichreimer/[email protected]
with:
token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/download-artifact@v3
name: Download artifacts
- name: Display structure of downloaded files
run: ls -R
- name: Release
uses: softprops/action-gh-release@v1
with:
body: ${{steps.changelog.outputs.changelog}}
files: |
${{ steps.linux_musl.outputs.BUILT_ARCHIVE }}
${{ steps.linux_musl.outputs.BUILT_CHECKSUM }}
${{ steps.apple.outputs.BUILT_ARCHIVE }}
${{ steps.apple.outputs.BUILT_CHECKSUM }}
${{ steps.windows.outputs.BUILT_ARCHIVE }}
${{ steps.windows.outputs.BUILT_CHECKSUM }}
${{ needs.jobs.compile_linux_musl.outputs.ARCHIVE }}
${{ needs.jobs.compile_linux_musl.outputs.CHECKSUM }}
${{ needs.jobs.compile_apple.outputs.ARCHIVE }}
${{ needs.jobs.compile_apple.outputs.CHECKSUM }}
${{ needs.jobs.compile_windows.outputs.ARCHIVE }}
${{ needs.jobs.compile_windows.outputs.CHECKSUM }}
README.md
LICENSE
10 changes: 9 additions & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,12 @@ sys-info = "0.9.1"
anyhow = "1.0.65"
clap = { version = "4.0.26", features = ["derive"] }
clap_complete = "4.0.5"
clap_mangen = "0.2.4"
clap_mangen = "0.2.4"


[profile.release]
# Enable Link Time Optimization
lto = true
codegen-units = 1
opt-level = 'z'
panic = "abort"
3 changes: 3 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM alpine:latest

COPY linux_musl linux_musl
66 changes: 41 additions & 25 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
</p>

# markdown-gem, a.k.a _gem_ - markdown code chunks executor

[![build](https://github.com/murabi-io/markdown-gem/actions/workflows/ci.yml/badge.svg)](https://github.com/murabi-io/markdown-gem/actions/workflows/ci.yml)
[![license](https://img.shields.io/github/license/murabi-io/markdown-gem)](https://github.com/murabi-io/markdown-gem/blob/main/LICENSE)
[![crates](https://img.shields.io/crates/v/markdown-gem.svg)](https://crates.io/crates/markdown-gem)


Inspired by [R Markdown code chunks](https://bookdown.org/yihui/rmarkdown/r-code.html), but for any code.

Our goal is to make markdown documentation, examples, instructions, or anything related to the code in your MD files - provable, easily maintainable, and integrate it into your CI pipelines.
Expand All @@ -17,23 +17,29 @@ Our goal is to make markdown documentation, examples, instructions, or anything
## Install

### From precompiled binaries

Binaries are made available at every [GitHub release](https://github.com/murabi-io/markdown-gem/releases).

The archives there contain all precompiled versions, as well as the licenses and other files.
> When you download executable files, you'll have to ensure the shell can find them. An easy solution on Linux is for example to put them in /usr/local/bin. You may also have to set them executable using `chmod +x markdown-gem`.

> When you download executable files, you'll have to ensure the shell can find them. An easy solution on Linux is for example to put them in /usr/local/bin. You may also have to set them executable using `chmod +x markdown-gem`.

### From crates.io

You'll need to have the [Rust development environment](https://rustup.rs/) installed and up to date.

Once it's installed, use cargo to install lfs:

```shell {cmd=sh}
cargo install markdown-gem
```

### From source

You'll need to have the [Rust development environment](https://rustup.rs/) installed.

Fetch the [murabi-io/markdown-gem](https://github.com/murabi-io/markdown-gem) repository, move to the `markdown-gem` directory, then run

```shell {cmd=sh}
git clone [email protected]:murabi-io/markdown-gem.git
cd markdown-gem
Expand All @@ -42,48 +48,58 @@ cargo install --path .

## Why use markdown-gem

* ***automate*** and **_document_** at the same time
* ***maintain*** your documentation and make sure the examples are working
* ***improve*** your user experience
- **_automate_** and **_document_** at the same time
- **_maintain_** your documentation and make sure the examples are working
- **_improve_** your user experience

## Code chunks

A code chunk is a code that is part of your markdown and you want it executed, e.g.
~~~

````
```sh {sys=[linux], linux_distro=[debian]}
apt install ...
```
~~~
````

The code chunk above defines a shell script to install something using `apt`, and it'll run only on **Linux** systems of the **Debian** family.
The metadata of the code chunk is ignored by most renderers and users will only see the shell code containing the `apt...` part.

### Attributes and filters

The attributes of the code chunk give the executor information on how to run the code, and filters define when not to run it.

#### Available attributes
| Attribute | Type | Optional/Default | Description |
|----------------|--------------------|------------------|---------------------------------------------------------------------------|
| cmd | string | no* | command name or path, e.g. `sh`, `node` and etc. |
| args | array of arguments | yes | command arguments |
| path | string | yes | `PATH` env variable for the command |
| as_file | boolean | yes/true | determines if markdown-gem should execute the code chunk as a file, default `true` |
| stdout | boolean | yes/true | determines if markdown-gem should display stdout of the code chunk, default `true` |
| allow_warnings | boolean | yes/true | determines if markdown-gem should allow warnings, default `true` |
| allow_errors** | boolean | yes/false | determines if markdown-gem should TODO: allow errors, default `true` |
| with_sudo*** | boolean | yes/false | tells markdown-gem to run the code chunk in sudo |

| Attribute | Type | Optional/Default | Description |
| ---------------- | ------------------ | ---------------- | ---------------------------------------------------------------------------------- |
| cmd | string | no\* | command name or path, e.g. `sh`, `node` and etc. |
| args | array of arguments | yes | command arguments |
| path | string | yes | `PATH` env variable for the command |
| as_file | boolean | yes/true | determines if markdown-gem should execute the code chunk as a file, default `true` |
| stdout | boolean | yes/true | determines if markdown-gem should display stdout of the code chunk, default `true` |
| allow_warnings | boolean | yes/true | determines if markdown-gem should allow warnings, default `true` |
| allow_errors\*\* | boolean | yes/false | determines if markdown-gem should TODO: allow errors, default `true` |
| with_sudo\*\*\* | boolean | yes/false | tells markdown-gem to run the code chunk in sudo |

> - \* the implementation of the default commands by code chunk lang attribute will make this attribute optional
> - ** the functionality for `allow_errors` is not there yet
> - *** sudo support is not available yet, you can still execute `gem` under sudo, but keep in mind that all code chunks will inherit the sudo privileges
> - \*\* the functionality for `allow_errors` is not there yet
> - \*\*\* sudo support is not available yet, you can still execute `gem` under sudo, but keep in mind that all code chunks will inherit the sudo privileges

#### Available filters
| Filter | Type | Optional/Default | Inclusive/Exclusive | Description |
|--------------|------------------|--------------------|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| sys | array of strings | yes/[] | inclusive | os system filter, possible values: `linux`, `macos`, `ios`, `freebsd`, `dragonfly`, `netbsd`, `openbsd`, `solaris`, `android`, `windows` |
| arch | array of strings | yes/[] | inclusive | system architecture filter, possible values: `x86`, `x86_64`, `arm`, `aarch64`, `m68k`, `mips`, `mips64`, `powerpc`, `powerpc64`, `riscv64`, `s390x`, `sparc64` |
| linux_distro | array of strings | yes/[] | inclusive | linux distro filter, e.g. `arch`, `debian` and etc, derived from linux release ID_LIKE |

| Filter | Type | Optional/Default | Inclusive/Exclusive | Description |
| ------------ | ---------------- | ---------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| sys | array of strings | yes/[] | inclusive | os system filter, possible values: `linux`, `macos`, `ios`, `freebsd`, `dragonfly`, `netbsd`, `openbsd`, `solaris`, `android`, `windows` |
| arch | array of strings | yes/[] | inclusive | system architecture filter, possible values: `x86`, `x86_64`, `arm`, `aarch64`, `m68k`, `mips`, `mips64`, `powerpc`, `powerpc64`, `riscv64`, `s390x`, `sparc64` |
| linux_distro | array of strings | yes/[] | inclusive | linux distro filter, e.g. `arch`, `debian` and etc, derived from linux release ID_LIKE |

## Security

The code chunks are executed on the host machine without any pre-checks in a form provided in the markdown documents. The execution is done through `tokio::process::Command` and the detailed documentation can be found here: [tokio::process::Command](https://docs.rs/tokio/latest/tokio/process/struct.Command.html).

> You should perceive running code chunks as running any other shell script, with all the inherent dangers.

## License and Acknowledgements
The implementation is based on [termimad](https://github.com/Canop/termimad) and few other projects. I would like to thank [Denys Séguret](https://github.com/Canop) and [crossterm-rs](https://github.com/crossterm-rs) for their work and the existing open source code I used as inspiration.

The implementation is based on [termimad](https://github.com/Canop/termimad) and few other projects. I would like to thank [Denys Séguret](https://github.com/Canop) and [crossterm-rs](https://github.com/crossterm-rs) for their work and the existing open source code I used as inspiration.