Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
159 commits
Select commit Hold shift + click to select a range
b9ee40e
Initial commit, already contains the structure to work on the first t…
klust Mar 7, 2022
183ef3b
Updated the "What is EasyBuild" page of the tutorial for LUMI."
klust Mar 7, 2022
6541e8c
Updated the structure, initial updates of terminology and a new page …
klust Mar 9, 2022
b2a55cd
Additions to the Lmod section and some restructuring, and better info…
klust Mar 10, 2022
1692a80
Corrections to the README file.
klust Mar 10, 2022
82dbfae
Corrections to the overview for part I.
klust Mar 10, 2022
29d5ba5
Further work on the Lmod section.
klust Mar 11, 2022
2eae491
Finished the Lmod section.
klust Mar 14, 2022
8ca50b8
Continued merging of the new CSC tutorial in the structure of the rep…
klust Mar 14, 2022
71966b5
Tutorial page about the Cray PE, and correction of a typo.
klust Mar 14, 2022
91e7c86
Corrected a typo in mkdocs.yml
klust Mar 16, 2022
766a0f2
Adding an example of implementing a hierarchy to the Lmod section of …
klust Mar 16, 2022
70889a5
More information about hierarchy in the Cray PE, and definition of to…
klust Mar 21, 2022
e101d7b
Proposed change to mkdocs.yml to support mermaid.js graphs.
klust Mar 21, 2022
df081ae
Add a mermaid.js diagram with the toolchain hierarchy.
klust Mar 21, 2022
0fd57c5
Finished terminology session, extended the installation section with …
klust Mar 21, 2022
2aee856
Tutorial on EasyBuild configuration added, and some restructuring.
klust Mar 21, 2022
53d564d
Restructuring continued.
klust Mar 21, 2022
fea45d2
Finished reworking the basic usage section, except for the exercises.
klust Mar 30, 2022
c844b96
Additions to the LUMI software stack page of the tutorial.
klust Mar 30, 2022
cbdb67e
Integrating parts of the old tutorial into the new one, texts still n…
klust Mar 30, 2022
8d6b6f3
Troubleshooting section reworked for LUMI.
klust Mar 30, 2022
51c7e3d
Section on creating easyconfig files.
klust Apr 1, 2022
c14df7a
Adding in an additional section about external modules.
klust Apr 1, 2022
f89ee1f
Adding in a section taken from a previous tutorial and integrating so…
klust Apr 1, 2022
28741b4
Implementing EasyBlocks adapted for LUMI.
klust Apr 5, 2022
5f5898e
Correction of typos.
klust Apr 5, 2022
bbe836e
Using EasyBuild as a library corrected for LUMI.
klust Apr 5, 2022
272f2fe
Section about hooks extended with references to additional examples.
klust Apr 5, 2022
3234fd9
Slurm job submission from EasyBuild a bit reworked.
klust Apr 6, 2022
ad2a7f5
Some explanation in the overview of part 3
klust Apr 6, 2022
4f1dcab
GitHub integration section, mostly old text and not all suited for LUMI.
klust Apr 6, 2022
83d63d8
Adapt the structure and include an additional reading section.
klust Apr 6, 2022
d1aee2d
Additional reading section.
klust Apr 6, 2022
3cf8a3c
Correcting a number of links.
klust Apr 6, 2022
7b1f470
Restructuring for nicer navigation bar on the left.
klust Apr 7, 2022
724e059
Link corrections.
klust Apr 7, 2022
64775be
Correction of links.
klust Apr 7, 2022
d915944
Corrected a number of spelling mistakes.
klust Apr 8, 2022
22772b6
Removed a new line from a module file as it caused problems with the …
klust Apr 8, 2022
6ea43a8
Removed some TODOs to complete the tutorial.
klust Apr 21, 2022
d1f6c7b
Corrected and updated several links.
klust Apr 21, 2022
8e0353a
Various corrections (typos etc.) and minor additions.
klust May 9, 2022
dc0895f
Reworked the exercises of the troubleshooting section.
klust May 9, 2022
a9e0d3e
Reworked the exercises for the basic usage section.
klust May 9, 2022
d851ae1
Multiple minor corrections.
klust May 10, 2022
8f29ae4
Corrected two typos.
klust May 10, 2022
8f4dd15
Corrections to the example
klust May 11, 2022
430f584
Removed an unnecessary accent.
klust May 11, 2022
10a3be8
Corrected wrong termination of code block.
klust May 11, 2022
728e972
Initial commit, already contains the structure to work on the first t…
klust Mar 7, 2022
f20177b
Updated the "What is EasyBuild" page of the tutorial for LUMI."
klust Mar 7, 2022
df7f0f0
Updated the structure, initial updates of terminology and a new page …
klust Mar 9, 2022
d873a20
Additions to the Lmod section and some restructuring, and better info…
klust Mar 10, 2022
78d54a5
Corrections to the README file.
klust Mar 10, 2022
48d6d02
Corrections to the overview for part I.
klust Mar 10, 2022
04b642e
Further work on the Lmod section.
klust Mar 11, 2022
71b0dcb
Finished the Lmod section.
klust Mar 14, 2022
46fe516
Continued merging of the new CSC tutorial in the structure of the rep…
klust Mar 14, 2022
ec860bd
Tutorial page about the Cray PE, and correction of a typo.
klust Mar 14, 2022
8764aa3
Adding an example of implementing a hierarchy to the Lmod section of …
klust Mar 16, 2022
e24aafa
More information about hierarchy in the Cray PE, and definition of to…
klust Mar 21, 2022
4d47bb7
Add a mermaid.js diagram with the toolchain hierarchy.
klust Mar 21, 2022
dac3c94
Finished terminology session, extended the installation section with …
klust Mar 21, 2022
99a46fe
Tutorial on EasyBuild configuration added, and some restructuring.
klust Mar 21, 2022
6a59f94
Restructuring continued.
klust Mar 21, 2022
e277aa5
Finished reworking the basic usage section, except for the exercises.
klust Mar 30, 2022
a8c9b04
Additions to the LUMI software stack page of the tutorial.
klust Mar 30, 2022
7570644
Integrating parts of the old tutorial into the new one, texts still n…
klust Mar 30, 2022
abfbf61
Troubleshooting section reworked for LUMI.
klust Mar 30, 2022
5881892
Section on creating easyconfig files.
klust Apr 1, 2022
0ba449e
Adding in an additional section about external modules.
klust Apr 1, 2022
a778c98
Adding in a section taken from a previous tutorial and integrating so…
klust Apr 1, 2022
ca73ca7
Implementing EasyBlocks adapted for LUMI.
klust Apr 5, 2022
bdc3324
Correction of typos.
klust Apr 5, 2022
87138ea
Using EasyBuild as a library corrected for LUMI.
klust Apr 5, 2022
492683e
Section about hooks extended with references to additional examples.
klust Apr 5, 2022
8e734c5
Slurm job submission from EasyBuild a bit reworked.
klust Apr 6, 2022
aff9979
Some explanation in the overview of part 3
klust Apr 6, 2022
855101c
GitHub integration section, mostly old text and not all suited for LUMI.
klust Apr 6, 2022
89e01df
Adapt the structure and include an additional reading section.
klust Apr 6, 2022
ed863b9
Correcting a number of links.
klust Apr 6, 2022
557ba61
Restructuring for nicer navigation bar on the left.
klust Apr 7, 2022
df0d8e5
Some improvments to the module naming schemes section based on the IS…
klust May 19, 2022
36934df
Several minor corrections, including a new least of EasyBuild communi…
klust Jun 2, 2022
f62ed31
Minor corrections to the Lmod section.
klust Jun 3, 2022
28c00ba
Updated the "What is EasyBuild" page of the tutorial for LUMI."
klust Mar 7, 2022
2086896
Updated the structure, initial updates of terminology and a new page …
klust Mar 9, 2022
c4d368f
Additions to the Lmod section and some restructuring, and better info…
klust Mar 10, 2022
861690e
Corrections to the overview for part I.
klust Mar 10, 2022
cd69082
Further work on the Lmod section.
klust Mar 11, 2022
0f401f5
Finished the Lmod section.
klust Mar 14, 2022
290a375
Continued merging of the new CSC tutorial in the structure of the rep…
klust Mar 14, 2022
744d0a7
Tutorial page about the Cray PE, and correction of a typo.
klust Mar 14, 2022
8496f45
Corrected a typo in mkdocs.yml
klust Mar 16, 2022
d5b2647
Adding an example of implementing a hierarchy to the Lmod section of …
klust Mar 16, 2022
7eda98d
More information about hierarchy in the Cray PE, and definition of to…
klust Mar 21, 2022
e1bde80
Add a mermaid.js diagram with the toolchain hierarchy.
klust Mar 21, 2022
a0f4141
Finished terminology session, extended the installation section with …
klust Mar 21, 2022
6338bb1
Tutorial on EasyBuild configuration added, and some restructuring.
klust Mar 21, 2022
42d40a4
Restructuring continued.
klust Mar 21, 2022
863b933
Finished reworking the basic usage section, except for the exercises.
klust Mar 30, 2022
4a51424
Additions to the LUMI software stack page of the tutorial.
klust Mar 30, 2022
20f05ed
Integrating parts of the old tutorial into the new one, texts still n…
klust Mar 30, 2022
158c78d
Troubleshooting section reworked for LUMI.
klust Mar 30, 2022
202a53b
Section on creating easyconfig files.
klust Apr 1, 2022
71e04e4
Adding in an additional section about external modules.
klust Apr 1, 2022
da4ad40
Adding in a section taken from a previous tutorial and integrating so…
klust Apr 1, 2022
fa09f7d
Implementing EasyBlocks adapted for LUMI.
klust Apr 5, 2022
1c6894a
Correction of typos.
klust Apr 5, 2022
81274e3
Using EasyBuild as a library corrected for LUMI.
klust Apr 5, 2022
4e5ac8f
Section about hooks extended with references to additional examples.
klust Apr 5, 2022
db5cdec
Slurm job submission from EasyBuild a bit reworked.
klust Apr 6, 2022
cc30e75
Some explanation in the overview of part 3
klust Apr 6, 2022
7a9f3c7
GitHub integration section, mostly old text and not all suited for LUMI.
klust Apr 6, 2022
9a01408
Adapt the structure and include an additional reading section.
klust Apr 6, 2022
fe44f87
Correcting a number of links.
klust Apr 6, 2022
064afbe
Restructuring for nicer navigation bar on the left.
klust Apr 7, 2022
7145870
Correction of links.
klust Apr 7, 2022
4a7d65d
Multiple minor corrections.
klust May 10, 2022
6b0ebef
Initial commit, already contains the structure to work on the first t…
klust Mar 7, 2022
0445175
Updated the "What is EasyBuild" page of the tutorial for LUMI."
klust Mar 7, 2022
e438194
Updated the structure, initial updates of terminology and a new page …
klust Mar 9, 2022
7ec3ad4
Additions to the Lmod section and some restructuring, and better info…
klust Mar 10, 2022
feae23b
Corrections to the overview for part I.
klust Mar 10, 2022
6513998
Further work on the Lmod section.
klust Mar 11, 2022
f41da00
Finished the Lmod section.
klust Mar 14, 2022
5415f72
Tutorial page about the Cray PE, and correction of a typo.
klust Mar 14, 2022
0f19843
Adding an example of implementing a hierarchy to the Lmod section of …
klust Mar 16, 2022
70f23f1
More information about hierarchy in the Cray PE, and definition of to…
klust Mar 21, 2022
ad10f45
Add a mermaid.js diagram with the toolchain hierarchy.
klust Mar 21, 2022
36439fc
Finished terminology session, extended the installation section with …
klust Mar 21, 2022
0e2a846
Tutorial on EasyBuild configuration added, and some restructuring.
klust Mar 21, 2022
b5ea26d
Restructuring continued.
klust Mar 21, 2022
555afb7
Finished reworking the basic usage section, except for the exercises.
klust Mar 30, 2022
d24d31b
Additions to the LUMI software stack page of the tutorial.
klust Mar 30, 2022
bbec7d0
Integrating parts of the old tutorial into the new one, texts still n…
klust Mar 30, 2022
6dbfb52
Troubleshooting section reworked for LUMI.
klust Mar 30, 2022
7f4a8a7
Section on creating easyconfig files.
klust Apr 1, 2022
1d27441
Adding in an additional section about external modules.
klust Apr 1, 2022
4fd6638
Adding in a section taken from a previous tutorial and integrating so…
klust Apr 1, 2022
0b02bd1
Implementing EasyBlocks adapted for LUMI.
klust Apr 5, 2022
cca0812
Correction of typos.
klust Apr 5, 2022
ed39db4
Using EasyBuild as a library corrected for LUMI.
klust Apr 5, 2022
1a1c143
Section about hooks extended with references to additional examples.
klust Apr 5, 2022
5da4db5
Slurm job submission from EasyBuild a bit reworked.
klust Apr 6, 2022
ad1142f
Some explanation in the overview of part 3
klust Apr 6, 2022
7f742b9
GitHub integration section, mostly old text and not all suited for LUMI.
klust Apr 6, 2022
10280f8
Adapt the structure and include an additional reading section.
klust Apr 6, 2022
bf1dd5e
Additional reading section.
klust Apr 6, 2022
50a3aa1
Correcting a number of links.
klust Apr 6, 2022
5119b5e
Restructuring for nicer navigation bar on the left.
klust Apr 7, 2022
806738e
Correction of links.
klust Apr 7, 2022
a1b93c9
Corrected a number of spelling mistakes.
klust Apr 8, 2022
8492956
Corrected and updated several links.
klust Apr 21, 2022
5bb56bd
Multiple minor corrections.
klust May 10, 2022
aabc649
Updated the first page of the CSC course to integrate with the regula…
klust Nov 4, 2022
d897f35
Corrected two typos.
klust Nov 4, 2022
f5bf0a8
Correction after rebase.
klust May 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updated the structure, initial updates of terminology and a new page …
…about the Cray PE and Lmod.
  • Loading branch information
klust committed Jun 2, 2022
commit 6541e8c506e49b5586be7cc0d6f4db592af0b7f7
4 changes: 2 additions & 2 deletions docs/2022-CSC_and_LO/1_00_part1_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
*(Tue March 4th 2021, 9am-12 CET)*

- [What is EasyBuild?](1_01_what_is_easybuild.md)

- [Terminology](terminology.md)
- [Terminology](1_02_terminology.md)
- [Installation](installation.md) *(hands-on)*
- [Configuration](configuration.md) *(hands-on)*
- [Basic usage](basic_usage.md) *(hands-on)*
34 changes: 17 additions & 17 deletions docs/2022-CSC_and_LO/1_01_what_is_easybuild.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ and provides a **command-line interface** to facilitate the installation and man
<img src="../../img/geek-and-poke-invaluable.png" style="float:right" width="45%"/>
</a>

### *Elevator pitch*
## Elevator pitch

EasyBuild intends to relieve HPC support teams from manually managing
software installations while at the same time **providing a consistent and well performing
Expand Down Expand Up @@ -46,7 +46,7 @@ is still under development on LUMI it is not yet covered in this LUMI-specific v

---

### *Key features*
## Key features

EasyBuild is capable of **fully autonomously installing (scientific) software**,
including making sure that all necessary dependencies are installed,
Expand All @@ -73,13 +73,13 @@ packaging tools ([FPM](https://fpm.readthedocs.io)), and so on.

---

### *Focus points*
## Focus points

EasyBuild was created specifically for **installing scientific software on HPC systems**,
which is reflected in some of the design choices that were made.


#### Performance
### Performance

EasyBuild strongly prefers to **build software from source code**, whenever possible.

Expand All @@ -99,7 +99,7 @@ idea to hard-code options such as ``-march=native`` or ``--xHost`` in a way that
cannot easily overwrite those.


#### Reproducibility
### Reproducibility

In addition to performance, **reproducibility of installations** is a core aspect of EasyBuild.

Expand All @@ -120,7 +120,7 @@ As we shall see when discussing the HPE Cray Programming Environment, the reprod
in that environment due to the way the programming environment selects shared libraries at runtime.


#### Community effort
### Community effort

In a number of different ways, we try to encourage EasyBuild users to **collaborate** and help each other out.

Expand Down Expand Up @@ -148,7 +148,7 @@ All together this leads to improved stability and consistency.

---

### *What EasyBuild is* ***not***
## What EasyBuild is ***not***

EasyBuild is ***not*** **YABT (Yet Another Build Tool)**: it does *not* replace established build
tools like CMake or ``make``, it wraps around them.
Expand All @@ -167,7 +167,7 @@ or even with the AMD compilers.

---

### *Requirements*
## Requirements

Since EasyBuild is primarily intended for installing (scientific) software on HPC systems, the main target is
systems running some flavor of **GNU/Linux**. EasyBuild also works on *macOS*,
Expand All @@ -187,7 +187,7 @@ On LUMI Lmod is used as the module tool.

---

### *Development*
## Development

<a href="http://github.com/easybuilders"><img src="../../img/github_logo.png" style="float:left" width="15%"/></a>
EasyBuild is **actively developed** by a worldwide community under the
Expand All @@ -213,7 +213,7 @@ consistent testing of incoming contributions (pull requests), and thorough regre

---

### *A worldwide community*
## A worldwide community

EasyBuild has grown into a **community project** since it was first released publicy in 2012,
and is used and developed by dozens (hundreds?) of HPC centres and consortia worldwide.
Expand Down Expand Up @@ -250,17 +250,17 @@ HPC sites, consortia, and companies that use EasyBuild include (but are not limi

---

### *Getting help*
## Getting help

#### *Documentation*
### *Documentation*

If you have any questions about EasyBuild, make sure to check the EasyBuild documentation first,
which is available at [**https://docs.easybuild.io**](https://docs.easybuild.io).

It is not unlikely that an answer to your question is available there already.
The [search box](https://docs.easybuild.io/en/latest/search.html) on the top left may be helpful.

#### GitHub
### GitHub

<a href="http://github.com/easybuilders"><img src="../../img/github_logo.png" style="float:left" width="15%"/></a>

Expand All @@ -275,7 +275,7 @@ the EasyBuild maintainers can move your issue if deemed appropriate.
Also here you can use the search box on the top left, to see if somebody else
has already asked a similar question or reported a similar problem.

#### Mailing list
### Mailing list

Feel free to ask any questions or provide feedback via the EasyBuild mailing list
<a href="mailto:[email protected]">`[email protected]`</a>.
Expand All @@ -287,7 +287,7 @@ In 2016-2017 over 1,000 messages were posted to the EasyBuild mailing list.
More recently this has decreased to about 700 messages per year, mostly due
to the EasyBuild Slack channel.

#### Slack
### Slack

<img src="../../img/slack_logo.png" alt="Slack logo" style="float:left" width="22%"/>

Expand All @@ -304,7 +304,7 @@ IRC channel (available on the [Freenode IRC network](https://freenode.net/)).
A bot relays messages between this IRC channel and the `#general` channel in the EasyBuild Slack.


#### Conference calls
### Conference calls

Every other week on Wednesday an EasyBuild conference call is set up to discuss recent
developments, provide an outlook to the next EasyBuild release, and
Expand All @@ -318,4 +318,4 @@ More information, including the dates and times of the next calls, is available

---

*[[next: Terminology]](terminology.md)*
*[[next: The HPE Cray Programming Environment and Lmod]](1_02_CPE_and_Lmod.md)*
71 changes: 71 additions & 0 deletions docs/2022-CSC_and_LO/1_02_CPE_and_Lmod.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# The HPE Cray Programming Environment and Lmod

*[[back: What is EasyBuild?]](1_01_what_is_easybuild.md)*

---

On LUMI, the main programming environment is the HPE Cray Programming Environment (further abbreviated
as Cray PE). The environment provides several tools, including compilers, communication libraries,
optmized math libraries and various other libraries, analyzers and debuggers.

The Cray PE is made available through *environment modules* tha allow to select particular versions of
tools and to configure the environment in a flexible way.

---

## Modules

*Module* is a massively overloaded term in (scientific) software and IT in general
(kernel modules, Python modules, and so on).
In the context of EasyBuild, the term 'module' usually refers to an **environment module (file)**.

[Environment modules](https://en.wikipedia.org/wiki/Environment_Modules_(software)) is a well established concept
on HPC systems: it is a way to specify changes that should be made to one or more
[environment variables](https://en.wikipedia.org/wiki/Environment_variable) in a
[shell](https://en.wikipedia.org/wiki/Shell_(computing))-agnostic way. A module file
is usually written in either [Tcl](https://en.wikipedia.org/wiki/Tcl) or
[Lua](https://en.wikipedia.org/wiki/Lua_(programming_language)) syntax,
and specifies which environment variables should be updated, and how (append,
prepend, (re)define, undefine, etc.) upon loading the environment module.
Unloading the environment module will restore the shell environment to its previous state.

Environment module files are processed via a **modules tool**, of which there
are several conceptually similar yet slightly different implementations.
The oldest module tool still in use today is Environment Modules 3.2, implemented in C and
supporting module files written in Tcl. After a gap in developement, Xavier Delaruelle of CEA
developed [Environment Modules 4 and 5](https://sourceforge.net/projects/modules/) which is
fully implemented on Tcl. An alternative module tool is [Lmod](https://lmod.readthedocs.io),
developed by Robert McLay at TACC and implemented in LUA. This tool supports natively LUA
module files but also offers a high degree of compatibility with Tcl-based module files
developed for Environment Modules fia a translation layer and some API translation.

The Cray PE offers a choice between the old-style Environment Modules 3.2 and Lmod, but no
packages or official support for Environment Modules 4 or 5. On LUMI, Lmod was selected as
the module tool. At the user level, Lmod and the various versions of Emvironment Modules
have very simmilar commands for managing the environment, but with different options.
The commands for searching for modules are very different though so if you are not familiar
with Lmod and its commands for users, it is worthwile to read the
[LUMI documentation page on Lmod](https://docs.lumi-supercomputer.eu/computing/Lmod_modules/).

---

## Lmod features


---

## Cray PE components


---

## Configuring the Cray PE through modules

---

## Further reading


---

*[[next: Terminology]](1_03_terminology.md)*
Loading