Lambda Ethereum Consensus Client

Prerequisites

Direct Installation

You can install the necessary components directly from official sources:

• Elixir
• Erlang
• Go
• Rust
• Protoc

Alternative (Recommended) Installation

For precise control over versions, it's recommended to use the asdf tool version manager and follow the versions specified in .tool-versions in this repository.

• asdf tool version manager

After installing asdf, add the required plugins for managing the tools:

asdf plugin add elixir
asdf plugin add erlang
asdf plugin add golang
asdf plugin add rust
asdf plugin add protoc

Finally, install the specific versions of these tools as specified in .tool-versions:

asdf install

Alternative (easier) Installation using Nix

To create a sandbox environment with all the required tool chains, use Nix. Steps to get Nix working are as follows:

1. Install Nix from the official website: https://nixos.org/download.
2. To allow experimental features (nix develop and nix-command) you might need to do the following:

mkdir ~/.config/nix
echo "experimental-features = nix-command flakes " > ~/.config/nix/nix.conf

Alternatively, for a smoother experience you can use the following script from Determinate Systems that takes care of setting up everything for you:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

• Check if Nix has been successfully installed: nix --version.
• To launch the environment: nix develop.

Installing and running

There are Makefile targets for these tasks.

make deps # Installs dependencies
make iex  # Runs a terminal with the application started

The iex terminal can be closed by pressing ctrl+c two times.

Checkpoint Sync

You can also sync from a checkpoint given by a trusted third-party. For that, get the URL that serves the checkpoint, and execute the following command:

iex -S mix run -- --checkpoint-sync <your_url_here>

Some public endpoints can be found in eth-clients.github.io/checkpoint-sync-endpoints

Tests, linting and formatting

Our CI runs tests, linters, and also checks formatting and typing. To run these checks locally:

make test      # Runs tests
make spec-test # Runs all spec-tests
make lint      # Runs linter and format-checker
mix dialyzer   # Runs type-checker

Source code can be formatted using make fmt. This formats not only the Elixir code, but also the code under native/.

Consensus spec tests

You can run all of them with:

make spec-test

Or only run those of a specific test runner with:

make spec-test-`runner`
# some examples
make spec-test-ssz_static
make spec-test-bls
make spec-test-operations

The complete list of test runners can be found here.

For more fine-grained filtering of tests, you can use mix test tag filters:

mix test --no-start --only <tag_name>[:<tag_value>]

Note

We specify the --no-start flag to stop ExUnit from starting the application, to reduce resource consumption.

Some useful tag filters:

• spectest: only run spec-tests, same as make spec-test
• config:general: only run spec-tests with "general" config
• fork:capella: only run spec-tests of the "capella" fork
• runner:ssz_generic: only run spec-tests of the "ssz_generic" runner
• handler:Checkpoint: only run spec-tests using the "Checkpoint" handler
• test:<name>: run one specific test named <name>, for example test:"test c:minimal f:capella r:ssz_static h:Checkpoint s:ssz_lenghty -> case_0"

Why Elixir?

Elixir is a functional programming language that runs atop the Erlang Virtual Machine (BEAM). It offers enhanced readability, syntactic sugar, and reduced boilerplate, enabling developers to achieve more with fewer lines of code compared to Erlang. Like Erlang, Elixir compiles to bytecode that is interpreted by the VM. As a result, it inherits several notable properties, including:

• Fault tolerance for increased reliability
• High availability
• Simplified construction of complex distributed systems
• Predictable latency

Erlang and its VM were originally developed in 1986 for telecommunication systems that demanded unparalleled uptime and reliability. We recognize that these attributes could be immensely beneficial for an Ethereum client, particularly in the realm of consensus. This is why our current focus is on building a consensus layer (CL) rather than an execution layer (EL). Elixir may not be tailored for sheer performance, but it excels in delivering predictable latency and creating systems designed for continuous operation—qualities essential for the CL.

Our aim is to infuse these strengths into the Ethereum consensus client ecosystem with our offering.

We also have for objective to bootstart an Ethereum Elixir community, and to make Elixir a first-class citizen in the Ethereum ecosystem.

Roadmap

1. Block Subscription - Mid September

• Libp2p discovery and block retrieval
• SSZ + snappy
• on_block callback: Save the latest block in fork-choice store, conduct basic checks. GetHead returns the last obtained block.
• Beacon API: Return block root (GET /beacon/states/{state_id}/root)

2. Checkpoint Sync - October

• Libp2p primitives for sync
• Support checkpoint Sync from a known provider
• Sync from the latest finalized block
• BeaconAPI: Return headers for head block
• EngineAPI: Validate incoming blocks

3. Attestations - Mid October

• Libp2p attestation retrieval
• Basic beacon state representation
• Store attestations (last message sent by each validator)
• on_attestation callback for attestations sent via Gossip
• Process attestations from blocks
• Beacon API: Return head block root (GET /beacon/states/head/root)

4. Deposits - November

• BLS signature checks
• Update consensus state view of deposit contract (process_eth1_data)
• Process deposit operation to update validator list (process_deposit)
• Verify block signatures (verify_block_signature)

5. Slots and Fork-choice - Mid November

• on_tick/process_slot in state transition; a GenServer that calls this periodically
• on_block: Add slot-related checks and epoch calculations (excluding finalization)
• Get-head uses the messages
• Block header validation
• EngineAPI: Process execution payload
• BeaconAPI: Ensure getting head values point to the heaviest

6. Finality and Slashing - Mid November

• Epoch processing
• on_block: Prune fork-choice store; reject blocks before finalization
• Add RANDAO mix to the beacon state
• BeaconAPI: Retrieve finality checkpoints, randao mixes
• Process attester slashings and proposer slashings
• EngineAPI: fork-choice updates

7. Rewards, Shuffling - December

• Process rewards on_epoch for a checkpoint
• Handle Deposits and Withdrawals
• Implement RANDAO
• Calculate committee for a given state
• Conduct shuffling
• Integrate with Grafana
• BeaconAPI: Retrieve randao mix for a given block

8. Validator Features - Mid December/January 2024

• Create attestations
• Monitor for slashings
• Create slashing proofs
• BeaconAPI: Post blocks, slashings, voluntary exits, and withdrawals

Contributor Package

Dream of becoming an Ethereum core developer? Eager to shape the protocol that will underpin tomorrow's world? Want to collaborate with a passionate team, learn, grow, and be a pivotal part of the Ethereum Elixir community?

Then you're in the right place! 🚀

Getting Started

1. Installation

• Prerequisites: Before diving in, ensure you have the necessary tools installed. Check out the Prerequisites section for guidance.

• Clone the Repository:

  git clone [REPO_URL]
  cd lambda_ethereum_consensus

• Setup: Once you've cloned the repository, follow the steps in the Installing and running section to set up your environment.

2. Prerequisite Knowledge

To contribute effectively, you'll need a foundational understanding of both the Ethereum protocol and the Elixir language, including the Erlang VM (BEAM). If you're new to these areas, we've curated a list of resources to get you started:

Learning Elixir:

• Videos:
  • Intro to Elixir
  • Hitchhiker's tour of the BEAM
• Blogs:
  • Zen of Erlang
  • Where Erlang Blooms
  • What can I only do in Erlang
  • Stacking theory for systems design
  • On Erlang States and Crashes
  • How Erlang does scheduling
• Books:
  • Elixir in Action
  • Learn You Some Erlang

With this foundation you should have a basic understanding of the Elixir language and the Erlang VM. You can then start (or in parallel) learning about the Ethereum protocol.

Learning Ethereum:

• Videos:
  • Basic technical details of Ethereum
  • Ethereum in 30 minutes
  • Foundations of Blockchains
  • Ethereum Foundation youtube channel
  • Ethereum youtube channel
• Posts
  • What happens when you send 1 DAI
• Books:
  • Inevitable Ethereum
• Blogs:
  • Vitalik Buterin's blog
  • Ethereum Foundation blog
  • Ethereum Magicians forum
  • Ethresear.ch forum
  • EIP's
  • ACD & Related meetings
• Papers:
  • Ethereum Whitepaper
  • Ethereum Yellowpaper
    • Yellow paper discussion
    • Yellow paper walkthrough
  • Ethereum Beige Paper
  • Ethereum Mauve Paper

Learning Ethereum Consensus:

• Books:
  • Eth2Book by Ben Edgington. This book is indispensable for understanding the Ethereum consensus protocol. If you can read only one thing, read this.
• Specifications:
  • Consensus specs
  • Vitalik Buterin's annotated specs
  • Eth2Book annotated specs

While some of the resources listed might appear outdated, it's important to understand that the Ethereum protocol is continuously evolving. As such, there isn't a definitive, unchanging source of information. However, these resources, even if older, provide foundational knowledge that remains pertinent to understanding the protocol's core concepts.

Truly mastering the Ethereum protocol is a complex endeavor. The list provided here is just a starting point, and delving deeper will necessitate exploring a broader range of readings and resources. As you immerse yourself in the project, continuous learning and adaptation will be key.

If you come across any resource that you find invaluable and believe should be added to this list, please don't hesitate to suggest its inclusion.

3. Dive In

With your newfound knowledge, explore the various areas of our project. Whether you're interested in the core consensus layer, networking, CLI, documentation, testing, or tooling, there's a place for you.

Start by browsing our issues, especially those tagged as good first issue. These are beginner-friendly and a great way to familiarize yourself with our codebase.

How to contribute

Found an issue you're passionate about? Comment with "I'd like to tackle this!" to claim it. Once assigned, you can begin your work. After completing your contribution, submit a pull request for review. Our team and other contributors will be able to provide feedback, and once approved, your contribution will be merged.

Please adhere to the Conventional Commits specification when crafting PR titles. Also, run make fmt to format source code according to the configured style guide. The repo enforces these automatically via GitHub Actions.

Important

We believe in fostering an inclusive, welcoming, and respectful community. All contributors are expected to adhere to our Code of Conduct. Please familiarize yourself with its contents before participating.

Communication

Open communication is key to the success of any project. We encourage all contributors to join our Telegram chat for real-time discussions, updates, and collaboration.

For more structured discussions or proposals, consider opening an issue or a discussion on the GitHub repository.

Recognition

We value every contribution, no matter how small. All contributors will be recognized in our project's documentation. Additionally, consistent and significant contributors may be offered more formal roles within the project over time.

Support

If you encounter any issues or have questions, don't hesitate to reach out. Our team and the community are here to help. You can ask questions in our Telegram chat or open an issue on GitHub for technical challenges.

Conclusion

Lambda Ethereum Consensus is more than just a project; it's a community-driven initiative to bring the power and reliability of Elixir to the Ethereum ecosystem. With your help, we can make this vision a reality. Dive in, contribute, learn, and let's shape the future of Ethereum together!

---

Thank you for being a part of our journey. Let's build an amazing future for Ethereum together! 🚀🌍

Profiling

QCachegrind

To install QCachegrind via Homebrew, run:

brew install qcachegrind

To build a qcachegrind profile, run, inside iex:

LambdaEthereumConsensus.Profile.build()

Options and details are in the Profile package. After the profile trace is generated, you open it in qcachegrind with:

qcachegrind callgrind.out.<trace_name>

If you want to group the traces by function instead of process, then you can the following before viewing it in qcachegrind:

grep -v "^ob=" callgrind.out.trace_name > callgrind.out.merged.trace_name

Etop

Another useful tool to quickly diagnose processes taking too much CPU is :etop, similar tu UNIX top command. This is installed by default in erlang, and included in the :observer extra application in mix.exs. You can run it with:

:etop.start

In particular, the reds metric symbolizes reductions, which can roughly be interpreted as the number of calls a function got. This can be used to identify infinite loops or busy waits.

Code of Conduct

Our Pledge

We, as members, contributors, and leaders of open source communities and projects pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community and project.

Our Standards

Examples of behavior that contributes to a positive environment for our community include:

• Demonstrating empathy and kindness toward other people.
• Being respectful of differing opinions, viewpoints, and experiences.
• Giving and gracefully accepting constructive feedback.
• Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience.
• Focusing on what is best not just for us as individuals, but for the overall community and project.

Examples of unacceptable behavior include:

• The use of sexualized language or imagery, and sexual attention or advances of any kind.
• Trolling, insulting or derogatory comments, and personal or political attacks.
• Public or private harassment.
• Publishing others' private information, such as a physical or electronic address, without their explicit permission.
• Other conduct which could reasonably be considered inappropriate in a professional setting.

Enforcement Responsibilities

Maintainers are responsible for clarifying and enforcing standards of acceptable behavior and will take appropriate and fair corrective action.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for behaviors that they deem inappropriate, threatening, offensive, or harmful.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported with proof to the maintainers through Telegram. All complaints will be reviewed and investigated promptly, fairly and anonymously.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 2.1.

Contributors

Paul-Henry Kajfasz 💻 📖	Tomás 💻	Martin Paulucci 💻	Tomás Arjovsky 💻	Akash S M 💻	berwin 💻 📖	Fernando Ledesma 💻
Mete Karasakal 💻	Federico Carrone 💻	Godspower Eze 💻	Naman Garg 💻	Ayush 📖 💻	Seungmin Jeon 💻