A substantial thank you for outstanding insights and suggestions from Sacha Saint-Leger, Joseph Schweitzer, Josh Stark, and protolambda.
I dedicate a significant amount of my time elucidating and addressing inquiries regarding eth2, and I mean a significant amount. Some of this involves a deep and technical approach as I assist in conveying research and specifications to technical contributors, but increasingly these days I’m tackling questions from the community concerning eth2 evolution, trajectory, motivations, design choices, delays, and more. I genuinely enjoy these discussions. I become incredibly enthusiastic as I clarify eth2, devise new ways to explain different elements, or identify suitable analogies based on the audience to stimulate thinking and ignite understanding.
However, this dynamic/dialogue-oriented approach, while beneficial, leaves many in the community unaware. I find myself answering the same questions repeatedly, and more worryingly, those same inquiries resurface six months later! Clearly, an information deficit exists. Such information is available, yet it is dispersed throughout the internet — research articles, specs, spec elucidations, public discussions, public channels, reddit, blog entries. My initial effort after devcon5 to bridge the information divide between those deeply immersed in eth2 and the broader community materialized as a new blog series, “eth2 quick update”. These are brief snippets intended to assist in following the developments, but I’m realizing they fail to convey the broader context. The overall context is however conveyed and debated in podcasts, AMAs, and conferences, but, even in those instances, a written form will still support these initiatives.
So here we find ourselves. This article is directed towards the community, to furnish you with a detailed overview of what eth2 is presently: where it’s headed, what it may evolve into, and what it signifies for you, the Ethereum community. I will endeavor to provide the appropriate amount of technical detail to illustrate the motivations, the vision, the current status of the project, and the forthcoming work, without becoming mired in excessive mathematics or convoluted jargon.
This article may also be advantageous for those deep Ethereum technical experts who have thus far kept eth2 at a distance. No worries, I comprehend. This endeavor is extensive, intricate, and has always appeared sufficiently far in the future that you could disregard it while addressing the pressing issues at hand. Hopefully, this article will facilitate a better grasp of what lies ahead.
As for the eth2 individuals, you may also gain insights from this article — a broader outlook on our current position, and how I conceptualize the upcoming developments.
Disclaimer: this reflects how I, Danny Ryan, personally perceive the situation today. Numerous voices and perspectives are fueling the continually expanding, ever-evolving eth2. This is merely a snapshot of a segment of my interpretation.
eth2, wtf
“Eth2 is a scalable proof-of-stake infrastructure.”
If you have heard me speak at all in the past six months, you’ve likely heard me repeat this over and over. Eth2 is designed for Ethereum and fundamentally is Ethereum. It endeavors to serve as a more secure and scalable backdrop for the current Ethereum mainnet, ensuring minimal disruption to existing operations. Simultaneously, it offers an enhanced context for us to evolve into.
Since prior to Ethereum’s launch, it was recognized that a singular blockchain model would not provide sufficient bandwidth to function as the backbone of a new decentralized internet. Research pertaining to Ethereum’s proof-of-stake and sharding dates back to as early as 2014. Both proof-of-stake and sharding strive to respond to the following question: With a specific amount of capital supporting a crypto-economic framework, can we enhance security and throughput while still allowing consumer hardware to participate in consensus and follow the chain? While I won’t delve into the historical context here, this exploration spanned years and was characterized by numerous false starts. Ultimately, the answer is a resounding yes, culminating in the manifestation of the eth2 project.
Eth2 is a bold, multi-year initiative that will be deployed in stages. This is thoroughly documented and widely discussed, but I’ll provide a brief, non-technical overview of what these phases entail.
Phase 0
Phase 0, known as the Beacon Chain, is the foundation of the new consensus mechanism. This is the location where all system-level activities and orchestration occur. Phase 0 focuses on achieving consensus with hundreds of thousands of consensus entities (validators) spread across thousands of nodes globally.
Due to the technical necessities of distributing subsets of validators across shards in phases 1 and beyond, we need to manage an enormous number of validators. Much of the engineering intricacy arises from this requirement. Other non-sharded, proof-of-stake systems generally accommodate hundreds or maybe thousands of validators, but eth2 is designed to support a minimum of around 16,000 validators with the expectation that this number will escalate into the hundreds of thousands within a few years.
Phase 1
Phase 0 focuses on achieving consensus, while Phase 1 involves achieving consensus on a multitude of elements. This “elements” appears in the form of several shard chains. You can envision a shard chain as an independent blockchain with approximately the same complexity as Ethereum today, but existing under the eth2 consensus (i.e., governed by and constructed/managed by the Beacon Chain). The validators from the Beacon Chain are assigned random short-term tasks to build and validate shard chains, making crypto-economic commitments to the state, availability, and validity of each chain back into the core system.
At present, we anticipate beginning with 64 shards, and the total data accessible to the system is expected to fall within the 1 to 4 MB/s range (YES, that’s an immense amount of data).
Phase 1.5
Phase 1.5 represents the incorporation of the Ethereum mainnet into the new eth2 consensus mechanism as a shard (existing as one of the many shards established in Phase 1). Instead of the Ethereum that we know and cherish being operated by a proof-of-work mining mechanism, it will be managed by the eth2 validators. For existing applications and users, this quick transition of the consensus method will largely be unnoticed. Applications will continue to function smoothly, but developers will now possess a significantly more robust system to build upon (enhanced security features, proper economic finality, more layer 1 data forrollups and additional engaging applications).
Phase 2
Phase 2 introduces state and execution across more shards than the initial Ethereum shard. This can manifest in numerous ways. Determining the specific form and the intricacies involved is a focal area of inquiry and development today. I’ll elaborate further on this in the subsequent sections.
Alright, we have all these phases ahead, and Phase 0 truly feels imminent. However, that pathway still seems somewhat extended. What can I genuinely anticipate from eth2 throughout these upgrade phases?
Excellent question! Generally, look forward to a series of improvements that progressively impact more of Ethereum and its community at each juncture. As a participant, you can choose to engage early through staking in Phase 0, or simply hold off until Ethereum completely transitions into eth2 during Phase 1.5 (a shift that should be seamless for both dapp developers and users). Regardless of your level of involvement and at which phase you engage, there are significant milestones and advantages worth noting as this all begins to unfold.
To start, I recognize many of you are passionate ETH supporters eager to join the staking opportunity. To all aspiring validators, especially hobbyists, Phase 0 is tailored for you. This phase comes with its own risks and timelines that may deter certain participants, so I sincerely hope it serves as a boon for both hobbyists and dedicated Ethereum advocates. This is a unique opportunity to get involved from the ground up, to influence the vision over time, and to earn a higher ETH reward for being an early participant.
What about Phase 1? Is there any valuable activity we can undertake with all this data prior to Ethereum’s integration into eth2? Yes, I’m glad you inquired!
Layer 1 data is immensely valuable even without intrinsic computation. In fact, the most promising layer 2 scaling solutions over the past year are the so-called “rollup” chains (both optimistic and ZK) that scale with the accessibility of layer 1 data. The eth2 data layer is projected to provide Ethereum with an availability range of between 1 and 4 MB/s, resulting in substantial scalability improvements when paired with rollup technology. Yet, due to the initial disjointed nature of Ethereum and the new sharded ecosystem at the outset, making assertions regarding the eth2 shard data is complex. This is one of the reasons EIP 2537 holds such significance for Ethereum mainnet. With a native BLS (new eth2 signing algorithm) precompile, we can craft an efficient eth2 light client as a Solidity contract, enabling Ethereum applications to assert claims about data in eth2 before the Phase 1.5 integration.
As mentioned earlier, Phase 1.5 is monumental. Eth2 is designed for Ethereum, and at this point, eth2 transforms into Ethereum. All our beloved applications become embedded within the enhanced eth2 consensus mechanism, preserving the features we are accustomed to while simultaneously unlocking the vast new realm of a secure proof-of-stake consensus with immediate accessibility to a highly scalable data layer. This, in my view, represents the crux of the process. This is the moment of profound accomplishment as we firmly anchor Ethereum into its new reality.
Moreover, future scalability enhancements will likely be realized over time by facilitating state/execution on additional shard chains. This could be achieved through the EVM or a new VM termed eWASM. Regardless of which VM is selected, the current Ethereum EVM shard and the new shard chains will be capable of interacting and communicating natively via the Beacon Chain, fulfilling the vision of multi-execution and sharding.
See? It’s a voyage, but significant benefits are to be gained along the path.
The challenges of this approach, and its value
So many validators
A crucial aspect of sharding is the random selection of consensus participants (validators) into groups to validate a segment of the protocol (e.g., a shard). With a sufficient number of validators within the protocol, and an assailant of a presumed maximum size (controlling 1/3 of the validators, for example), it becomes mathematically unlikely (nearly impossible, think probability bordering on 1 / 2^40) for the assailant to dominate any committee and disrupt the system. This enables us to craft the system such that anyone with a consumer-grade machine (e.g., a laptop or possibly even an old phone) can serve as a validator (given that validators are allocated to sections of the system, and validating any specific section can be performed using the computational capacity of a single machine).
This is precisely what makes sharding remarkable yet simultaneously challenging. Firstly, we must ensure there are ample validators to maintain the safety of this random sampling: meaning eth2 will have significantly more expected validators than most (if not all) other proof-of-stake protocols. This introduces difficulties across every layer of the process — from research to consensus mechanism definition, to networking, to resource utilization and optimizations in clients. Each additional validator increases the burden on the system, which must be taken into account at each step of the process. Eth2 client development teams have undertaken the formidable challenge of managing the consensus among hundreds of thousands of validators so we can effectively and safely incorporate numerous shards when Phase 1 arrives.
So many shards
Another essential design choice contributing to the complexity of our undertaking is that, in Ethereum, we opt to pursue scalability while maintaining decentralization.
It’s not particularly challenging to scale a blockchain to handle tens of thousands of transactions per second if we disregard whether users can feasibly validate the chain themselves, or if data is truly accessible to the network. The intricacy of a sharded consensus mechanism is necessary to allow the system to be divided into manageable, verifiable segments. Specifying and executing such a consensus framework is fundamentally a demanding task.
So many clients
A foundational principle of Ethereum is that Ethereum is protocol-first. Ethereum constitutes the abstract collection of rules that constitute the protocol rather than any distinct implementation of those rules. To that end, the Ethereum community has advocated for multiple client implementations.since day 0. On the Ethereum mainnet today, this materializes in the shape of besu, ethereumJS, geth, nethermind, nimbus, open-ethereum, trinity, and turbo-geth. In the eth2 ecosystem, this appears as cortex, lighthouse, lodestar, nimbus, prysm, teku, and trinity.
The multi-client model has numerous notable benefits:
- Having multiple clients permits a broader investigation of concepts, algorithms, and structures (each client contributes its approach and perspective). This process fosters a beneficial exchange of ideas as we collectively construct more resilient systems.
- Clients frequently pursue different design objectives. This results in a more varied set of users and applications as time evolves. Clients may focus more or less on aspects like performance, security, horizontal scalability, UI/UX, light clients, browsers, resource-limited devices, and so forth.
- With several production-level clients on the mainnet, a substantial assault capable of incapacitating any single client (e.g. a DoS attack) is countered with resilience as the other clients remain unaffected. This was evident early in Ethereum’s story during the “Shanghai DoS Attacks” when a succession of DoS assaults managed to take down geth and parity but never both simultaneously.
- Each client acts as a portal to a programming language community. The establishment of a client in a specific language fosters and encourages experimentation and innovation within that language. The foundational tools surrounding the client often amplify into a vigorous ecosystem of tools and contributors in that language. The multi-client model strengthens the gravitational center that is Ethereum.
Alongside these unique benefits arise several challenges:
- The specifications and testing must be foolproof to prevent any inadvertent forking on the mainnet. In case there exists only a singular implementation of the protocol, that implementation becomes the protocol. In a single client scenario, if a consensus “bug” were to affect the mainnet, it would then become entrenched in the reality of the protocol. This is not ideal from a purity standpoint, but it does eliminate any risk of an unintended fork. Conversely, if we maintain a healthy distribution of clients on the mainnet (e.g., no client possesses more than 1/3 of total nodes/validators), the network can remain operational despite a single client facing a consensus problem.
- Coordinating N clients yields a linear overhead at best in comparison to a single client, but in certain instances, it may incur a quadratic overhead (N^2). We utilize various strategies to minimize this overhead — for example, consensus (and soon network) test suites — but it will always persist in some form.
Current Status of eth2 Clients and Testnets
Phase 0 eth2 clients have evolved into quite refined pieces of software over the last 2 years, capable of managing the distributed consensus of hundreds of thousands of validators across numerous nodes. We are presently in the testnet stage and inching ever closer to launch each day. I anticipated the final stretch to be prolonged. It turns out that it indeed is.
I urge you, during this period leading up to launch, to step outside your comfort zone and experiment with multiple clients. There are various trade-offs between them, and you must engage hands-on to determine which is most suitable for you. As mentioned earlier, Ethereum functions within a multi-client framework. To reap the rewards of this model, we need users to operate a diverse array of clients (to foster a healthy distribution across all client types).
Moreover, the protocol incorporates anti-correlation incentives. In extreme scenarios where a significant client inadvertently causes validators to go offline or commit a slashable infraction, if your validator’s actions align with that client, you will incur penalties far greater than if you made a mistake that was uncorrelated with others. In simpler terms, during such circumstances, it is significantly preferable to operate a minority client instead of one representing a substantial part of the network.
To be completely explicit — if there exists more than one viable and secure client, it is your responsibility to utilize minority client software to support a healthy distribution of client software throughout the network.
Additionally, don’t hesitate. Should you encounter any discrepancies with the documentation, inform someone. If you spot a typo, submit a PR. If something crashes or a bug arises, kindly-please-kindly report it on GitHub or the client discord. You are the beta testers, and with your assistance, we can enhance the experience for all.
Testnets
We are currently operating small public devnets, rebooting them approximately every week to two weeks. I refer to them as “devnets” because they are primarily for client team developers to address bugs, optimizations, and more. They are public, and you are welcome to participate, but understand that they aren’t yet persistent like Goerli or Rinkeby. The latest launch, spearheaded by Afri Schoedon, is the Witti testnet running the v0.11 specification (check out the README here if you wish to operate some nodes).
Client teams are actively upgrading to the v0.12 specification which incorporates the latest iteration of the IETF BLS standard. Following that, we will transition the devnets to v0.12 as we broaden the scale of the nets, exerting more and more pressure on the clients. Once we have 2-3 clients adequately launching successful v0.12 nets and operating under high load, we will create a more public testnet where you will operate the majority of nodes and validators. The goal here is to establish a long-lasting multi-client testnet that closely resembles the mainnet (where users can reliably practice running nodes and test anything else they desire). The ideal situation is to set this up merely once and resolve any failures while sustaining the network. However, based on the frequency and severity of failures, we may require a few attempts before achieving success.
In addition to the standard testnets, we will also present an incentivized “attack net” where client teams maintain a stable testnet, and we invite you to attempt to disrupt it in various ways. For successful incursions, the Ethereum Foundation will award ETH rewards. More information on this soon — so stay tuned!
Whiletooling for eth2 is relatively new, it’s an exhilarating and expanding initiative. As previously stated, tools typically originate from a client codebase and the endeavors of the client group, but an increasing number of contributors are joining every day. To more effectively engage with, comprehend, secure, and improve eth2, we as a community need to develop and enhance fundamental eth2 tools.
I wish to extend a hearty acknowledgment to the teams and individuals who have already delivered significant value through their eth2 tools, and I invite all others to create new instruments and to augment and enhance what currently exists.
Eth2 tooling represents a fresh opportunity. This is a remarkable occasion to delve in, provide tangible value, and leave your imprint.
What follows is a sample of the ongoing efforts, but there is still a vast amount of work to accomplish!
And here’s a selection of some open tooling concepts:
- Eth2 validator notifications: offer a service that notifies node operators when their validators are underperforming
- Validator deposit monitoring: assist in bridging the existing Ethereum and eth2 explorers by tracking the validator deposit flow
- Validator safeguarding via proxies: utilize a proxy to monitor validator communications to ensure your client does not transmit hazardous messages
And there’s so much more — this represents the kind of contribution that transcends a specification. Imagination is crucial. If you wish to contribute, collaborate with eth2 client teams to begin.
State of eth1+eth2 integrations
In an Ethereum client presently (e.g., geth, etc.), nearly all of the intricacy resides in managing user-level activities — transaction pool, block generation, virtual machine computations, and state storage/retrieval. The actual core consensus — proof-of-work — is quite straightforward in protocol. The majority of the complexity is managed by advanced hardware outside of the core protocol.
Conversely, an eth2 client is exclusively consensus. In proof-of-stake and sharding, various complexities are incorporated into the protocol to attain the objectives of a scalable consensus.
This division of responsibilities creates a wonderful pairing of eth1 and eth2 clients.
Initial efforts are underway to merge the two by members of the geth (EF) and TXRX (ConsenSys) teams. The work encompasses (1) establishing a communication protocol between eth1 and eth2 clients, (2) integrating a consensus engine into eth1 clients that can be directed through the communication protocol, and (3) prototyping and simulating eth2 phase 1 behavior to test the coupling. We anticipate seeing some tangible outcomes on these matters this summer.
You can read more regarding the high-level eth1+eth2 client connection here, and about the technical scope of the merger here.
State of execution and communication across shards
As noted, the precise route to enable execution across multiple shards is a highly researched and debated subject. Numerous questions require addressing. For instance:
- How many shards should be enabled with execution?
- For additional shards, should we employ EVM or eWASM for the virtual machine?
- How do we effectively organize and process cross-shard transactions?
- What modifications are needed to the existing EVM to facilitate cross-shard transactions?
- Can/should execution and account architectures be generally extensible?
The eWASM (EF) and Quilt (ConsenSys) teams have undertaken substantial research in these realms over the past 12 months. It turns out the solution space is vast, and while we have now grasped the breadth of the domain, the recent focus has been on exploring simple, tangible solutions to enable testing, prototyping, and genuinely grounding the dialogue. From this emerged eWASM’s Eth1x64 initiative (read about the high-level overview of the project and check out some recent specs under consideration).
There has been swift advancement in transforming the abstract cross-shard concepts into concrete specifications for discussion and ultimately prototypes. Monitor this area of progress, especially if you are a dapp developer. We plan to offer something you can comprehend, experiment with, and provide feedback on in the upcoming months.
Relationship of Stateless Ethereum to eth2
Another significant R&D initiative is occurring in parallel to eth2 termed “Stateless Ethereum”. Stateless Ethereum aims to tackle the state size expansion issue. It empowers participants to validate blocks without needing to retain the totality of the state locally. Currently, there is an implicit input in the Ethereum state transition function: the entirety of the state. With Stateless Ethereum, proofs (witnesses) regarding the necessary state will be included within blocks. This enables a block to be transitioned/validated solely as a function of the block itself.
For users, this means a landscape where you can track the chain and even trace segments of the state that interest you, without having to store the entire state. Some network participants may choose to store the complete state (block producers, block explorers, state-for-a-fee providers), but the majority will likely become partial stateful participants.
For eth2, this constitutes a crucial technical mechanism to guarantee that nodes and validators can authenticate and safeguard the protocol without the burden of retaining the complete user state of each shard. Instead, validators will probably choose to be block producers for certain shards, while the base validator may only validate stateless blocks. Stateless Ethereum is an incredibly valuable enhancement to the eth2 vision, maintaining the foundational aspect of the sharded protocol very slim. While we plan for eth2 to operate in a stateless manner, we do have some alternatives should the stateless option ultimately prove unworkable (although I remain quite optimistic about statelessness myself 😄).
I won’t delve deeper into Stateless Ethereum for this post. Just be aware that it’s an exhilarating parallel R&D path to ensure Ethereum’s longevity in the long term. If you’re keen to learn more, explore Griffin’s The 1.x Files blog series.
tl;dr
Eth2 represents a monumental effort to deliver an advanced, next-gen, highly-scalable, and secure, decentralized consensus for Ethereum. Numerous teams and hundreds of individuals are collaborating every day to turn this into reality. The path we have selected is challenging, yet remarkable advancements have been made and continue to occur.
The essence of this new mechanism is just around the corner.
If you aspire to be a validator, now is the moment to immerse yourself. Support the multi-client approach by experimenting with multiple clients, and help foster a robust foundation of diverse clients from eth2’s inception.
If you’re a user or dapp developer, continue pushing on Ethereum today while we persist in creating this more secure and scalable environment for you. When the time arrives, the transition to eth2 will be as seamless as possible.
Thank you to the amazing teams and individuals maintaining Ethereum’s vitality today; thank you to all those preparing for Ethereum’s future in eth2; and thank you to all the users and developers that contribute to making Ethereum extraordinary 🚀
