Heartfelt appreciation to Gavin Wood for igniting my interest in advancements in abstraction, along with Martin Becze, Vlad Zamfir, and Dominic Williams for their continuous dialogues.
For an extended period, we have been transparent about our intentions to further enhance the Ethereum protocol progressively and our extensive development blueprint, deriving insights from our oversights that we either were unable to rectify in time for 1.0 or only recognized after the fact. Nonetheless, the development cycle for the Ethereum protocol has recommenced, with a Homestead release on the horizon, and we have subtly begun crafting proof-of-concepts for the most significant achievement we had set for ourselves in our development plan: Serenity.
Serenity is aimed to encompass two primary features: abstraction, a concept I first elaborated on in this blog entry here, and Casper, our security-deposit-based proof of stake mechanism. Moreover, we are examining the possibility of incorporating at least the framework that will facilitate the gradual rollout of our scalability propositions, while also addressing parallelizability issues mentioned here – a significant immediate benefit for private blockchain versions of Ethereum with nodes being hosted on massively multi-core dedicated servers, and even the public chain might experience a 2-5 times enhancement in scalability. In recent months, research on Casper and the formalization of scalability and abstraction (for example, through EIP 101) has been advancing quickly between myself, Vlad Zamfir, Lucius Greg Meredith, and a few others. I am now delighted to declare that the initial proof of concept release for Serenity, albeit in a very limited capacity suitable only for experimentation, is now available.
The PoC can be executed by navigating to the ethereum directory and running python test.py (ensure you download and install the latest Serpent from https://github.com/ethereum/serpent, develop branch); if the output resembles something like this then you are good to go:
vub@vub-ThinkPad-X250 15:01:03 serenity/ethereum: python test.py REVERTING 940534 gas from account 0x0000000000000000000000000000000000000000 to account 0x98c78be58d729dcdc3de9efb3428820990e4e3bf with data 0x Warning (file "casper.se.py", line 74, char 0): Warning: function return type inconsistent! Running with 13 maximum nodes Warning (file "casper.se.py", line 74, char 0): Warning: function return type inconsistent! Warning (file "casper.se.py", line 74, char 0): Warning: function return type inconsistent! Length of validation code: 57 Length of account code: 0 Joined with index 0 Length of validation code: 57 Length of account code: 0 Joined with index 1 Length of validation code: 57
This simulates 13 nodes operating the Casper+Serenity protocol at a 5-second block interval; this is fairly close to the upper limit of what the client can currently manage, though note that (i) this is python, while C++ and Go will likely demonstrate much better performance, and (ii) this represent all nodes running on a single computer concurrently, so in a more “normal” setting you could anticipate python Casper managing at least ~169 nodes (though, on the contrary, we intend for consensus overhead to fall well below 100% CPU time, so these two points combined do NOT imply that Casper will be expected to run with thousands of nodes!). If your machine cannot sufficiently handle the 13 nodes, try python test.py 10 to simulate with 10 nodes instead (or python test.py 7 for 7 nodes, and so forth). Naturally, research into enhancing Casper’s efficiency, albeit likely at the expense of slightly slower convergence to finality, is still ongoing, and these challenges should diminish over time. The network.py file mimics a fundamental P2P network interface; future endeavors will involve replacing this with actual computers functioning on a real network.
The code is segmented into several main files as follows:
- serenity_blocks.py – the code that defines the block class, the state class, and the block and transaction-level transition functions (approximately 2x simpler than prior)
- serenity_transactions.py – the code that details transactions (roughly 2x simpler than before)
- casper.se.py – the Serpent code for the Casper contract, which encourages correct betting
- bet.py – the betting strategy for Casper and complete client implementation
- ecdsa_accounts.py – account code that allows you to replicate the account validation capability currently available in a Serenity context
- test.py – the script for testing
- config.py – configuration parameters
- vm.py – the virtual machine (more efficient implementation at fastvm.py)
- network.py – the network simulator
In this article, we will concentrate on the abstraction functionalities, so serenity_blocks.py, ecdsa_accounts.py, and serenity_transactions.py are the most pivotal; for the subsequent article addressing Casper in Serenity, casper.se.py and bet.py will be the main focus.
Abstraction and Accounts
At present, two categories of accounts exist within Ethereum:externally managed accounts, governed by a private key, and agreements, governed by code. For externally managed accounts, we define a specific digital signature method (secp256k1 ECDSA) and a specific sequence number (also known as nonce) system, wherein each transaction must encompass a sequence number incremented by one from the last, to avert replay attacks. The key modification we plan to implement to enhance abstraction is this: instead of maintaining these two separate types of accounts, we will transition to having merely one – agreements. There exists also a unique “entry point” account, 0x0000000000000000000000000000000000000000, from which anyone can send from by executing a transaction. Thus, rather than integrating the signature+nonce validation logic of accounts into the protocol, it is now the user’s responsibility to embed this into a contract that will protect their own account.
The most straightforward type of agreement that proves useful is likely the ECDSA verification agreement, which simply delivers the exact same capability that currently exists: transactions are processed only if they hold valid signatures and sequence numbers, with the sequence number incrementing by 1 upon a successful transaction. The code for the agreement appears as follows:
# We presume that data follows this format:
# bytes 0-31: v (ECDSA sig)
# bytes 32-63: r (ECDSA sig)
# bytes 64-95: s (ECDSA sig)
# bytes 96-127: sequence number (previously referred to as "nonce")
# bytes 128-159: gas price
# bytes 172-191: to
# bytes 192-223: value
# bytes 224+: data
# Obtain the hash for transaction signing
~mstore(0, ~txexecgas())
~calldatacopy(32, 96, ~calldatasize() - 96)
~mstore(0, ~sha3(0, ~calldatasize() - 64))
~calldatacopy(32, 0, 96)
# Invoke ECRECOVER agreement to retrieve the sender
~call(5000, 1, 0, 0, 128, 0, 32)
# Validate sender; trigger exception if incorrect
if ~mload(0) != 0x82a978b3f5962a5b0957d9ee9eef472ee55b42f1:
~invalid()
# Sequence number handling
with minusone = ~sub(0, 1):
with curseq = self.storage[minusone]:
# Confirm sequence number validity; trigger exception if invalid
if ~calldataload(96) != curseq:
~invalid()
# Increment sequence number
self.storage[minusone] = curseq + 1
# Execute the sub-call and disregard output
with x = ~msize():
~call(msg.gas - 50000, ~calldataload(160), ~calldataload(192), 160, ~calldatasize() - 224, x, 1000)
# Compensate for gas
~mstore(0, ~calldataload(128))
~mstore(32, (~txexecgas() - msg.gas + 50000))
~call(12000, ETHER, 0, 0, 64, 0, 0)
~return(x, ~msize() - x)
This code would reside as the agreement code of the user’s account; when the user intends to dispatch a transaction, they would send a transaction (from the zero address) to this account, embedding the ECDSA signature, the sequence number, the gas price, destination address, ether value, and the actual transaction data, utilizing the encoding outlined in the code above. The code verifies the signature against the transaction gas ceiling and the supplied data, and then checks the sequence number, and if both are accurate it subsequently increments the sequence number, transmits the intended message, and lastly sends a second message to cover gas costs (note that miners can statically inspect accounts and opt not to process transactions directed at accounts that lack gas payment code at the conclusion).
A significant outcome of this is that Serenity introduces a framework wherein all transactions (that meet fundamental formatting verifications) are legitimate; transactions presently deemed “invalid” will in Serenity merely have no consequence (the invalid opcode in the previous code simply references an unused opcode, promptly initiating an exit from code execution). This implies that transaction inclusion in a block no longer guarantees that the transaction was indeed executed; to address this, every transaction now receives a receipt entry indicating whether it was successfully executed or not, offering one of three return codes: 0 (transaction not executed due to block gas cap), 1 (transaction executed but resulted in an error), 2 (transaction executed successfully); more elaborate information can be supplied if the transaction produces data (which is now automatically logged) or generates its own logs.
The primary substantial advantage of this is that it provides users significantly greater freedom to innovate regarding account policies; potential avenues include:
- Bitcoin-style multisig, where an account anticipates signatures from multiple public keys simultaneously before dispatching a transaction, rather than receiving signatures one at a time and storing intermediate results in memory
- Alternative elliptic curves, encompassing ed25519
- Enhanced integration for more sophisticated cryptography, e.g. ring signatures, threshold signatures, ZKPs
- More sophisticated sequence number systems allowing higher degrees of parallelism, enabling users to send multiple transactions from a single account and have them processed more swiftly; envision a blend of a traditional sequence number with a bitmask. Additionally, one can incorporate timestamps or block hashes into the validity verification in various innovative ways.
- UTXO-based token management – some users disfavor Ethereum’s approach of using accounts instead of Bitcoin’s “unspent transaction output” (UTXO) paradigm for managing token possession, partly for privacy considerations. Now, you can establish a system within Ethereum that is genuinely UTXO-based, and Serenity no longer overtly “privileges” one over the other.
- Advancements in payment models – for certain dapps, “contract pays” is a more suitable model than “sender pays” since senders may lack ether; individual dapps can now establish such models, and if coded in a manner allowing miners to statically analyze and confirm they will actually receive compensation, they can promptly accept them (essentially, this offers what Rootstock aims to do with optional author-pays, but in a far more abstract and adaptable manner).
- Stronger integration for “ethereum alarm clock”-style applications – the verification code for an account doesn’t have to validate signatures; it could also verify Merkle proofs of receipts, state of other accounts, etc.
In all these scenarios, the central point is that through abstraction, all these additional mechanisms become considerably simpler to code since there’s no longer a necessity to establish a “pass-through layer” to channel the information through Ethereum’s default signature method; when no application is exceptional, every application becomes.
A particularly fascinating implication is that under the current blueprint for Serenity, Ethereum will optionally be quantum-safe; if you are concerned about the NSA possessing access to a quantum computer and wish to safeguard your account more securely, you can take personal measures to do so.transition to Lamport signatures at any moment. Proof of stake further strengthens this, as even if the NSA possessed a quantum computer and no one else, they would be unable to exploit that to conduct a 51% assault. The only cryptographic security premise that will endure at the protocol level in Ethereum is the collision-resistance of SHA3.
Consequently, as a result of these modifications, transactions are set to become significantly more straightforward. Rather than containing nine fields, as is currently the case, transactions will only have four fields: destination address, data, start gas, and initialization code. The destination address, data, and start gas remain unchanged; “init code” is a field that can optionally include the code for contract creation for the address to which you are sending.
The rationale behind the latter mechanism is outlined as follows. One crucial characteristic that Ethereum presently offers is the capability to send to an account prior to its existence; there is no necessity to already possess ether to establish a contract on the blockchain before you can receive ether. To facilitate this in Serenity, an account’s address can be predicted from the intended initialization code for the account in advance, utilizing the formula sha3(creator + initcode) % 2**160 where creator refers to the account that initiated the contract (the zero account by default), and initcode constitutes the initialization code for the contract (the result of executing the initcode will become the contract code, just as it is currently for CREATEs). Thus, you can generate the initialization code for your contract locally, compute the address, and allow others to send to that address. Then, when you wish to dispatch your first transaction, you incorporate the init code in the transaction, and the init code will be executed automatically, creating the account before executing the actual transaction (this logic can be found implemented here).
Abstraction and Blocks
Another clear distinction that will be introduced in Serenity is the complete division of blocks (which are currently merely bundles of transactions), state (i.e., current contract storage, code, and account balances), and the consensus layer. Consensus incentivization occurs within a contract, and consensus-level objects (e.g., PoW, wagers) should be included as transactions sent to a “consensus incentive manager contract” if one aims to incentivize them.
This should simplify the process of taking the Serenity codebase and replacing Casper with any consensus algorithm – Tendermint, HoneyBadgerBFT, subjective consensus, or even traditional proof of work; we encourage research in this area and strive for maximum adaptability.
Abstraction and Storage
At present, the “state” of the Ethereum system is rather intricate and comprises various components:
- Account balance, code, nonce, and storage
- Gas limit, difficulty level, block number, timestamp
- The last 256 block hashes
- During block processing, the transaction index, receipt tree, and current gas consumption
These data structures are distributed across different locations, including the block state transition function, the state tree, the block header, and prior block headers. In Serenity, this will be significantly streamlined: while many of these variables will still be present, they will all be transferred to specialized contracts in storage; as such, the ONLY notion of “state” that will persist is a tree, which can be mathematically considered as a mapping {address: {key: value} }. Accounts will essentially be trees; account code will be stored at key “” for each account (not changeable by SSTORE), balances will reside in a specialized “ether contract,” and sequence numbers will be determined individually by each account to manage storage. Receipts will also transition to storage; they will be contained in a “log contract” where the contents are overwritten with each block.
This enables the State object in implementations to be greatly simplified; what remains is a two-level map of tries. The scalability enhancement may increase this to three levels of tries (shard ID, address, key), but that has yet to be determined, and even so, the complexity will be significantly reduced compared to the present situation.
It’s important to note that the relocation of ether into a contract does NOT equate to complete ether abstraction; indeed, it can be argued that it is not a substantial deviation from the current state, as opcodes that interact with ether (the value parameter in CALL, BALANCE, etc.) still remain for backward compatibility. Instead, this represents merely a reorganization of how data is stored.
Future Plans
For POC2, the ambition is to advance abstraction even further. Presently, significant complexity persists within the block and transaction-level state transition function (e.g., updating receipts, gas limits, the transaction index, block number, state roots); the objective will be to construct an “entry point” object for transactions that manages all this additional “boilerplate logic” required per transaction, along with an “initiate block” and “conclude block” entry point. A theoretical ultimate aim is to devise a protocol that contains only one entry point, with the state transition function consisting simply of sending a message from the zero address to the entry point, containing the block data. The goal here is to minimize the size of the actual consensus-critical client implementation as much as feasible, integrating as much logic directly into the Ethereum code itself; this guarantees that Ethereum’s multi-client architecture can persist even under a vigorous development agenda willing to embrace hard forks and some degree of new complexity to reach our objectives of transaction speed and scalability without necessitating an excessively large ongoing development workload and security auditing.
In the longer run, I plan to continue creating proof-of-concepts in Python, while the Casper team collaborates on enhancing efficiency and validating the safety and correctness of the protocol; eventually, the protocol will be sufficiently mature to support a public testnet of some sort, possibly (though not certainly) with actual value on-chain to provide stronger incentives for individuals to attempt to “compromise” Casper as we inevitably anticipate that they will once the main chain becomes operational. This is merely a preliminary step, albeit a crucial one, as it represents the first instance when the research surrounding proof of stake and abstraction is finally transitioning from theoretical concepts, calculations on whiteboards, and blog narratives to a functioning implementation written in code.
The subsequent segment of this series will address the other main feature of Serenity, the Casper consensus algorithm.