Metropolis is at long last (nearly) upon us! The fork for Byzantium, the initial and more substantial segment of Metropolis, was successfully executed on the testnet over two weeks ago, and the anticipated date for the fork on the mainnet has been scheduled for block 4.37 million, expected on Oct 17. New attributes encompass opcodes like REVERT and RETURNDATACOPY, along with precompiles that can aid in supporting a diverse range of cryptographic algorithms. Concurrently, enhancements to Ethereum core code, Whisper, Swarm, and Ethereum’s forthcoming scaling strategies have also been occurring.
- Casper PoC4 has been unveiled. This includes an execution of the fork choice rule, the Casper contract, along with a comprehensive pyethereum library. However, a full node capable of connecting to the network is not yet available.
- A “testing language” has been adopted, enabling us to swiftly implement tests for the Casper chain. This can also theoretically facilitate testing for the proof of work chain, with an extension to sharding underway.
- The implementation of a proof of concept for sharding is ongoing.
- The execution of the account redesign within the sharding PoC is ongoing.
- The Casper documents continue to progress.
- The count of message types in Casper FFG has been reduced from 2 to 1, which will further streamline the incentive structure. A formal proof of the safety property has been formulated. This will be integrated in PoC5.
- The fork choice rule has also been made simpler.
- Pyethapp now accommodates python 3.
- The “scalable light client data availability verification” notice has been revised with an enhanced scheme
The research group has onboarded one full-time new developer and one part-time to Viper, with a growing number of community contributors. Features currently in development include:
- Cross-contract ABI calling
- ABI logging
- Support for ECADD (Elliptic Curve Addition), ECMUL (Elliptic Curve Multiplication) and various precompiles
- Support for features related to the existing sharding specifications
- General code restructuring
- Additional examples
- Simplified compatibility with ERC20 and other ABIs developed in Solidity
cpp-ethereum
Progress is being made in synchronizing with the test and main networks. Currently, cpp-ethereum can import warp snapshots and proceed with syncing from the snapshot block. The implementation of Byzantium has been completed. Since the previous roundup:
Geth
Since the last roundup, Geth experienced a significant release (v1.7.0) followed by a maintenance release (v1.7.1) primarily aimed at the imminent Byzantium hard fork (enabled for all networks as of v1.7.1).
Alongside Byzantium, several features have been finalized (some of which you may have noted in the v1.7.0 Megara blogpost):
- EVM log storage and indexing has been thoroughly overhauled, significantly reducingthe filtering duration of the complete chain for contract occurrences by 2-3 magnitudes, decreasing from minutes to less than a second.
- The primary peer-to-peer protocol employed by all Ethereum sub-protocols was enhanced to adopt Snappy compression, which reduces the bandwidth required for a swift synchronization from 33.6GB to 13.5GB, with similar decreases anticipated for everyday operational bandwidth needs as well. Since operational bandwidth is contingent on updated peers, its impact will manifest in the long term.
- Rinkeby was augmented with an authenticated signer and a bootnode thanks to the INFURA team, which should assist in locating peers more rapidly, as well as offer a more stable experience due to the surplus available bandwidth capacity on the test network.
Several key features we are presently developing (but may require some time due to Devcon preparations) include:
- Generational disk writes to retain recent trie modifications in system memory and only occasionally store checkpoint tries. This capability considerably accelerates processing batches of blocks and substantially reduces the final disk space utilized. Nonetheless, there are still some minor hurdles to address to guarantee we do not disrupt sync for remote nodes due to the updated data availability pattern in the network.
- An innovative variant of fast sync that avoids downloading the entire state-trie node by node; instead, it retrieves only trie leaf nodes (along with corresponding Merkle proofs) and constructs the final trie locally, which should significantly impact the quantity of network packets required for the initial sync. However, aside from implementing it, this feature necessitates adequate benchmarking to validate it and an EIP process to discuss its inclusion as the next version of the `eth` protocol.
- Version 5 of the DEVp2p discovery protocol (currently in the experimental stage and utilized by the light clients) to allow it to operate simultaneously with v4 on the same UDP port, while also addressing the issues encountered from running the light client on top.
- Version 2 of the light client protocol, founded on the new bloombits filtering approach, which will ultimately enable light clients to retrieve and subscribe to EVM events. The updated version should also facilitate tracking transactions for chain inclusion without the burden of downloading entire blocks merely to validate their transaction contents.
- Investigation into an ultra-light operational mode where clients connect to a set of user-defined backing servers akin to the classical client-server model, but where the node can cryptographically cross-verify the multiple backends to prevent any malfeasance.
Numerous bug fixes have also been integrated into both releases, which you can glance through at the GitHub milestones for v1.7.0 and v1.7.1.
Testing
Test scenarios for Byzantium are prepared and some clients have successfully passed all of them. Following the last round up, we:
Yellow Paper
All Byzantium modifications have been recorded on pull-requests. The next concern to resolve is the handling of the empty account states on precompiled contracts, where different clients behave differently. A GitHub issue has been established to discuss this matter.
Formal Verification
@pirapira presented talks at the ARM Research Summit and at TU Berlin. We have implemented RLP in OCaml, accessible with opam install rlp.
Bamboo
We have commenced drafting a specification. External interfaces are currently being developed.
EVM 1.5
Solidity assembly can now accommodate EIP #615 opcodes for subroutines and the like, thanks to Alex. Initial implementations ofboth EIP #615 and EIP #616 have been composed and assessed by Andrei, and testing has commenced. Greg is preparing to present preliminary performance metrics at Devcon.
web3.js
The web3.js 1.0 beta has received substantial community contributions, which is thrilling as it looks like 1.0 is already widely utilized. Caution is advised while using account generation and signing features, as these have not undergone an audit. Such accounts should not store significant volumes of ether. We are presently at beta.22. If all progresses smoothly, we anticipate a 1.0.0 Release candidate by Devcon3.
web3.py
Recently, Web3.py introduced enhancements and features, including:
- Launch of the Middleware API
- Revamping the testing infrastructure to evaluate against go-ethereum, pyethereum, and py-evm.
- Corrections and enhancements to the web3.eth.sign and web3.sha3 APIs, as well as advancements towards a cohesive method of handling various string encodings.
- A trial integration with the ethereum-tester library.
py-evm
Development continues effectively, bringing us closer to an early alpha release, which will permit running py-evm as a light client.
- Implementation of the fundamental DevP2P networking API.
- Journaling for changes in the state database.
Ethereum Tester
Ethereum Tester is a novel tool for Python developers aimed at providing a standard API for testing applications that engage with the EVM. The library utilizes a pluggable backend system that enables swapping the underlying VM implementation. This library can also be used seamlessly with web3.py via the experimental EthereumTesterProvider
Remix
Two exceptionally important PRs have been recently implemented: (1) a terminal that enables tracking transactions and interacting with web3, and (2) a redesign of the UI.
We are currently addressing bugs and assisting the community in adapting to this new GUI.
Several new features are in development for release in the upcoming weeks:
– Highlighting of declarations and references, basic autocomplete features, and renaming
https://github.com/ethereum/browser-solidity/pull/819
- Command interpreter for the terminal (enabling commands for Remix from the terminal)
https://github.com/ethereum/browser-solidity/pull/806
- Save transactions as scenarios that can be replayed in any context
https://github.com/ethereum/browser-solidity/pull/802
https://github.com/ethereum/browser-solidity/pull/799
Solidity
We are nearing completion of the new ABI en/decoder utilizing the Julia language. This facilitates the passing of structs and arbitrarily nested arrays through function calls. It also imposes stricter restrictions regarding invalid input data, including invalid array lengths. You can enable it using pragma experimental ABIEncoderV2; With a similar directive, pragma experimental “v0.5.0”; you can enable forthcoming breaking changes that we are implementing sequentially. You can now use f.selector to retrieve the selector (also referred to as the “function hash”) of a function. Utilizing the default visibility of a function will now generate a warning, which should make you cognizant of unintentional public functions. Ultimately, Solidity is advising regarding (and more rigorously enforcing if you choose to already enable v0.5.0 features) breaches of “view” (previously termed “constant”) and “pure” designations for functions. Following Byzantium, invoking a “view” function will utilize the new STATICCALL opcode and thus uphold the “view” attribute of the function at the VM level.
Mist
Launched version 0.9.1:
- Incorporated the light client functionality as an experimental feature
- Enhanced electron, geth, and solidity to bolster security
- A new core developer @marcgarreau commenced work on restructuring state management in Mist using Redux
Security
A cross-language differential fuzzer for operations on the alt_bn128 elliptic curve, necessary for executing zkSNARK verifications, has been created in three distinct clients (Parity, Geth, and CPP) to identify inconsistencies among these implementations. Powered by the libFuzzer engine, the program autonomously generates inputs that lead to new code paths in any of the implementations. Furthermore, it verifies that the outputs of each implementation are identical for any input.
Building on this effort, a fuzzer aimed at testing the Parity and Geth implementations of the EVM for equivalence is under development.
We have been progressing cross-EVM fuzzing initiatives to detect inconsistencies among EVM-implementations (Geth, CPP, and Parity), and have increased the bounty rewards for issues related to consensus or denial-of-service (geth).
The hive-testing framework has been improved to evaluate four clients (geth, cpp, py, parity) with the new Byzantium tests.
An independent audit of Mist is nearing completion, and an RCE vulnerability has been fixed and released (kudos to Yoonho Kim via the bug bounty program). We strongly advise upgrading to the latest version of Mist!
EthereumJS
Since the previous roundup, numerous new contributors have joined to support development. EthereumJS is now successfully passing most Byzantium state and blockchain evaluations. The development work continues with overall bug fixes and code refinement. The most remarkable of these initiatives is the restructuring of the library to facilitate support for JavaScript big integers.
Swarm
The Swarm team has integrated significant new updates and features into master, which includes the network testing and simulations framework for p2p alongside a 3D visualizer, enhanced scheduling for subdividing and reassembling data — the pyramid chunker with the newly introduced Binary Merkle Tree chunk hash, enhancing compact inclusion proofs verifiable on the blockchain, and providing more informative error messages. The network layer overhaul (our major features for POC3) has successfully completed the initial phase, and pss v1.0 is now development-ready, featuring Diffie-Hellman key exchange and constantly updated keystream.
Our team is engaging in increasingly extensive community collaboration to develop specific features on our roadmap, enabling external contributors to feel part of the larger team. We are establishing 5 working groups concerning 5 of our most pertinent subprojects: network simulation, pss node-to-node messaging, media and data streaming, database services, and the swap swear and swindle framework aimed at incentivizing bandwidth sharing and long-term data storage. The latter is gaining traction as work proceeds on the third orange paper, with a draft version scheduled to be introduced on 11th October at the London Ethereum Meetup.
We have just released the preliminary agenda for the Devcon3 breakout session focusing on p2p technology and blockchain (10:30am-4:30pm, 4th November, 2017).
Whisper
- A new core developer @gballet has begun addressing issues with Whisper v5 and updating the documentation.
- v5 message delivery was disrupted by a series of updates over the summer; corrections have been implemented and will be integrated shortly.
- EIP 627 has been finalized, and implementation will commence soon.
Our appreciation goes to all developers and team leaders who contributed to the sections regarding their projects!
