“`html
The Go Ethereum team is extremely proud to finally unveil Geth 1.5, which can nearly be considered a thorough internal overhaul of the Go Ethereum (go-ethereum) codebase.
We’ve incorporated a substantial number of modifications in this release, and merely enumerating them wouldn’t do them justice. Instead, we’ve chosen to articulate them in a more conversational manner, clarifying not just what’s new, but also why it’s essential and why it’s impressive!
Go Ethereum website
The go-ethereum project has never truly had a website. There was something auto-generated ages ago by GitHub, but it couldn’t genuinely be referred to as a respectable website as it lacked valuable information, didn’t appear particularly appealing, and lacked proper maintenance. At that time, it was acceptable since the dedicated developers were more focused on the source repository and wiki pages than on a web presence.
However, as Ethereum gains fame and momentum, we are now striving to make Geth, its code, and related resources more accessible and user-friendly for everyone involved, not merely a select group of core developers. As an initial step in this direction, we’ve started constructing a new website for go-ethereum. You can explore it at: https://geth.ethereum.org.
The website still has a vast journey ahead, but we’ve made every effort to include information that is not obtainable elsewhere yet is crucial for anyone commencing with go-ethereum: a comprehensive installation manual for every platform, and a downloads section compiling all our binaries from every build service we manage. You can anticipate a detailed developer guide in the coming weeks, followed by a thorough user guide.
Library access
Go Ethereum, one of the three initial clients alongside C++ Ethereum and Py Ethereum, has developed concurrently with the Ethereum networking and consensus protocol specification. This journey involved rapid prototyping, frequent rewrites, and shelved features. The net outcome was a codebase that functioned effectively but was challenging to embed into other projects due to its untidy internals.
In the Geth 1.4.x series, we initiated the process of untangling go-ethereum, but it required more time than we initially expected to tidy up the majority of the public API pathways. With Geth 1.5, we’ve finally reached the juncture where we can confidently endorse our programmatic APIs as both functional and something we aim to support in the long term. The final touches are still being refined, but we’re certain you’ll appreciate the outcome immensely!
Our primary focus areas were: a) streamlined client-side account management, b) remote clients via HTTP, IPC, and WebSockets; c) contract interactions and binding generation, and d) in-process embedded nodes. With these four principal use cases addressed, we’re optimistic that most server-side or mobile applications can thrive.
Check out the teaser slide presentation about our new APIs presented by @karalabe at Devcon2, our Ethereum developers conference in Shanghai, a few weeks ago.
https://www.youtube.com/watch?v=R0Ia1U9Gxjg
Mobile platforms
With Geth 1.5 emphasizing library reusability, it is only sensible to explore how far we can extend the boundaries. There has been considerable investigation into running (or at least interfacing with) Ethereum from web browsers; our current release is aimed at doing so from desktop/server processes. The only element missing from the puzzle was mobile devices… until now.
The 1.5 release of go-ethereum debuts our first experimental endeavor at providing genuine Android and iOS library reusability of our codebase. This manifests in the form of a native Java and ObjC wrapper around our code, officially packaged as an Android archive and iOS XCode framework. The former is more developed, while the latter requires some API refinements due to the challenges in automatically wrapping Go to ObjC/Swift code.
We’re also offering native dependencies for both platforms through Maven Central packages (or Sonatype for development snapshots) for Android, and CocoaPod packages for iOS. Since this is the very first instance we’re making the strides to these package managers, there might be a few obstacles that could emerge, so we’ll issue a separate announcement when both are stable for use. Until then, we recommend sticking to the downloadable library bundles.
Experimental protocols
The 1.5 release of Geth serves as a foundational attempt for the future direction and features we’d like to explore and stabilize in forthcoming releases. In our view, the optimal method to advance the desired new features is to launch them as experimental (opt-in exclusively) protocols so anyone can engage with them and provide feedback. In light of this, we’ve integrated several components that we (and hopefully the community) have anticipated for quite some time.
Discovery v5
If you’ve experimented with joining the official testnet (Morden) or executing a publicly accessible private testnet, you know it can often take a significant amount of time to synchronize, as the node frequently seems to remain idle.
One of the primary causes of testnet sync issues is that the peer discovery protocol cannot distinguish between machines operating different blockchains, or even entirely different network protocols. The only method to discover suitable peers is to connect to as many peers as possible and retain the ones that are relevant. This strategy works for the mainnet, but for smaller protocols (testnet, light clients, swarm, whisper) it’s akin to searching for a needle in a haystack of advertised peers.
Geth 1.5 includes a new iteration of the peer discovery protocol that enhances the “shooting in the dark” approach with topic-based peer-querying. To summarize, peers can actively search for other peers that have explicitly advertised feature sets, protocols, or configurations. This new discovery protocol should allow nodes to quickly find others of interest, even when there are only a few among thousands of “boring.”
“`ones.
Kindly be aware: the v5 discovery protocol is in an experimental stage, thus it is presently enabled solely for light clients and light servers. This will enable us to accumulate valuable insights and scrutinize its behavior/anomalies without altering the primary Ethereum P2P network in any manner.
Light client
Blockchains are vast entities, there’s no doubt about it. Regardless of optimizations, there will perpetually be devices that lack sufficient resources to actively participate in blockchain networks (e.g., smartphones, IoT gadgets). Although unexpected, we witnessed this phenomenon during the DoS assault, which led to HDDs struggling with synchronization.
The only viable solution for executing a blockchain on minuscule embedded gadgets is for them to operate as light clients, where they do not carry the complete responsibility of supporting the network, but instead only manage their own operation. This not only benefits the smaller devices, but also aids the network as a whole by eliminating slow connections, thereby rendering the core network smaller, tighter, and more efficient.
We are excited to finally feature an alpha version of a light client within Geth 1.5. It can synchronize in minutes (or even less) and require only megabytes of storage space, yet it interacts fully with the Ethereum blockchain and is even accessible through the Mist browser (though there have been some glitches).
You can run Geth as a light client using the –light flag. If you are operating a full node, feeling generous, and aren’t managing a sensitive production setup, consider activating the light server protocol to assist small devices in the network via –lightserv 25 –lightpeers 50 flags (the first configures the percentage of system resources allocated to light clients, while the second determines the number of light clients permitted to connect).
Swarm
In addition to the consensus protocol, the Ethereum vision comprises two other foundational elements: real-time dark messaging (Whisper) and decentralized file storage (Swarm). All three are essential to create genuinely decentralized, high-availability applications. Whisper is generally available as an experimental protocol, but Swarm has always seemed like a distant aspiration.
With the release of 1.5, we are thrilled to present an initial proof-of-concept implementation of the Swarm protocol for developers to experiment with. It is launched as an independent daemon process (and inherently executable binary), not embedded within Geth. This ensures users can run Swarm against any Ethereum client while also avoiding any conflicts with the main node’s operations.
RPC subscriptions
If you have developed a more intricate DApp against a Geth node (or any other Ethereum node for that matter), you may have observed that polling the node for data on RPC can detrimentally impact performance. Not polling it, conversely, negatively affects user experience as the DApp becomes less responsive to new events.
The dilemma lies in the fact that polling for changes is an inefficient choice since most of the time there are no alterations, only the potential for one. A superior strategy, rather than querying the node for changes intermittently, is to subscribe to specific events and allow the node to provide notifications when a change occurs. Geth 1.5 facilitates this through a new RPC subscription framework. Any DApp (or external process) can subscribe to a range of events and depend on the node to alert when necessary. As this mechanism is unattainable over standard HTTP (as it is over IPC), the 1.5 release also offers support for operating the RPC API via WebSockets.
JavaScript tracing
During the recent DoS attacks, we dedicated considerable time investigating various transactions to enhance our understanding of their functionalities. These efforts involved trying to generate diverse traces, scrutinizing precisely what the EVM executes, and how that impacts the foundational implementation.
Even though Geth has presented an EVM tracing API endpoint for quite some time, it did not offer much detail regarding configurability. It executed the EVM bytecode, returned the incurred opcodes, any errors that occurred, and optionally a difference of stack, memory, and storage adjustments performed by the transaction. This is helpful, but costly in terms of resources, both to generate and to relay through the RPC layer.
With the 1.5 release, we are introducing a new method for tracing transactions, a JavaScript map-reduce framework. Rather than the traditional trace options previously available, you will now be able to define two JavaScript functions: a mapper called for each opcode with access to all trace data, and a reducer invoked at the conclusion of the trace to determine the final data to return to the requester.
The benefit of the JavaScript tracing approach is that it runs inside the Go Ethereum node itself, allowing the tracer to access all data available at no performance cost and collect only what is necessary while discarding everything else. It is also significantly simpler to compose custom trace code than parsing some predetermined output format.
Vendored dependencies
Prior to the 1.4.x release phases of Geth, the go-ethereum codebase utilized the godep tool as its dependency manager due to Go itself not offering a suitable alternative aside from manually copying dependencies or depending on upstream repositories to remain stable over time.
This scenario was unfortunate as it carried several disadvantages: a) building go-ethereum required both a custom tool as well as understanding the idiosyncrasies of said tool, b) dependency updates via godep were exceedingly cumbersome as they polluted the local workspaces and could not function in temporary directories, and c) utilizing go-ethereum as a library was incredibly challenging as dependencies were not an integral part of the Go workflow.
With the Geth 1.5 launch, we’ve transitioned to the officially endorsed method of vendoring dependencies (fully supported starting with Go 1.6), specifically by positioning all external dependencies into directories native to the Go compiler and toolchain (vendor), and adopting a different dependency management instrument to more efficiently manage our requirements (named trash).
From an external viewpoint, the primary advantage is eliminating the need to grapple with any arbitrary dependency management tool that we might have used during the construction of go-ethereum, or when employing it as a library in other ventures. Now you can adhere to the traditional Go tools and everything will function seamlessly!
Build infrastructure
From the inception of the Ethereum project, all official clients relied on a build infrastructure constructed and maintained by @caktux utilizing Amazon EC2 instances, Ansible, and a substantial suite of Python scripts (referred to as the Ethereum Buildbot).
Initially, this infrastructure functioned efficiently when the original implementations provided a limited number of significant platform, architecture, and deliverable bundles. However, as time progressed and projects began to concentrate on smaller, distinct builds, the maintenance burden escalated as the buildbot began to degrade. When the maintainer departed from the Ethereum project, it became evident that a shift to new build flows was necessary, although establishing them was a substantial undertaking.
A significant milestone of the Geth 1.5 release is the full transition from the traditional build infrastructure to one that is entirely self-sufficient within our repositories. We migrated all builds onto the various continuous integration services we depend on (Travis, AppVeyor, CircleCI), and integrated all the build code ourselves as an inherent part of the go-ethereum sources.
The outcome is that we can now compile everything the go-ethereum project requires without relying on specific service providers or external code beyond the team’s jurisdiction. This will guarantee that go-ethereum won’t suffer from peculiar missing packages or outdated package managers.
Build artifacts
Commencing with Geth 1.5, we are distributing significantly more build artifacts than previously. Our two primary deliverables are archives containing only Geth, and bundles containing Geth along with any other tools considered beneficial for developers and/or users of the Ethereum platform. These artifacts are pre-compiled for every stable release as well as for each individual develop commit to a broad range of targets: Linux (386, amd64, arm-5, arm-6, arm-7 and arm64), macOS (amd64) and Windows (386, amd64).
One of our feature enhancements includes library bundles for utilizing go-ethereum in mobile applications. On Android, we offer official builds for .aar archives containing binaries for 386, amd64, arm-7 and arm64, covering all popular mobile devices as well as local simulator builds. On iOS, we provide official XCode framework bundles containing binaries for amd64, arm-7 and arm64, encompassing all iPhone architectures along with local simulator builds.
In addition to the standalone binary archives, we are also disseminating all the above in the form of Homebrew bundles for macOS, launchpad PPA packages for Ubuntu, NSIS installers for Windows (Chocolatey distribution will require further administrative steps to address), Maven Central dependencies for Android and CocoaPods dependencies for iOS!
All of the artifacts mentioned above can be accessed from the go-ethereum downloads page.
Digital signatures
For an extended period, our binary distributions were somewhat disorganized, sometimes offering checksums, and other times not, depending on who prepared the release packages and the amount of time available to finalize loose ends. The absence of checksums frequently resulted in inquiries from users on how to authenticate bundles circulating online, and more critically, it led to the emergence of several counterfeit developer and project clones distributing malware.
To resolve this definitively, starting from Geth 1.5 onward, all our officially produced archives will be digitally signed using several OpenPGP keys. We will no longer depend on checksums to establish the authenticity of our distributed bundles, but will require security-conscious users to verify any downloads through their attached PGP signatures. The list of signing keys we employ can be found in our OpenPGP Signatures section.
Repository branches
A short time before the Frontier release last July, we transitioned to a source repository model where the master branch housed the latest stable code, while develop contained the bleeding-edge source code we were working on.
This repository model, however, had several drawbacks: a) newcomers to the project wishing to contribute frequently began working on master, only to discover later that their efforts were based on outdated code; b) each time a major release occurred, master needed to be force-pushed, which appeared quite negative from a repository history viewpoint; c) developers attempting to utilize the go-ethereum codebase in their own projects rarely recognized that a more advanced branch was available.
Starting with Geth 1.5, we will discontinue maintaining a separate master branch for the latest stable, and a develop branch for the latest edge, instead, we will designate master as the default and development branch of the project, with each stable release generation possessing its own perpetually living branch (e.g. release/1.4, release/1.5). The release branches will allow individuals to rely on older generations (e.g. 1.4.x) without encountering unexpected git issues related to history rewrites. Furthermore, having master as the default development branch would enable developers to work with the latest code.
