Blog

A deep-dive into Eth-staking-smith

No items found.

January 13, 2023

•

5 min read

January 13, 2023

•

5 min read

Authors: Jennifer Parak, Maksym Kulish

One of the most important events of 2022 in the crypto community was The Merge upgrade of the Ethereum protocol, which switched Ethereum from a Proof-of-Work legacy chain implementation to a Proof-of-Stake Beacon chain. It has proved that principal innovation is possible for the oldest and largest decentralized systems, without any disruption to the protocol users.

At Chorus One, we worked on securing next-generation Ethereum since the Beacon Chain took off in 2020, and we operate multiple thousands of validators on the mainnet today. Our new product OPUS — an Ethereum Validation-as-a-Service API — is designed to enable any organization and individual to run staking validator clients on Ethereum Beacon Chain, with a non-custodial, permission-less approach where we require customers to specify their own withdrawal and fee recipient addresses, so they remain in possession of both their stake funds and rewards. This post focuses on the technology implementation of validator keys provision and storage approach within our Validation-as-a-Service API product and shows off some challenges we faced and solutions we created in the process.

Background

Ethereum Staking Keys

The Merge has introduced two new types of keys involved in securing the Ethereum chain, in addition to legacy chain wallet keys that are remaining unchanged within Beacon Chain [1]. These keys are composed of the Signing (Validator) key pair and the Withdrawal key pair. In addition to new key functionality, the Signing key is also using a new cryptographic signature scheme, called BLS, which stands for Boneh–Lynn–Shacham. This means older key generation tools will not work for creating Signing keys. BLS signatures, specifically those over the BLS12–381 curve are used in Beacon chain block signatures and attestations. This makes it possible to aggregate multiple signatures and verify them in a single operation, which is an outstanding improvement in scalability [2].

Like most other Proof-Of-Stake blockchains, next-generation Ethereum depends on the functioning of validators for securing the transaction flow. Validators are members of the network who lock a portion of their Ethereum coins (with a minimum amount of 32 ETH) to become responsible for proposing new signed blocks of transactions, and verifying such signatures of other validators, which is called attesting. Normally, every Ethereum validator should attest signatures for a slot once per Ethereum epoch (around 6.4 minutes); and for every slot, in every epoch, one validator is pseudo-randomly chosen to produce a block of transactions to be attested by others. Validators are being rewarded for both block proposals and block attestations. The mechanism of signing the blocks and verifying the signatures of others relies on the Signing key pair. The verification mechanism works because every public part of a Signing key (Public Signing Key) is published on-chain, so every signature done with the private part of the Signing key (Private Signing Key) can be verified by every other validator. Despite having the power for creating blockchain content, the Signing keys can not be used to move any funds including staking funds, and they only listen for and sign the transaction content provided by the peering network of Ethereum nodes.

The Withdrawal key pair is neither used for blocks nor for attestations, but it has control over staked funds. After the Shanghai fork, withdrawals will be activated, which will enable the funds to be moved to an owner-controlled withdrawal address specified in the deposit contract. With EIP-4895 withdrawals will be enabled in a push-based fashion [3], such that funds that were previously locked on the consensus layer on depositing are automatically pushed to the execution layer as a system-level operation. This means users won’t have to pay any gas for a withdrawal transaction. For users who have specified a BLS withdrawal address in their deposit contract, they would need to broadcast a BLS_TO_EXECUTION_CHANGE message to the beacon chain to update their withdrawal address to an execution address.

Finally, when the validator successfully proposes a block, a special Fee Recipient address receives the accumulated gas fees from the block. Since the Fee Recipient is not directly involved in staking, we will largely omit it in this post.

More information about different types of keys involved in Ethereum staking can be found in the following resources: [4], [5]

Managing Validator keys for OPUS Validation-as-a-Service API

As part of the OPUS Validation-as-a-Service API, we require customers to retain ownership of Withdrawal keys, so that staked funds can never be controlled or accessed by Chorus One. A Signing key, however, is different: since Chorus One is a responsible party for hosting and maintaining the Ethereum validator, the inner workings of the Validation-as-a-Service API require us to generate, load, and store Signing keys. Thus, a robust solution for key management is an essential part of our Validation-as-a-Service API.

Early into the project lifecycle, we used the staking deposit command line interface (CLI) provided by the Ethereum Foundation (https://github.com/ethereum/staking-deposit-cli). While the staking CLI is a great tool for solo/home stakers, we realized that it was not designed for our use case. First of all, staking-deposit-cli by default stores the newly generated keystore into a filesystem, posing a potential security threat from leaking key material. While it is possible to use infrastructure-specific workarounds like ramdisks to mitigate the threat, such workarounds would add complexity and failure points to the platform. The open-source nature of staking-deposit-cli allowed us to fork the source code and modify it to cater to our needs, but the lack of thoroughly automated test suites meant we had a hard time syncing our changes with upstream updates. Finally, all of our codebase is Rust, and having to support Python CLI within the infrastructure, including keeping a good security track record by timely patching all the Python dependencies, puts an additional burden on the development team. In the end, we decided to pursue an alternative approach to generating keys, which we describe in the next paragraph.

Eth-staking-smith

Having endured even more difficulties with staking-cli when generating Ethereum keys on a large scale, our Ethereum Team decided to tackle the problem during our company-wide engineering hackathon where we built an MVP for an Ethereum key generation tool written in Rust. This was the birth of the Eth-staking-smith project.

Eth-staking-smith can be used as a CLI tool or as a Rust library to generate Signing keys and deposit data derived from a new mnemonic or to regenerate deposit data from an existing mnemonic. These use cases were implemented, in order to provide the same functionality as the staking-deposit-cli whilst avoiding all problems mentioned above.

Example command to generate keys from a newly generated mnemonic:

eth-staking-smith new-mnemonic --chain mainnet --keystore_password testtest --num_validators 1

‍

Example command to generate keys from an existing mnemonic:

eth-staking-smith existing-mnemonic --chain mainnet --keystore_password testtest --mnemonic "entire habit bottom mention spoil clown finger wheat motion fox axis mechanic country make garment bar blind stadium sugar water scissors canyon often ketchup" --num_validators 1 --withdrawal_credentials "0x0100000000000000000000000000000000000000000000000000000000000001"

‍

For both use cases, Eth-staking-smith will generate the following key material:

Private Signing keys
Keystores
Mnemonic (existing or newly generated)
Deposit data smart contract properties

Let’s zoom into the generated key material

Private Signing keys

As mentioned above, the Private Signing key is the key used to provide a signature to any action taken by the validator. The Eth-staking-smith Signing key output is done without encryption because in our use case, we use remote API to store the key material immediately upon the generation. Remote API implements encryption for both data transfer and data at rest. We decided to make keystore use optional, but Eth-staking-smith can still generate encrypted keystores for the users who need that.

Example:

{
…
 "private_keys": [
    "6d446ca271eb229044b9039354ecdfa6244d1a11615ec1a46fc82a800367de5d"
  ]
…
}

‍

Keystores

The keystore is an encrypted version of the private Signing key in the specified format [6]. When generating keys with eth-staking smith, a keystore password can be specified and in that case, the keystore data will be output. Using a key derivation function (e.g. <code-text>scrypt<code-text>, or <code-text>pbkdf2<code-text>), a decryption-key is derived using the given passphrase and a set of strong built-in derivation arguments. The example below highlights the field <code-text>function<code-text> that shows the key derivation function used. The keystore is a useful alternative that is less vulnerable to an attacker than storing the private Signing keys in a plaintext file, since they would need the keystore file, as well as the passphrase to decrypt the file.

Example:

{
…
 "keystores": [
  {
    "crypto": {
    "checksum": {
      "function": "sha256",
      "message": "af14321c3083de535a0dd895b4e2fb156e6b0eda346120c8d7afb5277d3a489f",
      "params": {}
    },
    "cipher": {
    "function": "aes-128-ctr",
    "message": "8032685ad92a579e66328bbd6c747e41497dc6897c17cebbd83958394943924b",
    "params": {
      "iv": "da5699bb18ee7fea6095634a2fa05d18"
    }
  },
  "kdf": {
    "function": "pbkdf2",
    "message": "",
      "params": {
        "c": 262144,
        "dklen": 32,
        "prf": "hmac-sha256",
        "salt": "afb431f05b7fe02f253d9bc446ac686776541d38956fa6d39e14894f44e414d8"
       }
    }
 },
  "description": "",
  "name": null,
  "path": "m/12381/3600/0/0/0",
  "pubkey": "8844cebb34d10e0e57f3c29ada375dafe14762ab85b2e408c3d6d55ce6d03317660bca9f2c2d17d8fbe14a2529ada1ea",
  "uuid": "6dbae828-d0f0–42ed-9c06-d9079642ea08",
  "version": 4
  }
],
…
}

‍

Mnemonic

The mnemonic, passed in by the user or the one generated, is returned as part of the output so that the user can store it safely. Further information on mnemonics can be found under the reference [7].

Example:

{
…
 "mnemonic": {
    "seed": "ski interest capable knee usual ugly duty exercise tattoo subway delay upper bid forget say"
  },
…
}

‍

Deposit data

Finally, the deposit data is returned, which is used to make the deposit of 32ETH using the Ethereum deposit contract to activate the validator. One of the most important fields in the deposit data is the withdrawal credentials.

By default, withdrawal credentials are BLS addresses derived from the mnemonic, however, there exists a use case where a user might want to overwrite the derived withdrawal credentials with already existing ones.

The BLS address format is called <code-text>0x00<code-text> credentials, and is actually set to be deprecated sometime after withdrawals will be enabled. Another alternative way to provide withdrawal credentials is to use a legacy Ethereum wallet address, prefixed by <code-text>0x01<code-text>. Ethereum will be pivoting from using <code-text>0x00<code-text> (formerly eth2) to <code-text>0x01<code-text> execution (formerly eth1) addresses. To learn more about this, we recommend watching the panel from Devcon 2022 [8] and looking into the Ethereum specification [4]. Eth-staking-smith, therefore, allows the user to pass in a <code-text>0x00<code-text>, <code-text>0x01<code-text> execution withdrawal credentials, as well as an execution address to overwrite the withdrawal credentials.

Example deposit data with BLS credentials (<code-text>--withdrawal_credentials 0x0045b91b2f60b88e7392d49ae1364b55e713d06f30e563f9f99e10994b26221d<code-text>)

{
…
"deposit_data": [
  {
    "amount": 32000000000,
    "deposit_cli_version": "2.3.0",
    "deposit_data_root": "95ac4064aabfdece592ddeaba83dc77cf095f2644c09e3453f83253a8b7e0ae1",
    "deposit_message_root": "6a0c14a9acd99ab4b9757f2ff2f41e04b44c0c53448fdf978c118841cd337582",
    "fork_version": "00001020",
    "network_name": "goerli",
    "pubkey": "8844cebb34d10e0e57f3c29ada375dafe14762ab85b2e408c3d6d55ce6d03317660bca9f2c2d17d8fbe14a2529ada1ea",
    "signature": "82effe6d57877b7d642775ae3d56f9411d41a85218b552c6318925c7ba23f7470ebe3a35045e2fc36b0e848e6f4ec1d503f2014dc5a7ad94a267f5b237f2475b5da9ff358fbd5a8e9f497f1db0cfb15624e686991d002077a6cd4efda8bdc67e",
    "withdrawal_credentials": "01000000000000000000000071c7656ec7ab88b098defb751b7401b5f6d8976f"
  }
],
…
}

‍

Below we present a full-fledged example output of the key material generated by Eth-staking-smith:

Command:

eth-staking-smith existing-mnemonic --chain goerli --keystore_password testtest --mnemonic "ski interest capable knee usual ugly duty exercise tattoo subway delay upper bid forget say" --num_validators 1

‍

Output:

{
"deposit_data": [
  {
    "amount": 32000000000,
    "deposit_cli_version": "2.3.0",
    "deposit_data_root": "2abc7681f73a01acbc1974ab47119766bf57d94f86a72828f8875295f5bd92de",
    "deposit_message_root": "bfd9d2c616eb570ad3fd4d4caf169b88f80490d8923537474bf1f6c5cec5e56d",
    "fork_version": "00001020",
    "network_name": "goerli",
    "pubkey": "8844cebb34d10e0e57f3c29ada375dafe14762ab85b2e408c3d6d55ce6d03317660bca9f2c2d17d8fbe14a2529ada1ea",
    "signature": "97c0ad0d4f721dc53f33a399dbf0ff2cab6f679f4efdcdaa9f8bdd22cd11b5e37c12fdd2cd29369b1b907a51573a9ef60f93d768fd2d47a99b5d55fe6516a87b9090e16c42f5a8fcbf91d24883359bffb074a02d6d4d7f6c3cd04c8e09f8dc02",
    "withdrawal_credentials": "0045b91b2f60b88e7392d49ae1364b55e713d06f30e563f9f99e10994b26221d"
  }
],
"keystores": [
  {
    "crypto": {
    "checksum": {
      "function": "sha256",
      "message": "af14321c3083de535a0dd895b4e2fb156e6b0eda346120c8d7afb5277d3a489f",
      "params": {}
  },
  "cipher": {
  "function": "aes-128-ctr",
  "message": "8032685ad92a579e66328bbd6c747e41497dc6897c17cebbd83958394943924b",
  "params": {
    "iv": "da5699bb18ee7fea6095634a2fa05d18"
    }
  },
  "kdf": {
    "function": "pbkdf2",
    "message": "",
    "params": {
      "c": 262144,
      "dklen": 32,
      "prf": "hmac-sha256",
      "salt": "afb431f05b7fe02f253d9bc446ac686776541d38956fa6d39e14894f44e414d8"
      }
    }
  },
  "description": "",
  "name": null,
  "path": "m/12381/3600/0/0/0",
  "pubkey": "8844cebb34d10e0e57f3c29ada375dafe14762ab85b2e408c3d6d55ce6d03317660bca9f2c2d17d8fbe14a2529ada1ea",
  "uuid": "6dbae828-d0f0–42ed-9c06-d9079642ea08",
  "version": 4
  }
],
"mnemonic": {
  "seed": "ski interest capable knee usual ugly duty exercise tattoo subway delay upper bid forget say"
},
"private_keys": [
  "6d446ca271eb229044b9039354ecdfa6244d1a11615ec1a46fc82a800367de5d"
  ]
}

‍

Security Improvements

Since writing key material on disk was a major security vulnerability for us, Eth-staking-smith removes this issue entirely by not writing any files on disk.

To avoid heavy-lifting and re-creating crypto primitives from scratch, we’re re-using functionalities from the lighthouse client implementation [9] for key generation, which builds on top of blst — a BLS12–381 signature library [10], which is currently undergoing formal verification.

For entropy collection, one customization was made: Eth-staking-smith defers entropy collection to the operating system by using <code-text>getrandom()<code-text> on Linux and thereby making use of Linux’s state-of-the-art randomness approach.

Tweaking Security <> Performance parameters

Finally, since key generation at scale was notoriously slow for us with staking-deposit-cli, we took initiative to add additional arguments for our users to tweak performance <> security parameters depending on their specific use case.

As per our use case, our API does not require the keystore file, but only the private key in raw format. We, therefore, enable the user to opt-out of keystore generation in order to improve performance. This can be done by omitting the <code-text>--keystore_password<code-text> argument as follows:

eth-staking-smith new-mnemonic --chain goerli --num_validators 1

‍

We measured that omitting keystore speeds up the key generation process by 99%. The key generation performance we experience with Eth-staking-smith is consistently sub-second, with slight variability depending on hardware and platform.

In case the user requires the keystore file for redundancy, there’s another option to speed up the keystore generation process by choosing a different key-derivation function. By default, Eth-staking-smith will use <code-text>pbkdf2<code-text> to derive the decryption key to achieve better performance. There’s also the option to use <code-text>scrypt<code-text> which offers better security, however, consequently, worse performance. This can be done by choosing the key-derivation function using the <code-text>--kdf<code-text> argument as follows:

eth-staking-smith new-mnemonic --chain goerli --keystore_password testtest --num_validators 1 --kdf scrypt

‍

Converting BLS withdrawal to execution address

As mentioned above, users who had previously specified a BLS (0x00) withdrawal address, will need to make a request to the beacon chain to update their validators’ withdrawal address to point to an execution address. To perform this operation, the user will need to have the BLS withdrawal key mnemonic phrase. Once done, withdrawals will be automatically funded on the execution address.

Eth-staking-smith enables the user to generate a signed <code-text>BLS_TO_EXECUTION_CHANGE<code-text> message which they can send to the beacon chain to update their withdrawal address.

eth-staking-smith bls-to-execution-change --chain mainnet --mnemonic "entire habit bottom mention spoil clown finger wheat motion fox axis mechanic country make garment bar blind stadium sugar water scissors canyon often ketchup" --validator_index 0 --withdrawal_credentials "0x0045b91b2f60b88e7392d49ae1364b55e713d06f30e563f9f99e10994b26221d" --execution_address "0x71C7656EC7ab88b098defB751B7401B5f6d8976F"

‍

Users can use the response to make the request to the beacon node as follows:

```
curl -H "Content-Type: application/json" -d '{
  "message": {
    "validator_index": 0,
    "from_bls_pubkey": "0x0045b91b2f60b88e7392d49ae1364b55e713d06f30e563f9f99e10994b26221d",
    "to_execution_address": "0x71C7656EC7ab88b098defB751B7401B5f6d8976F"
  },
  "signature": "0x9220e5badefdfe8abc36cae01af29b981edeb940ff88c438f72c8af876fbd6416138c85f5348c5ace92a081fa15291aa0ffb856141b871dc807f3ec2fe9c8415cac3d76579c61455ab3938bc162e139d060c8aa13fcd670febe46bf0bb579c5a"
}' http://localhost:3500/eth/v1/beacon/pool/bls_to_execution_change
```

‍

Conclusion

Throughout this post, we explained the basics of Ethereum Beacon Chain block validation, the key material involved in the process, and walked through an automation tool we created at Chorus One for Proof-of-Stake key management.

We hope the tool can be useful to some of our readers, especially those who use Rust for their blockchain automation work. It is also open-source, and we will welcome bug reports and pull requests on Github.

If the reader is interested in using OPUS Validation-as-a-Service API which builds upon that automation, you are welcome to join the wait-list for private beta, contact via sales@chorus.one.