Solana Validator Effectiveness Rating
Methodology that underlies the effectiveness rating of validator and operator performance on Solana.
This effectiveness rating is a measure of how well a validator has been performing its rewarded duties over time.
It seeks to be a true measure of performance, attributing points to the validator for attributes of their onchain footprint that they have influence over (e.g. whether they are successfully voting for slots, which means they are actively participating in consensus), while dampening the effect of metrics that allow randomness to creep into the measurement (e.g. voting performance of other validators).
The Solana Effectiveness Rating is designed to work with the validator’s vote account as the base unit of account. What the metric effectively does is to tally up the duties that a validator is called to perform over a given time period (e.g. 2 epochs, a day, a year), and assign points to those instances when duties were completed successfully. The result is a percentage score that illustrates the job completion rate of a given validator vote account.
Components of Effectiveness
Proposer Effectiveness
Voting Effectiveness
In the following sections, we examine those one-by-one and then synthesize them together in what constitutes the effectiveness rating.
Proposer Effectiveness
Validators are expected to produce blocks of transactions. On Solana, the primary unit of time is an epoch which consists of 432,000 slots. Each slot has a target duration of around 400 milliseconds, which puts one epoch at 48 to 52 hours currently.
It is important to understand that in Solana, a slot presents an opportunity for a block to be produced by the selected leader (i.e. proposer) for that slot. A block is a batch of transactions produced by said leader. When blocks are not produced in slots, these slots are considered missed as the validator has failed to fulfill its duty to produce a block and advance the chain.
Another thing to note is that consensus messages by validators on Solana (i.e. votes) are propagated through transactions as well. These vote transactions are important when it comes to confirming the latest state of the chain, which we go deeper into inVoting Effectiveness. Given these are transactions as well, it is entirely possible for proposers to build blocks only containing vote transactions. These are perfectly valid blocks however this means that no user transactions were confirmed/executed for the time being which degrades the experience of end-users, and arguably goes against the main purpose of the chain.
Given this, we attribute scores to a validator’s proposer effectiveness based on the following:
| Case | Block Produced | Score |
|---|---|---|
A.1 | Has end-user transactions | 1 |
A.2 | Only vote transactions | 0.25 |
B | Missed | 0 |
Case A.1
In this case, a block was produced, containing both end-user and vote transactions, in which case a full point is given for the proposer effectiveness ratio.
Case A.2
In this case, a block was produced but it only contains vote transactions (i.e. Vote Only Blocks). End-user transactions are not proposed which degrades the experience of the user.
Case B
Lastly, in this case no block is produced and the proposer has completely missed their block production duty. This gets no point.
Thus we compute the proposer effectiveness for a validator as:
Voting Effectiveness
Validators are called to vote for every observed block that is on the correct fork or what they perceive to be the correct state of the chain.
Forking occurs frequently on Solana due to the fast-paced nature of the network. When a validator is slow to produce a block X, the next validator may begin their allocated slot X+1 before it has received the previous block, therefore assuming it does not exist and building its block on top of the prior block X-1. This causes two competing forks both building on the ancestor X-1. The validator at X+2 will then determine which to continue building upon based on when it receives them. Most likely it will receive X first and continue on that and X+1 will end up being a dead fork and those slots will appear as skipped (i.e. missed).
A validator can only correctly vote for a slot with a block, and a confirmed fork means a slot that has more than ⅔ of stake voting on it. For every successful vote on a valid fork, the validator receives one credit which is used to compute the voting rewards at every epoch.
Using this information, we can then compute for the first component of voting effectiveness which is the voting success rate:
The second aspect of voting effectiveness is vote latency. As mentioned earlier, voting on Solana occurs through onchain transactions. The earliest a validator can vote for a slot is immediately the slot after and the latest they can do so is 512 slots after.
Vote latency is the slot distance between the slot being voted for and the slot where the vote transaction lands. While this does not directly impact the number of credits a validator receives (i.e. a correct vote will always get one credit), this impacts the speed of achieving consensus as it affects the time it takes to reach the ⅔ stake threshold needed to confirm a block.
There is also a system design proposal in Solana called Timely Vote Credits which will modulate voting rewards based on latency. The proposal has passed an onchain governance vote, and is undergoing testing on Solana's testnet. Implementation to come soon on Solana mainnet-beta.
Given that the protocol has yet to fully penalize validators for vote latency, we have decided to not include this in the vote effectiveness calculation for now. Once the Timely Vote Credits feature has been enabled on mainnet-beta, we will modulate voting success rate accordingly using latency, as aligned with the protocol.
Effectiveness Rating
In order to come up with a unified validator effectiveness score, we combine proposer effectiveness and voting effectiveness in a weighted average. We attribute the following weights to each component based on the distribution of rewards using the most recent data from Solana:
Proposer Effectiveness: 30%
Voting Effectiveness: 70%
Such that the formula becomes:
If a particular validator has not been assigned a particular duty for a given time period, we only take the effectiveness of whatever duty they were assigned with and had a chance to fulfil. Example is when a validator has not been assigned a block production duty for a given period. In this case, effectiveness == voting effectiveness.
We remain open to inputs from the community on how to best capture the effectiveness of a validator on Solana. Head on over to our forum to share your thoughts.
Last updated