Aptos Blockchain Deep Dive
For a deeper understanding of the lifecycle of an Aptos transaction (from an operational perspective), we will follow a transaction on its journey, from being submitted to an Aptos fullnode, to being committed to the Aptos blockchain. We will then focus on the logical components of Aptos nodes and take a look at how the transaction interacts with these components.
Life of a Transaction
- Alice and Bob are two users who each have an account on the Aptos blockchain.
- Alice’s account has 110 Aptos Coins.
- Alice is sending 10 Aptos Coins to Bob.
- The current sequence number of Alice’s account is 5 (which indicates that 5 transactions have already been sent from Alice’s account).
- There are a total of 100 validator nodes — V1 to V100 on the network.
- An Aptos client submits Alice’s transaction to a REST service on an Aptos Fullnode. The fullnode forwards this transaction to a validator fullnode which in turn forwards it to validator V1.
- Validator V1 is a proposer/leader for the current round.
The Journey
In this section, we will describe the lifecycle of transaction T5, from when the client submits it to when it is committed to the Aptos blockchain.
For the relevant steps, we’ve included a link to the corresponding inter-component interactions of the validator node. After you are familiar with all the steps in the lifecycle of the transaction, you may want to refer to the information on the corresponding inter-component interactions for each step.
The arrows in all the visuals in this article originate on the component initiating an interaction/action and terminate on the component on which the action is being performed. The arrows do not represent data read, written, or returned.
The lifecycle of a transaction has five stages:
- Accepting: Accepting the transaction
- Sharing: Sharing the transaction with other validator nodes
- Proposing: Proposing the block
- Executing and Consensus: Executing the block and reaching consensus
- Committing: Committing the block
We’ve described what happens in each stage below, along with links to the corresponding Aptos node component interactions.
Transactions are validated upon entering a mempool and prior to execution by consensus. The client only learns of validation results returned during the initial submission via the REST service. Transactions may silently fail to execute, especially in the case where the account has run out of utility token or changed its authentication key in the midst of many transactions. While this happens infrequently, there are ongoing efforts to improve the visibility in this space.
Client submits a transaction
An Aptos client constructs a raw transaction (let’s call it Traw5) to transfer 10 Aptos Coins from Alice’s account to Bob’s account. The Aptos client signs the transaction with Alice’s private key. The signed transaction T5 includes the following:
- The raw transaction.
- Alice’s public key.
- Alice’s signature.
The raw transaction includes the following fields:
| Fields | Description |
|---|---|
| Account address | Alice’s account address |
| Payload | Indicates an action or set of actions Alice’s behalf. In the case this is a Move function, it directly calls into Move bytecode on the chain. Alternatively, it may be Move bytecode peer-to-peer transaction script. It also contains a list of inputs to the function or script. For this example, it is a function call to transfer an amount of Aptos Coins from Alice account to Bob’s account, where Alice’s account is implied by sending the transaction and Bob’s account and the amount are specified as transaction inputs. |
| Gas unit price | The amount the sender is willing to pay per unit of gas, to execute the transaction. This is represented in Octas. |
| Maximum gas amount | The maximum gas amount in APT Alice is willing to pay for this transaction. Gas charges are equal to the base gas cost covered by computation and IO multiplied by the gas price. Gas costs also include storage with an Apt-fixed priced storage model. This is represented in Octas. |
| Expiration time | Expiration time of the transaction. |
| Sequence number | The sequence number (5, in this example) for an account indicates the number of transactions that have been submitted and committed on-chain from that account. In this case, 5 transactions have been submitted from Alice’s account, including Traw5. Note: a transaction with sequence number 5 can only be committed on-chain if the account sequence number is 5. |
| Chain ID | An identifier that distinguishes the Aptos networks (to prevent cross-network attacks). |
Accepting the transaction
| Description | Aptos Node Component Interactions |
|---|---|
| 1. Client → REST service: The client submits transaction T5 to the REST service of an Aptos fullnode. The fullnode uses the REST service to forward the transaction to its own mempool, which then forwards the transaction to mempools running on other nodes in the network. The transaction will eventually be forwarded to a mempool running on a validator fullnode, which will send it to a validator node (V1 in this case). | 1. REST Service |
| 2. REST service → Mempool: The fullnode’s mempool transmits transaction T5 to validator V1’s mempool. | 2. REST Service, 1. Mempool |
| 3. Mempool → Virtual Machine (VM): Mempool will use the virtual machine (VM) component to perform transaction validation, such as signature verification, account balance verification and replay resistance using the sequence number. | 4. Mempool, 3. Virtual Machine |
Sharing the transaction with other validator nodes
| Description | Aptos Node Component Interactions |
|---|---|
| 4. Mempool: The mempool will hold T5 in an in-memory buffer. Mempool may already contain multiple transactions sent from Alice’s address. | Mempool |
| 5. Mempool → Other Validators: Using the shared-mempool protocol, V1 will share the transactions (including T5) in its mempool with other validator nodes and place transactions received from them into its own (V1) mempool. | 2. Mempool |
Proposing the block
| Description | Aptos Node Component Interactions |
|---|---|
| 6. Consensus → Mempool: — As validator V1 is a proposer/leader for this transaction, it will pull a block of transactions from its mempool and replicate this block as a proposal to other validator nodes via its consensus component. | 1. Consensus, 3. Mempool |
| 7. Consensus → Other Validators: The consensus component of V1 is responsible for coordinating agreement among all validators on the order of transactions in the proposed block. | 2. Consensus |
Executing the block and reaching consensus
| Description | Aptos Node Component Interactions |
|---|---|
| 8. Consensus → Execution: As part of reaching agreement, the block of transactions (containing T5) is shared with the execution component. | 3. Consensus, 1. Execution |
| 9. Execution → Virtual Machine: The execution component manages the execution of transactions in the VM. Note that this execution happens speculatively before the transactions in the block have been agreed upon. | 2. Execution, 3. Virtual Machine |
| 10. Consensus → Execution: After executing the transactions in the block, the execution component appends the transactions in the block (including T5) to the Merkle accumulator (of the ledger history). This is an in-memory/temporary version of the Merkle accumulator. The necessary part of the proposed/speculative result of executing these transactions is returned to the consensus component to agree on. The arrow from “consensus” to “execution” indicates that the request to execute transactions was made by the consensus component. | 3. Consensus, 1. Execution |
| 11. Consensus → Other Validators: V1 (the consensus leader) attempts to reach consensus on the proposed block’s execution result with the other validator nodes participating in consensus. | 3. Consensus |
Committing the block
| Description | Aptos Node Component Interactions |
|---|---|
| 12. Consensus → Execution, Execution → Storage: If the proposed block’s execution result is agreed upon and signed by a set of validators that have the quorum of votes, validator V1’s execution component reads the full result of the proposed block execution from the speculative execution cache and commits all the transactions in the proposed block to persistent storage with their results. | 4. Consensus, 3. Execution, 4. Execution, 3. Storage |
Alice’s account will now have 100 Aptos Coins, and its sequence number will be 6. If T5 is replayed by Bob, it will be rejected as the sequence number of Alice’s account (6) is greater than the sequence number of the replayed transaction (5).
Aptos node component interactions
In the Life of a Transaction section, we described the typical lifecycle of a transaction (from transaction submission to transaction commit). Now let’s look at the inter-component interactions of Aptos nodes as the blockchain processes transactions and responds to queries. This information will be most useful to those who:
- Would like to get an idea of how the system works under the covers.
- Are interested in eventually contributing to the Aptos blockchain.
You can learn more about the different types of Aptos nodes here:
For our narrative, we will assume that a client submits a transaction TN to a validator VX. For each validator component, we will describe each of its inter-component interactions in subsections under the respective component’s section. Note that subsections describing the inter-component interactions are not listed strictly in the order in which they are performed. Most of the interactions are relevant to the processing of a transaction, and some are relevant to clients querying the blockchain (queries for existing information on the blockchain).
The following are the core components of an Aptos node used in the lifecycle of a transaction:
Fullnode
Validator node
REST Service
Any request made by a client goes to the REST Service of a fullnode first. Then, the submitted transaction is forwarded to the validator fullnode, which then sends it to the validator node VX.
1. Client → REST Service
A client submits a transaction to the REST service of an Aptos fullnode.
2. REST Service → Mempool
The REST service of the fullnode transfers the transaction to its mempool. After mempool does some initial checks, the REST Service will return a status to the client indicating whether the transaction was accepted or rejected. For example, out-of-date transactions will be rejected: mempool will accept the transaction TN only if the sequence number of TN is greater than or equal to the current sequence number of the sender’s account.
3. Mempool -> Mempool
The mempool on the fullnode sends the transaction to the mempool of a validator fullnode, which then sends the transaction to validator node VX’s mempool. Note that the transaction will not be sent to the next mempool (or passed to consensus) until the sequence number matches the sequence number of the sender’s account. Furthermore, each mempool performs the same initial checks upon receiving a transaction, this may result in a transaction being discarded on its way to consensus. The current implementation of mempool does not provide any feedback if a transaction is discarded during this process.
4. REST Service → Storage
When a client performs a read query on the Aptos blockchain (for example, to get the balance of Alice’s account), the REST service interacts with the storage component directly to obtain the requested information.
Virtual Machine (VM)
The Move VM verifies and executes transaction scripts written in Move bytecode.
1. Virtual Machine → Storage
When mempool requests the VM to validate a transaction via VMValidator::validate_transaction(), the VM loads the transaction sender’s account from storage and performs verifications, some of which have been described in the list below.
- Checks that the input signature on the signed transaction is correct (to reject incorrectly signed transactions).
- Checks that the sender’s account authentication key is the same as the hash of the public key (corresponding to the private key used to sign the transaction).
- Verifies that the sequence number for the transaction is greater than or equal to the current sequence number for the sender’s account. Completing this check prevents the replay of the same transaction against the sender’s account.
- Verifies that the program in the signed transaction is not malformed, as a malformed program cannot be executed by the VM.
- Verifies that the sender’s account balance contains at least the maximum gas amount multiplied by the gas price specified in the transaction, which ensures that the transaction can pay for the resources it uses.
2. Execution → Virtual Machine
The execution component utilizes the VM to execute a transaction via ExecutorTask::execute_transaction().
It is important to understand that executing a transaction is different from updating the state of the ledger and persisting the results in storage. A transaction TN is first executed as part of an attempt to reach agreement on blocks during consensus. If agreement is reached with the other validators on the ordering of transactions and their execution results, the results are persisted in storage and the state of the ledger is updated.
3. Mempool → Virtual Machine
When mempool receives a transaction from other validators via shared mempool or from the REST service, mempool invokes VMValidator::validate_transaction() on the VM to validate the transaction.
For implementation details refer to the Move Virtual Machine README.
Mempool
Mempool is a shared buffer that holds the transactions that are “waiting” to be executed. When a new transaction is added to the mempool, the mempool shares this transaction with other validator nodes in the system. To reduce network consumption in the “shared mempool,” each validator is responsible for delivering its own transactions to other validators. When a validator receives a transaction from the mempool of another validator, the transaction is added to the mempool of the recipient validator.
1. REST Service → Mempool
- After receiving a transaction from the client, the REST service sends the transaction to its own mempool, which then shares the transaction with the mempool of a validator fullnode. The mempool on the validator fullnode then shares the transaction with the mempool of a validator.
- The mempool for validator node VX accepts transaction TN for the sender’s account only if the sequence number of TN is greater than or equal to the current sequence number of the sender’s account.
2. Mempool → Other validator nodes
- The mempool of validator node VX shares transaction TN with the other validators on the same network.
- Other validators share the transactions in their respective mempools with VX’s mempool.
3. Consensus → Mempool
- When the transaction is forwarded to a validator node and once the validator node becomes the leader, its consensus component will pull a block of transactions from its mempool and replicate the proposed block to other validators. It does this to arrive at a consensus on the ordering of transactions and the execution results of the transactions in the proposed block.
- Note that just because a transaction TN was included in a proposed consensus block, it does not guarantee that TN will eventually be persisted in the distributed database of the Aptos blockchain.
4. Mempool → VM
When mempool receives a transaction from other validators, mempool invokes VMValidator::validate_transaction() on the VM to validate the transaction.
Consensus
The consensus component is responsible for ordering blocks of transactions and agreeing on the results of execution by participating in the consensus protocol with other validators in the network.
1. Consensus → Mempool
When validator VX is a leader/proposer, the consensus component of VX pulls a block of transactions from its mempool via: Mempool::get_batch(), and forms a proposed block of transactions.
2. Consensus → Other Validators
If VX is a proposer/leader, its consensus component replicates the proposed block of transactions to other validators.
3. Consensus → Execution, Consensus → Other Validators
- To execute a block of transactions, consensus interacts with the execution component. Consensus executes a block of transactions via
BlockExecutorTrait::execute_block()(Refer to Consensus → execution) - After executing the transactions in the proposed block, the execution component responds to the consensus component with the result of executing these transactions.
- The consensus component signs the execution result and attempts to reach agreement on this result with other validators.
4. Consensus → Execution
If enough validators vote for the same execution result, the consensus component of VX informs execution via BlockExecutorTrait::commit_blocks() that this block is ready to be committed.
Execution
The execution component coordinates the execution of a block of transactions and maintains a transient state that can be voted upon by consensus. If these transactions are successful, they are committed to storage.
1. Consensus → Execution
- Consensus requests execution to execute a block of transactions via:
BlockExecutorTrait::execute_block(). - Execution maintains a “scratchpad,” which holds in-memory copies of the relevant portions of the Merkle accumulator. This information is used to calculate the root hash of the current state of the Aptos blockchain.
- The root hash of the current state is combined with the information about the transactions in the proposed block to determine the new root hash of the accumulator. This is done prior to persisting any data, and to ensure that no state or transaction is stored until agreement is reached by a quorum of validators.
- Execution computes the speculative root hash and then the consensus component of VX signs this root hash and attempts to reach agreement on this root hash with other validators.
2. Execution → VM
When consensus requests execution to execute a block of transactions via BlockExecutorTrait::execute_block(), execution uses the VM to determine the results of executing the block of transactions.
3. Consensus → Execution
If a quorum of validators agrees on the block execution results, the consensus component of each validator informs its execution component via BlockExecutorTrait::commit_blocks() that this block is ready to be committed. This call to the execution component will include the signatures of the validators to provide proof of their agreement.
4. Execution → Storage
Execution takes the values from its “scratchpad” and sends them to storage for persistence via DbWriter::save_transactions(). Execution then prunes the old values from the “scratchpad” that are no longer needed (for example, parallel blocks that cannot be committed).
For implementation details refer to the Execution README.
Storage
The storage component persists agreed upon blocks of transactions and their execution results to the Aptos blockchain. A block of transactions (which includes transaction TN) will be saved via storage when there is agreement between more than a quorum (2f+1) of the validators participating in consensus. Agreement must include all the following:
- The transactions to include in the block
- The order of the transactions
- The execution results of the transactions in the block
Refer to Merkle accumulator for information on how a transaction is appended to the data structure representing the Aptos blockchain.
1. VM → Storage
When mempool invokes VMValidator::validate_transaction() to validate a transaction, VMValidator::validate_transaction() loads the sender’s account from storage and performs read-only validity checks on the transaction.
2. Execution → Storage
When the consensus component calls BlockExecutorTrait::execute_block(), execution reads the current state from storage combined with the in-memory “scratchpad” data to determine the execution results.
3. Execution → Storage
Once consensus is reached on a block of transactions, execution calls storage via DbWriter::save_transactions() to save the block of transactions and permanently record them. This will also store the signatures from the validator nodes that agreed on this block of transactions. The in-memory data in “scratchpad” for this block is passed to update storage and persist the transactions. When the storage is updated, every account that was modified by these transactions will have its sequence number incremented by one.
Note: The sequence number of an account on the Aptos blockchain increments by one for each committed transaction originating from that account.
4. REST Service → Storage
For client queries that read information from the blockchain, the REST service directly interacts with storage to read the requested information.
For implementation details refer to the Storage README.