# On Decoding Raw EVM Calldata

Source: https://jbecker.dev/research/decoding-raw-calldata
Kiwi News: https://news.kiwistand.com/stories/On-Decoding-Raw-EVM-Calldata?index=0x65855837d9d39639de20bb4f8a27ff7bc5c6e668fe4857ddc41adfd75db2ef3d75bf6118
Submitted by destiner.eth on 2023-12-22 UTC, 3 upvotes, 0 comments

## Summary

Heimdall-rs 0.7.0 can decode raw EVM calldata into its constituent types without a contract ABI or function signature, though the method may still be ambiguous when identifying types. It splits calldata into 32-byte words and uses ABI encoding rules to identify dynamic types, requiring offset pointers to be positive multiples of 32 that point to valid words. For static words, it applies padding heuristics: right-padded words are likely bytesN or strings, left-padded words with 00 padding are likely uintN, left-padded words with ff padding are likely intN, and unpadded words are likely bytes32.

## Article

With the release of heimdall-rs 0.7.0, the toolkit gained the ability to decode raw EVM calldata into its constituent types, without the need for a contract ABI or signature resolution. In this technical article, we'll dive deep into the inner workings of EVM calldata, how it's encoded, and how we can decode it arbitrarily.

Note: knowledge of the EVM and Solidity/Vyper is assumed, but not required.

Disclaimer: The method presented in this paper is not perfect and may still have some ambiguity when it comes to decoding types. However, without the ABI or signature, this is the best we can do.

Brief: EVM Calldata

When interacting with an EVM contract, a caller must provide a set of arguments in order to invoke a function call. These arguments are encoded into a byte array known as the calldata, which is passed to the contract as part of the transaction. The contract can then access the calldata, and decode1 it into its constituent types.

The EVM has three opcodes for accessing calldata: CALLDATASIZE, CALLDATALOAD, and CALLDATACOPY. These opcodes allow a contract to access the calldata, but do not provide any information about the calldata's structure, or the types of the arguments it contains.

As an exercise, let's build the calldata for a simple function call. Consider the following Solidity function:

Function Signatures & Selectors

The first four2 bytes of the calldata are typically used to identify the function being called. These bytes are known as the function selector, and are generated by taking the first four bytes of the keccak256 hash of the function's signature.

Our balanceOf function has the signature3:

The keccak256 hash of this signature is:

So the function selector is the first four bytes of this hash: 0x70a08231, and we can begin building the calldata:

Encoding Arguments

Arguments in calldata are encoded according to the ABI specification. The encoding of an argument is dependent on its type, and the encoding of a function's arguments is simply the concatenation of the encodings of each argument (with a few exceptions we'll touch on later).

Encoding Static Types

For elementary types, the encoding is straightforward:

TypeEncoding

bool00...00 for false, 00...01 for true

uint<N>Hex-encoded big-endian representation of the integer

addressEncoded as a uint160

bytes<N>Hex-encoded bytes, left-padded

int<N>Hex-encoded big-endian representation, padded with ff bytes if negative and 00 bytes if positive

enumEncoded as a uint8

These types are encoded as a single word (32 bytes) in calldata, and are padded to the left with zeroes if necessary.

So, for our balanceOf function, the argument is an address, which is encoded as a uint160. The address we want to query is 0xd8da6bf26964af9d7eed9e03e53415d37aa96045 (vitalik.eth), so the encoding is:

And we can add this to our calldata:

The calldata is now complete, and we can pass it to the contract using Foundry's cast:

And we get the expected result:

Encoding Dynamic Types

Encoding dynamic types is a bit more complicated, since their encoding is dependent on the length of the data being encoded. The encoding of a dynamic type is as follows:

The first word is the offset of the data in bytes from the start of context. We'll call this offset.

In most cases, context is the start of the calldata argument block (that is, the first word of the calldata after the function selector). However, if the dynamic type is nested within another dynamic type, context is the start of the outer dynamic type's data block.

The offset is encoded as a uint256, and is padded to the left with zeroes if necessary.

The word at context[offset] is the length of the data in bytes. We'll call this length. The length is encoded as a uint256, and is padded to the left with zeroes if necessary.

For bytes and string, length is the number of bytes the encoded data takes up. The encodings themselves are right-padded with zeroes if necessary, and may span multiple words if the length is greater than 32 bytes.

For dynamic-length arrays (i.e, T[]), length is the number of elements in the array. The encoding is then the concatenation of the encodings of each element, in order.

Note: The above is a simplification of the ABI specification. For the purposes of this article, I believe this is sufficient.

Again, we'll use an example to illustrate. Let's encode the following signature f(uint256,uint32[],bytes10,bytes) (selector is 0x8be65246) with arguments (0x123, [0x456, 0x789], "1234567890", "Hello, world!").

The first parameter is a uint256, which is a simple elementary type. This encoding is straightforward:

The second parameter is a dynamic-length array of uint32.

The third parameter is a bytes10, which is a static type with a length of 10 bytes. The encoding is:

The fourth parameter is a bytes, which is a dynamic type. The encoding is:

Now we can concatenate these encodings to get the final calldata:

Now we can fill in the placeholders (a) and (b) with the offsets of the second and fourth parameters, respectively. Since the fourth word (zero-indexed) of the calldata is the start of the second parameter, the offset of the second parameter is 4∗32=1284 * 32 = \boxed{128}. Likewise, the offset of the fourth parameter is 7∗32=2247 * 32 = \boxed{224}.

And we're done! The final calldata is:

You may notice that for any function f(arg1,...argn)f({arg}_1, ... {arg}_n), the first nn words of the calldata are either the arguments themselves, or the offsets of the arguments. This will be important later.

Decoding Calldata

Before we begin decoding raw calldata, let's set a few assumptions:

The calldata is well-formed. That is, it is a valid sequence of bytes that can be decoded according to the ABI specification. Additionally, len(calldata) - 4≡320\text{len(calldata) - 4} \equiv_{32} 0.

We do not have access to the contract ABI, nor do we have access to the function signature. All we have is the raw calldata.

Offset pointers to dynamic types are valid. That is, offset>0,offset≡320\text{offset} \gt 0, \text{offset} \equiv_{32} 0. This is reasonable, since an offset pointer must point to the start of a word.

Helper Functions

We'll define a few helper functions that we'll use throughout the decoding process. For brevity, we'll use pseudocode to describe these functions, but for those interested, the full implementation can be found here.

The Decoding Process

Now that we have our helper functions, we can begin decoding raw calldata.

The first step of the decoding process is to convert the calldata into a list of words (split into 32-byte chunks, removing the function selector). We'll call this list calldata_words.

Now we create a HashSet<usize> called covered_words. We'll use this to keep track of which words we've already covered, so we don't accidentally decode the same word twice, miss decoding a word, or incorrectly decode a word. We'll also create a mutable Vec<ParamType> called potential_inputs, which we'll use to keep track of the types of the function's inputs.

Next, we'll use a while loop to iterate over each word in calldata_words. The loop will terminate when all words in calldata_words have been covered and decoded. Here's what each iteration of the loop looks like:

We'll attempt to decode the current word as an ABI-encoded dynamic type. If this succeeds, we'll add the type to potential_inputs and add the indices of all words covered by this type to covered_words. We'll discuss how this works in detail later.

If this is not a dynamic type, we'll attempt to decode this word as a static type. We'll explain how this works in detail later.

NOTE: If f(arg1,...argn)f({arg}_1, ... {arg}_n) is a function with nn arguments, then the first nn words of the calldata are either the arguments themselves, or the offsets of the arguments. When we have decoded the first nn words of calldata, covered_words will contain the indices of all words in the calldata that are part of the function's arguments. We can then use this to determine the function's signature, and the types of its arguments.

Decoding Static Types

Let's start with the simple case: static types. Given a word of calldata that is not an ABI-encoded dynamic type, we can determine the potential types of the word by using the get_potential_types_for_word function we defined earlier.

This function will return a list of potential types, as well as the maximum size of the word in bytes. We'll use this to determine the most likely type of the word by performing a few simple heuristics:

bytesN and strings are always right-padded, so if this word is right-padded, it's probably a bytesN or string.

uintN and intN are always left-padded, so if this word is left-padded, it's probably a uintN or intN.

We can also check if the padding is 00 or ff, indicating whether the number is positive or negative. If the padding is 00, it's probably a uintN. If the padding is ff, it's probably an intN.

If this word is not padded, it's probably a bytes32.

Decoding Dynamic Types

Now we'll move on to the more complicated case: ABI-encoded dynamic types. Let's take a look at how try_decode_dynamic_parameter works behind the scenes:

We initialize a HashSet<usize> called word_coverages (not to be confused with covered_words) with parameter_index. We'll use this HashSet to keep track of which words we've covered while attempting to ABI-decode the current word. If we successfully decode a dynamic type, this HashSet will contain all indices of words in calldata_words that are part of this ABI-encoded type, and will be joined with covered_words at the end of the decoding process.

Now, we'll perform the first validation step by checking if the current word could be a valid pointer to an ABI-encoded dynamic type in the calldata. We do this by leveraging assumption (3), which states that all pointers to dynamic types are:

Greater than zero

Divisible by 32, since this should point to the start of a word (the size of the dynamic type).

Next, we'll check if this pointer is actually valid and points to a valid word in calldata_words. If it is not, we return None. If it is, we'll parse the word into a U256 and call this size. We'll also add word_offset to word_coverages, since this word (size) is part of the ABI-encoded type and should not be decoded again.

We can now define data_start_word_offset as word_offset + 1, since the word at word_offset is the size of the ABI-encoded type, and the data starts at the next word. This is the start of the data block for this dynamic type, and we'll use this later. data_end_word_offset is data_start_word_offset + size, since the data block is size words long. In the case of bytes and string, we'll need to recalculate this later.

Since size varies depending on the type of the item, we'll perform a simple check to see if there are enough words left in the calldata to contain the ABI-encoded item. If there aren't, it doesn't necessarily mean that this is not a valid ABI-encoded type, but it does indicate that this type cannot be an array, since there aren't enough words left to store the array elements. If there are enough words left, we'll call try_decode_dynamic_parameter_array, which we'll cover later. If there aren't, we'll call try_decode_dynamic_parameter_bytes.

Decoding bytes

Let's take a look at how try_decode_dynamic_parameter_bytes works behind the scenes:

First, we'll join all words from data_start_word_offset (the index in calldata_words where the encoded data starts) to the end of calldata_words. This is where the encoded data may be stored. We'll call this data_words. This may contain extra words that are not part of the ABI-encoded type, but that's okay, since we'll be checking the size of the encoded data later.

Next, we'll perform a quick validation check to see if there are enough remaining bytes in data_words to contain the ABI-encoded item. If there aren't, we return an Error::BoundsError.

Now, we'll calculate how many words are needed to store the encoded data with size size. We'll call this word_count_for_size. We'll also calculate the end of data_words by adding word_count_for_size to data_start_word_offset. We'll call this data_end_word_offset.

Now, we can perform a check on the last word in data_words. There should be size % 32 bytes in this word, and the rest should be null bytes. If the padding size of this last word is greater than 32 - last_word_size, there are too many bytes in the last word, and this is not a valid ABI-encoded type. We return an Error::BoundsError.

We extend word_coverages with the indices of all words from data_start_word_offset to data_end_word_offset, since we've now covered all words in the ABI-encoded type. We then return an AbiEncoded struct containing the type (bytes) and the coverages to be joined with covered_words at the end of the decoding process.

Decoding string

Decoding string is very similar to decoding bytes, with a few minor differences. Let's take a look at how try_decode_dynamic_parameter_string works behind the scenes:

You may notice that this is almost identical to try_decode_dynamic_parameter_bytes, with a few minor differences:

We first perform a padding conformity check on data_words. If all words in data_words have the same padding, we can assume that this is not a string and is instead an array. This is because strings will typically be of the form:

Where the first word is the length of the string, and the rest of the words are the string itself. If the words have conforming padding, we can assume that this is not a string and is instead an array.

The remaining steps for string validation are identical to try_decode_dynamic_parameter_bytes, since strings are essentially just bytes!

Decoding T[]

Finally, we'll cover decoding dynamic-length arrays. Let's take a look at how try_decode_dynamic_parameter_array works behind the scenes:

First, we'll join all words from data_start_word_offset to data_end_word_offset. This is where the encoded data may be stored. We'll call this data_words. This will have a length of size, since size is the number of elements in the array.

Next, we'll check if this is a string type, since some string encodings may appear to be arrays. If it is, we'll stop here and return the AbiEncoded struct containing the type string and the coverages to be joined with covered_words at the end of the decoding process.

If this is not a string type, we can assume that it is an array. We can extend word_coverages with the indices of all words from data_start_word_offset to data_end_word_offset, since we've now covered all words in the ABI-encoded type.

Now, we'll get the potential type of the array elements. Under the hood, this function:

Iterates over each word in data_words

Checks if the word is a dynamic type by recursively calling try_decode_dynamic_parameter

If it is a dynamic type, we know the type of the array elements and can return it

If it is a static type, find the potential types that can represent each element in the array

We'll call this potential type potential_type. We'll then return an AbiEncoded struct containing the type {potential_type}[] and the coverages to be joined with covered_words at the end of the decoding process.

A Quick Example

Now that we've covered the decoding process, let's walk through a quick example. Let's say we have the following calldata (the one we built earlier):

We'll start by converting this into a list of words:

We'll also keep track of covered_words and potential_inputs:

Now, we'll iterate over each word in calldata_words:

This word is not an ABI-encoded dynamic type, because U256::from(word) % 32 != 0, so this is not a pointer to a word in calldata_words. We'll attempt to decode this word as a static type:

So this word is probably a uint16 or int16. We'll add uint16 to potential_inputs and add i to covered_words:

Now, we'll increment i and continue:

This word does pass the first validation step, since U256::from(word) % 32 == 0. We'll attempt to decode this word as an ABI-encoded dynamic type:

So this is probably an array of uint16s. We'll add uint16[] to potential_inputs and add word_coverages to covered_words:

Now, we'll increment i and continue:

This word is not an ABI-encoded dynamic type, because U256::from(word) % 32 != 0, so this is not a pointer to a word in calldata_words. We'll attempt to decode this word as a static type:

So this word is probably a bytes10. We'll add bytes10 to potential_inputs and add i to covered_words:

Now, we'll increment i and continue:

This word does pass the first validation step, since U256::from(word) % 32 == 0. We'll attempt to decode this word as an ABI-encoded dynamic type:

So this a bytes. We'll add bytes to potential_inputs and add word_coverages to covered_words:

We've now covered all words in calldata_words, so we're done! Our final potential_inputs is:

We can now use this to determine the function's signature and decode the rest of the calldata.

Everything looks good! We've successfully decoded the raw calldata, including it's dynamic types!

Note: The sizes of parameters, namely the first uint16 and the uint16[] are smaller than the original uint256 and uint32[]. This is fine, and there's nothing we can do about it.

Conclusion

In this paper, we've covered how to decode raw calldata, including dynamic types -- all without the contracts ABI or signature resolution. This functionality is automated in heimdall-rs, and I can't wait to see what you create with it!

Note: It's probable that this method is not 100% accurate, and can be iterated on and improved. If you notice any edge cases or bugs that I'm missing, please let me know by opening an issue or PR.

Resources & Citations

The Solidity Authors, "Contract ABI Specification", Solidity Lang, Aug 2023. Available: https://docs.soliditylang.org/en/v0.8.23/abi-spec.html#argument-encoding

The word "decode" is used loosely here. The EVM does not provide any mechanism for decoding calldata, and CALLDATALOAD / CALLDATACOPY only provide access to the raw bytes. For example, if the calldata contains an address, the corresponding assembly to cast the calldata to an address is:

This is true in most cases. Some languages such as huff may use function selectors that are shorter than four bytes.

The signature is the function name, followed by a comma-separated list of argument types, enclosed in parentheses. The argument names are not preserved in the signature.