Overcoming ethclient’s filter restrictions

using the rpc client to build your own queries when you need null entries

As you are probably aware, the ERC20 transfer event is defined as such

event Transfer(address indexed from, address indexed to, uint256 value)

The use of “indexed” causes the source and destination addresses to be available to be used in filters by being included in the bloom filters.

You use them by defining “topics” in your filter query in addition to topic[0] which is the event signature.

The event signature for the Transfer event is

keccak256(“Transfer(address,address,uint256)")

which is

0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef

Using data taken from the USDC contract :

Valid filters could be

{
    "fromBlock" : "0x70000",
    "toBlock"   : "0x74000",
    "address"   : "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "topics"    : [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"    ]
}

This filters ALL transfer events in the range from 0x70000 to 0x74000

{
    "fromBlock" : "0x70000",
    "toBlock"   : "0x74000",
    "address"   : "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "topics"    : [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x00000000000000000000000000ee047a66d5cff27587a61559138c26b62f7ceb"
    ]
}

This filters transfers FROM the address

0x00EE047A66d5cff27587A61559138c26b62F7CEb

and this :

{
    "fromBlock" : "0x70000",
    "toBlock"   : "0x74000",
    "address"   : "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
    "topics"    : [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
null,
"0x00000000000000000000000000ee047a66d5cff27587a61559138c26b62f7ceb"
    ]
}

filters transfers TO the same address from ANY address

the go-ethereum codebase allows you to track blockchain events such as erc20 transfers very simply using the ethclient module as follows

transferString := "Transfer(address,address,uint256)"
transferTopic := common.BytesToHash(crypto.Keccak256([]byte(transferString)))
filter := ethereum.FilterQuery{}
filter.Addresses = make([]common.Address, 0)
filter.Addresses = append(filter.Addresses, erc20Address)
filter.Topics = [][]common.Hash{[]common.Hash{transferTopic}}
filter.FromBlock = big.NewInt(fromBlock)
filter.ToBlock = big.NewInt(toBlock)
ctx := context.Background()
logEntries, err := client.FilterLogs(ctx, filter)

At which point, assuming no error occurred, you receive an array of log entries that you can iterate over.

You can add source and destination addresses to the topics or just a source address but the ethclient implementation does not allow you to omit the source address

You can overcome this digging a bit deeper and using the rpc client instead.

ethclient.FilterLogs uses the rpc client’s CallContext function. We will use that but redefine the arguments that we supply to it.

You create an rpc client in a similar way, add the following to your import list

 github.com/ethereum/go-ethereum/rpc

create the client as usual

rpcClient, err = rpc.Dial(endPoint)

where endPoint is your usual connection string

The arguments get passed in as a map[string]interface{} where the topics are an array of hashes but an entry can also be null. The source and destination addresses must be converted to pointers to hashes or set to nil

func addrToHash(addr *common.Address) *common.Hash {
    if addr == nil {
        return nil
    }
    res := common.HexToHash(addr.Hex())
    return &res
}

using this we can create the filter arguments

args := make(map[string]interface{})
var topix []interface{}
args["address"] = address
args["fromBlock"] = hexutil.EncodeBig(fromBlock)
args["toBlock"] = hexutil.EncodeBig(toBlock)var topix []interface{}
topix = append(topix, event)
topix = append(topix, addrToHash(source))
topix = append(topix, addrToHash(dest))
args["topics"] = topix

If you set source to nil, this can be used to filter token transfers to a specific address

var replies []types.Log
err = client.CallContext(context.Background(), &replies, "eth_getLogs", args)

returning you an array of log data of transfers to the specified address.

Complete functions

note : using Steve Francia’s viper to get the endpoint

func addrToHash(addr *common.Address) *common.Hash {
  if addr == nil {
    return nil
  }
  res := common.HexToHash(addr.Hex())
  return &res
}
func getLogs(event common.Hash, address common.Address, fromBlock *big.Int, toBlock *big.Int, source *common.Address, dest *common.Address) (replies []types.Log, err error) {
  args := make(map[string]interface{})
  var topix []interface{}
  topix = append(topix, event)
  topix = append(topix, addrToHash(source))
  topix = append(topix, addrToHash(dest))
  args["topics"] = topix
  args["address"] = address
  args["fromBlock"] = hexutil.EncodeBig(fromBlock)
  args["toBlock"] = hexutil.EncodeBig(toBlock)
  endpoint := viper.GetString("ETH_CONNECT")
  client, err := GetRPClient(endpoint)
  if err != nil {
    return
  }
  defer client.Close()
  err = client.CallContext(context.Background(), &replies, "eth_getLogs", args)
  return
}