From f565689d75ed8f13538607bfa71047cefe27f76f Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Mon, 18 Sep 2023 17:37:45 -0600 Subject: [PATCH 1/3] TheGraph node environment variables recommended values for HashIO Relay Signed-off-by: Alfredo Gutierrez --- tools/subgraph-example/README.md | 15 +++++++++++++++ tools/subgraph-example/docker-compose.yml | 8 ++++++++ 2 files changed, 23 insertions(+) diff --git a/tools/subgraph-example/README.md b/tools/subgraph-example/README.md index f76806c041..a0757c06b7 100644 --- a/tools/subgraph-example/README.md +++ b/tools/subgraph-example/README.md @@ -144,3 +144,18 @@ The easiest way to run a local `graph-node` against `testnet`, `previewnet` or ` 3. In the `subgraph.yaml` file change the dataSources network with to the network you want to index. Also don't forget to update the address (and the startBlock). Advanced info on how to set up an indexer could be found in [The Graph Docs](https://thegraph.com/docs/en/indexing/operating-graph-node/) and the [official graph-node GitHub repository](https://github.com/graphprotocol/graph-node) + + +## TheGraph Node recommended configuration + +| Environment Variable | Description | Default | Recommended Value | +|------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-------------------| +| ETHEREUM_REORG_THRESHOLD | Maximum expected reorg size, if a larger reorg happens, subgraphs might process inconsistent data | 250 | 0 | +| ETHEREUM_POLLING_INTERVAL | how often to poll Ethereum for new blocks (in ms) | 500ms | 2500ms | +| GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE | The ideal amount of triggers to be processed in a batch. If this is too small it may cause too many requests to the ethereum node, if it is too large it may cause unreasonably expensive calls to the ethereum node and excessive memory usage. | 100 | 10 | +| ETHEREUM_BLOCK_BATCH_SIZE | number of Ethereum blocks to request in parallel. Also limits other parallel requests such such as trace_filter. Defaults to 10. | 10 | 1 | +| GRAPH_ETHEREUM_MAX_BLOCK_RANGE_SIZE | Maximum number of blocks to scan for triggers in each request (Needs to be set explicitly, default varies and does not respect the 1000 value) | 1000 | 1000 | +| GRAPH_ETHEREUM_MAX_EVENT_ONLY_RANGE | Maximum range size for eth.getLogs requests that dont filter on contract address, only event signature (defaults to 500). | 500 | 50 | +| GRAPH_ETHEREUM_BLOCK_INGESTOR_MAX_CONCURRENT_JSON_RPC_CALLS_FOR_TXN_RECEIPTS | The maximum number of concurrent requests made against Ethereum for requesting transaction receipts during block ingestion. Defaults to 1,000. | 1000 | 100 | +| GRAPH_ETHEREUM_GENESIS_BLOCK_NUMBER | Specify genesis block number. If the flag is not set, the default value will be 0. | 0 | 0 | + diff --git a/tools/subgraph-example/docker-compose.yml b/tools/subgraph-example/docker-compose.yml index 65d8e267b8..a6a2a720bc 100644 --- a/tools/subgraph-example/docker-compose.yml +++ b/tools/subgraph-example/docker-compose.yml @@ -21,7 +21,15 @@ services: ipfs: "ipfs:5001" ethereum: "local:http://host.docker.internal:7546" GRAPH_LOG: debug + ETHEREUM_REORG_THRESHOLD: 250 + ETHEREUM_POLLING_INTERVAL: 1000 + GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE: 10 + ETHEREUM_BLOCK_BATCH_SIZE: 1 + GRAPH_ETHEREUM_MAX_BLOCK_RANGE_SIZE: 1000 + GRAPH_ETHEREUM_MAX_EVENT_ONLY_RANGE: 50 + GRAPH_ETHEREUM_BLOCK_INGESTOR_MAX_CONCURRENT_JSON_RPC_CALLS_FOR_TXN_RECEIPTS: 100 GRAPH_ETHEREUM_GENESIS_BLOCK_NUMBER: 1 + restart: unless-stopped ipfs: image: ipfs/go-ipfs:v0.10.0 ports: From c63d7ecb97264749f6fc7712869c3a6dcac81eac Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Mon, 18 Sep 2023 17:44:46 -0600 Subject: [PATCH 2/3] ETHEREUM_REORG_THRESHOLD to 0 Signed-off-by: Alfredo Gutierrez --- tools/subgraph-example/docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/subgraph-example/docker-compose.yml b/tools/subgraph-example/docker-compose.yml index a6a2a720bc..93eadf7444 100644 --- a/tools/subgraph-example/docker-compose.yml +++ b/tools/subgraph-example/docker-compose.yml @@ -21,7 +21,7 @@ services: ipfs: "ipfs:5001" ethereum: "local:http://host.docker.internal:7546" GRAPH_LOG: debug - ETHEREUM_REORG_THRESHOLD: 250 + ETHEREUM_REORG_THRESHOLD: 0 ETHEREUM_POLLING_INTERVAL: 1000 GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE: 10 ETHEREUM_BLOCK_BATCH_SIZE: 1 From 8dc6eed4a56a595f1924b731982c2e599d975e81 Mon Sep 17 00:00:00 2001 From: Alfredo Gutierrez Date: Mon, 18 Sep 2023 17:55:27 -0600 Subject: [PATCH 3/3] update doc so ETHEREUM_POLLING_INTERVAL is without ms and to 1000 that way it works better Signed-off-by: Alfredo Gutierrez --- tools/subgraph-example/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/subgraph-example/README.md b/tools/subgraph-example/README.md index a0757c06b7..761c1780f2 100644 --- a/tools/subgraph-example/README.md +++ b/tools/subgraph-example/README.md @@ -151,7 +151,7 @@ Advanced info on how to set up an indexer could be found in [The Graph Docs](htt | Environment Variable | Description | Default | Recommended Value | |------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-------------------| | ETHEREUM_REORG_THRESHOLD | Maximum expected reorg size, if a larger reorg happens, subgraphs might process inconsistent data | 250 | 0 | -| ETHEREUM_POLLING_INTERVAL | how often to poll Ethereum for new blocks (in ms) | 500ms | 2500ms | +| ETHEREUM_POLLING_INTERVAL | how often to poll Ethereum for new blocks (in ms) | 1000 | 2500ms | | GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE | The ideal amount of triggers to be processed in a batch. If this is too small it may cause too many requests to the ethereum node, if it is too large it may cause unreasonably expensive calls to the ethereum node and excessive memory usage. | 100 | 10 | | ETHEREUM_BLOCK_BATCH_SIZE | number of Ethereum blocks to request in parallel. Also limits other parallel requests such such as trace_filter. Defaults to 10. | 10 | 1 | | GRAPH_ETHEREUM_MAX_BLOCK_RANGE_SIZE | Maximum number of blocks to scan for triggers in each request (Needs to be set explicitly, default varies and does not respect the 1000 value) | 1000 | 1000 |