Merge main into testnet7 (#3354)

* 2 harvesting features (#3331) * 3 harvesting features: - Debug level shows the time for every quality lookup - Warning level if takes longer than 5 seconds - Allow configuration of plot loading interval (default 2 minutes) * Comment out super logging * Improve wallet consistency (#3305) * Improve wallet consistency * Improve CLI significantly, and fix self-tx balances * Fix await * Fix deadlock and test * Remove spam.sh * Changelog for 1.1.3 (#3345) * Changelog for 1.1.3 * minor updates * updates part 3 * Those engineers who don't update changelogs... :) Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com> Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Chia-Network · May 1, 2021 · 40faaa4 · 40faaa4
1 parent 12a8ac9
commit 40faaa4
Show file tree

Hide file tree

Showing 12 changed files with 365 additions and 273 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,15 +6,49 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project does not yet adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 for setuptools_scm/PEP 440 reasons.
 
+## 1.1.3 Chia Blockchain 2021-05-01
 
-## UNRELEASED
+### Added
+
+- Significant speed improvements have been added to the Node during P2P operations. This is mostly a performance improvement to make sure there is plenty of overhead in Node for transaction processing. Smaller machines like the Pi 4 are strongly advised to update to this release before Monday 5/3/21 at 10AM PDT when transactions start.
+- Significant syncing speed improvements have been added. However, there is a lot of low hanging fruit to continue to increase overall chain sync speed in subsequent releases.
 
 ### Changed
 
+- We now require node 12.x to build the GUI. Installers have been building using node 12.x for quite some time.
+- Node will now farm while syncing.
+- We changed chialisp singletons to take a puzzlehash as its origin. We also updated the DID wallet to use this.
+- Transactions are now cached for 10 minutes in mempool to retry if there is a failure of a spending attempt.
+- Thank you to @fiveangle for making install.sh more resilient across python installations.
+- Standard transactions now all have announcements.
+- VDF verification during sync and normal operations are now cached to enhance node performance.
+- Target peer count has been increased from 60 to 80 to increase the number of peer Nodes available to connect to. Outbound connections were lowered from 10 to 8.
+- The streamables class has had significant performance improvements.
+- If a proof of space lookup takes longer than 5 seconds we will now log an error at WARNING level.
+- Allow configuration of the plot loading interval (default is 2 minutes) in config.yaml.
+- CLI wallet UI was improved and shows syncing status of wallet.
+- @martomi added logging of added coins back.
+- Thank you to @aisk for additional type checking.
+- @aisk added error checking in bech32m
+- Chialisp programs now remained serialized in Node for better performance.
+- Mempool is now set to be 50 times the single block size.
+- Mitigate 1-3 mojo dust attacks.
+- CLI now switches to EiB for netspace display as appropriate.
+
 ### Fixed
 
-- Fixed excess memory use when displaying plot logs in GUI
-- Fixed plot update issue in GUI
+- We fixed excess memory use when displaying plot logs in GUI. This was causing the GUI application to fail in many "fun" ways after plotting a lot of plots in parallel.
+- Fixed plot update issues in the GUI.
+- Long syncing will now correctly show "Syncing" around the application.
+- Nonce collisions in request IDs is fixed.
+- Some duplicate plots were not being caught and logged by harvester on loading.
+- We have removed many places where Node was making duplicate requests of other nodes.
+- Daemon now waits for services to end to attempt to avoid zombie processes.
+- Node is less likely to lose sync due to state inconsistency.
+- A large thank you to @sargonas for diagnosing why so many technical support questions were flooding GitHub issues, PRing a fix, and mass migrating issues to Discussions.
+- Thanks to @jeolcho for fixing a bug in full_node_rpc_api.py.
+- Thanks @antoniobg for a typo fix in keychain.py.
+- Thanks to @altendky for catching a Copyright date error.
 
 ## 1.1.2 Chia Blockchain 2021-04-24
 

diff --git a/chia/cmds/wallet_funcs.py b/chia/cmds/wallet_funcs.py
@@ -104,46 +104,41 @@ async def print_balances(args: dict, wallet_client: WalletRpcClient, fingerprint
     address_prefix = config["network_overrides"]["config"][config["selected_network"]]["address_prefix"]
 
     print(f"Wallet height: {await wallet_client.get_height_info()}")
+    print(f"Sync status: {'Synced' if (await wallet_client.get_synced()) else 'Not synced'}")
     print(f"Balances, fingerprint: {fingerprint}")
     for summary in summaries_response:
         wallet_id = summary["id"]
         balances = await wallet_client.get_wallet_balance(wallet_id)
         typ = WalletType(int(summary["type"])).name
         if typ != "STANDARD_WALLET":
             print(f"Wallet ID {wallet_id} type {typ} {summary['name']}")
-            print(f"   -Confirmed: " f"{balances['confirmed_wallet_balance']/units['colouredcoin']}")
-            print(f"   -Confirmed - Pending Outgoing: {balances['unconfirmed_wallet_balance']/units['colouredcoin']}")
-            print(f"   -Spendable: {balances['spendable_balance']/units['colouredcoin']}")
-            print(f"   -Pending change: {balances['pending_change']/units['colouredcoin']}")
+            print(f"   -Total Balance: " f"{balances['confirmed_wallet_balance']/units['colouredcoin']}")
+            print(f"   -Pending Total Balance: {balances['unconfirmed_wallet_balance']/units['colouredcoin']}")
+            print(f"   -Spendable Balance: {balances['spendable_balance']/units['colouredcoin']}")
         else:
             print(f"Wallet ID {wallet_id} type {typ}")
             print(
-                f"   -Confirmed: {balances['confirmed_wallet_balance']} mojo "
-                f"({balances['confirmed_wallet_balance']/units['chia']} {address_prefix})"
+                f"   -Total Balance: {balances['confirmed_wallet_balance']/units['chia']} {address_prefix} "
+                f"({balances['confirmed_wallet_balance']} mojo)"
             )
             print(
-                f"   -Unconfirmed: {balances['unconfirmed_wallet_balance']} mojo "
-                f"({balances['unconfirmed_wallet_balance']/units['chia']} {address_prefix})"
+                f"   -Pending Total Balance: {balances['unconfirmed_wallet_balance']/units['chia']} {address_prefix} "
+                f"({balances['unconfirmed_wallet_balance']} mojo)"
             )
             print(
-                f"   -Spendable: {balances['spendable_balance']} mojo "
-                f"({balances['spendable_balance']/units['chia']} {address_prefix})"
-            )
-            print(
-                f"   -Pending change: {balances['pending_change']} mojo "
-                f"({balances['pending_change']/units['chia']} {address_prefix})"
+                f"   -Spendable: {balances['spendable_balance']/units['chia']} {address_prefix} "
+                f"({balances['spendable_balance']} mojo)"
             )
 
 
 async def get_wallet(wallet_client: WalletRpcClient, fingerprint: int = None) -> Optional[Tuple[WalletRpcClient, int]]:
-    fingerprints = await wallet_client.get_public_keys()
+    if fingerprint is not None:
+        fingerprints = [fingerprint]
+    else:
+        fingerprints = await wallet_client.get_public_keys()
     if len(fingerprints) == 0:
         print("No keys loaded. Run 'chia keys generate' or import a key")
         return None
-    if fingerprint is not None:
-        if fingerprint not in fingerprints:
-            print(f"Fingerprint {fingerprint} does not exist")
-            return None
     if len(fingerprints) == 1:
         fingerprint = fingerprints[0]
     if fingerprint is not None:

diff --git a/chia/harvester/harvester.py b/chia/harvester/harvester.py
@@ -53,6 +53,7 @@ def __init__(self, root_path: Path, config: Dict, constants: ConsensusConstants)
         self.log = log
         self.state_changed_callback: Optional[Callable] = None
         self.last_load_time: float = 0
+        self.plot_load_frequency = config.get("plot_loading_frequency_seconds", 120)
 
     async def _start(self):
         self._refresh_lock = asyncio.Lock()

diff --git a/chia/harvester/harvester_api.py b/chia/harvester/harvester_api.py
@@ -70,7 +70,7 @@ async def new_signage_point_harvester(
         assert len(new_challenge.challenge_hash) == 32
 
         # Refresh plots to see if there are any new ones
-        if start - self.harvester.last_load_time > 120:
+        if start - self.harvester.last_load_time > self.harvester.plot_load_frequency:
             await self.harvester.refresh_plots()
             self.harvester.last_load_time = time.time()
 
@@ -151,11 +151,13 @@ def blocking_lookup(filename: Path, plot_info: PlotInfo) -> List[Tuple[bytes32,
                 self.harvester.log.error(f"Unknown error: {e}")
                 return []
 
-        async def lookup_challenge(filename: Path, plot_info: PlotInfo) -> List[harvester_protocol.NewProofOfSpace]:
+        async def lookup_challenge(
+            filename: Path, plot_info: PlotInfo
+        ) -> Tuple[Path, List[harvester_protocol.NewProofOfSpace]]:
             # Executes a DiskProverLookup in a thread pool, and returns responses
             all_responses: List[harvester_protocol.NewProofOfSpace] = []
             if self.harvester._is_shutdown:
-                return []
+                return filename, []
             proofs_of_space_and_q: List[Tuple[bytes32, ProofOfSpace]] = await loop.run_in_executor(
                 self.harvester.executor, blocking_lookup, filename, plot_info
             )
@@ -169,7 +171,7 @@ async def lookup_challenge(filename: Path, plot_info: PlotInfo) -> List[harveste
                         new_challenge.signage_point_index,
                     )
                 )
-            return all_responses
+            return filename, all_responses
 
         awaitables = []
         passed = 0
@@ -193,8 +195,19 @@ async def lookup_challenge(filename: Path, plot_info: PlotInfo) -> List[harveste
 
         # Concurrently executes all lookups on disk, to take advantage of multiple disk parallelism
         total_proofs_found = 0
-        for sublist_awaitable in asyncio.as_completed(awaitables):
-            for response in await sublist_awaitable:
+        for filename_sublist_awaitable in asyncio.as_completed(awaitables):
+            filename, sublist = await filename_sublist_awaitable
+            time_taken = time.time() - start
+            if time_taken > 5:
+                self.harvester.log.warning(
+                    f"Looking up qualities on {filename} took: {time.time() - start}. This should be below 5 seconds "
+                    f"to minimize risk of losing rewards."
+                )
+            else:
+                pass
+                # If you want additional logs, uncomment the following line
+                # self.harvester.log.debug(f"Looking up qualities on {filename} took: {time.time() - start}")
+            for response in sublist:
                 total_proofs_found += 1
                 msg = make_msg(ProtocolMessageTypes.new_proof_of_space, response)
                 await peer.send_message(msg)