invalid ristretto encoding at #10516963

[EclesioMeloJunior]

Describe the bug

• Gossamer cannot decode a next epoch data consensus digest
• The problematic consensus digest exists in the block #10516963 and is related to epoch 17866

2023-12-01T03:35:11Z DEBUG    ⏩ next epoch data stored in memory: epoch 17866, hash: 0xa9e190ef90d38f4001294fe58cabdde23bbb6b8374fc25e8b80cff78e0564cb1	pkg=state
2023-12-01T03:35:11Z DEBUG    stored BABENextEpochData data: {{[{[162 245 89 73 149 115 181 135 130 135 248 204 24 171 251 93 7 2 41 166 88 74 18 81 19 114 90 56 186 98 153 111] 1} {[82 76 43 198 18 197 166 242 40 69 230 168 9 68 219 158 25 123 234 165 12 36 47 52 162 164 90 54 102 162 235 122] 1} {[44 193 155 216 162 60 83 79 159 42 233 22 81 200 75 194 253 239 117 126 138 202 21 110 220 94 146 238 181 137 4 61] 1} {[183 142 200 145 89 248 156 108 112 1 161 147 44 146 102 38 5 215 104 71 27 214 104 138 80 214 228 164 121 49 92 88] 1} {[54 72 76 90 227 79 47 205 58 86 169 172 118 129 161 131 236 22 40 228 149 221 20 45 83 13 16 228 232 193 67 36] 1} {[240 138 26 158 184 83 106 115 91 35 176 77 61 101 1 188 114 13 13 106 64 168 156 145 26 119 31 28 45 122 66 96] 1} {[238 177 108 159 119 34 81 111 202 107 60 27 192 250 178 112 165 163 40 15 81 206 27 33 232 52 216 167 243 212 82 33] 1} {[178 76 237 5 209 225 251 173 249 135 145 228 82 249 210 101 163 17 142 152 40 169 206 15 100 237 115 217 23 100 234 23] 1} {[136 195 59 242 67 82 142 104 78 155 65 201 154 188 32 131 224 214 205 57 27 203 45 131 64 114 59 18 180 112 101 20] 1} {[86 100 96 69 238 174 229 172 25 75 210 67 100 32 201 7 13 129 186 13 23 98 72 50 193 235 18 195 179 83 198 72] 1} {[86 91 234 140 249 219 18 60 113 165 1 83 128 169 133 96 131 155 18 144 48 40 126 32 8 220 131 55 175 252 75 121] 1} {[172 92 92 53 86 25 151 12 156 32 73 172 224 85 19 56 230 243 69 173 1 84 115 56 169 15 49 124 192 67 168 32] 1} {[188 83 225 175 249 96 110 27 176 214 70 60 253 180 169 127 24 11 194 69 181 194 69 228 131 146 209 146 60 232 228 100] 1} {[100 43 96 189 170 168 1 5 117 102 69 233 209 226 103 172 124 56 74 115 237 0 79 101 233 17 128 249 155 73 239 6] 1} {[52 233 190 38 239 68 227 104 212 212 56 226 120 43 91 8 5 252 64 37 124 116 54 2 106 244 17 58 128 36 94 83] 1} {[196 138 238 82 150 226 145 28 75 58 144 81 254 10 76 74 232 23 65 225 16 182 174 62 137 177 56 205 135 77 144 41] 1}] [204 121 26 144 5 202 94 24 106 163 251 142 112 134 155 23 132 60 213 175 12 252 122 227 74 173 191 60 30 209 151 183]} map[1:{[] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]} 2:{0} 3:{<nil> map[1:{0 0 0}]}]} for hash: 0xa9e190ef90d38f4001294fe58cabdde23bbb6b8374fc25e8b80cff78e0564cb1 to epoch: 17866	pkg=state
2023-12-01T03:35:12Z INFO     handling justification for block 0xdb021409...d04daeee (#10516992)	pkg=sync
2023-12-01T03:35:12Z DEBUG    notifying finalised block channels...	pkg=state
2023-12-01T03:35:12Z INFO     🔨 finalised block number 10516992 with hash 0xdb021409b7f68900ef6d0e36aa567852553b22428395afda64948d4ed04daeee	pkg=sync
2023-12-01T03:35:12Z ERROR    failed to persist babe next epoch data: cannot transform epoch data: creating sr25519 public key: invalid Ristretto encoding	pkg=digest

2023-12-01T12:19:57Z ERROR    block data processing for block with hash 0x8bd00f6feed1cf9335293643cc0bb87aee7ea0b6963d10d47085a0bf8771cade failed: processing block data with header and body: babe verifying block: failed to get verifier info for block 10517518: failed to get epoch data for epoch 17866: cannot transform into epoch data: creating sr25519 public key: invalid Ristretto encoding    pkg=sync
2023-12-01T12:19:57Z ERROR    requesting max blocks from best block header: while handling workers results: while handling ready block: processing block data with header and body: babe verifying block: failed to get verifier info for block 10517518: failed to get epoch data for epoch 17866: cannot transform into epoch data: creating sr25519 public key: invalid Ristretto encoding       pkg=sync
2023-12-01T12:19:57Z INFO     🚣 currently syncing, 60 peers connected, 75 available workers, target block number 18452419, finalised block number 10517504 with hash 0xf182b6ef5d07dca792d068af81d20a6cd7ff822aa98ed8797729f8248f1d6143   pkg=sync

[EclesioMeloJunior]

Got an answer from Tomaka at host implementors matrix chat:

For what it's worth, in that situation smoldot would just accept the public key, as it doesn't verify whether it's a valid sr25519 key. Any block authored by this validator would be invalid as it would fail the signature check. I'm not sure whether this is the "correct" behaviour, though.

[EclesioMeloJunior]

Response from Andre at host implementors guide

substrate would have the same behavior. the public key is passed around from runtime to client as raw bytes, and the publickey object is only constructed when trying to verify a signature. the signature verification would just fail at that point (https://github.com/paritytech/polkadot-sdk/blob/e6b9da132dcb0570870a6f319fbe5aa7e5030b76/substrate/primitives/core/src/sr25519.rs#L507-L511)

[EclesioMeloJunior]

So, to fix this problem we should still store the authority key coming from the runtime and not parsing it to a public key until it is needed to verify signatures, in such case the verification will fail and the block rejected.