Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dot/parachain): implement candidate validation pre-check #4223

Merged
merged 53 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
50cc3ab
add pvf worker pool skeleton
edwardmack Jul 10, 2024
3427da0
add test to validation host
edwardmack Jul 19, 2024
6537104
create worker pool skeleton for PVF host
edwardmack Jul 23, 2024
7e314f3
add validation logic to workers
edwardmack Aug 6, 2024
0b2b5fc
address lint issues.
edwardmack Aug 6, 2024
20fd4c4
add functionality for validate from chainstate to pvf host and tests
edwardmack Aug 10, 2024
6c05d82
add updated mock files
edwardmack Aug 10, 2024
a6b2652
clean-up unused comments
edwardmack Aug 12, 2024
e978c38
Merge branch 'feat/parachain' into ed/feat/candidate_validation_pvf_h…
edwardmack Aug 16, 2024
0655dd3
address merge conflicts
edwardmack Aug 16, 2024
674fefa
Merge branch 'feat/parachain' into ed/feat/candidate_validation_pvf_h…
edwardmack Aug 21, 2024
01905af
resolve merge conflicts
edwardmack Aug 21, 2024
afde6fd
refactor ValidationHost to Host, and validationWorkerPool to workerPool
edwardmack Aug 21, 2024
0f25072
Merge branch 'feat/parachain' into ed/feat/candidate_validation_pvf_h…
edwardmack Aug 23, 2024
dbf3cf8
refactor go concurrancy
edwardmack Aug 28, 2024
c95c3e7
Merge branch 'feat/parachain' into ed/feat/candidate_validation_pvf_h…
edwardmack Aug 29, 2024
79ba043
revert run to match interface
edwardmack Aug 29, 2024
ce721e7
remove redundant structs for simplicty, removed go routines to simpli…
edwardmack Aug 29, 2024
48faacd
removed pvf package, simplified code removed concurrency
edwardmack Aug 29, 2024
391c6b4
added missing mock file
edwardmack Aug 29, 2024
7988888
address deep source comments
edwardmack Aug 29, 2024
5820e3f
address PR comments
edwardmack Sep 3, 2024
3e5ac28
make private functions/structs private
edwardmack Sep 4, 2024
3e6f025
implement function timeout
edwardmack Sep 5, 2024
020fb6f
refactor submitRequest to executeRequest
edwardmack Sep 6, 2024
cdc3514
add test for candidate-validation timeout
edwardmack Sep 6, 2024
9b0f2e6
Merge branch 'ed/feat/candidate_validation_pvf_host_base' into ed/fea…
edwardmack Sep 6, 2024
31b83f1
handle validation errors InvalidOutputs and BadParent
edwardmack Sep 10, 2024
c96f6e8
add addition validation timeout tests
edwardmack Sep 11, 2024
b8bbeb2
address PR comments
edwardmack Sep 11, 2024
25d95bd
address PR comments, rename variables and functions
edwardmack Sep 13, 2024
6677479
fix test
edwardmack Sep 13, 2024
9429632
remove printf from debugging
edwardmack Sep 13, 2024
f869cba
address PR comments
edwardmack Sep 17, 2024
d71e539
Merge branch 'feat/parachain' into ed/feat/candidate_validation_pvf_h…
edwardmack Sep 18, 2024
1f9d339
Merge branch 'ed/feat/candidate_validation_pvf_host_base' into ed/fea…
edwardmack Sep 18, 2024
f1b7164
fix merge conflicts
edwardmack Sep 18, 2024
a430a3e
Merge branch 'feat/parachain' into ed/feat/candidate_validation_confi…
edwardmack Sep 19, 2024
8e54519
handle get validationCodeByHash pre check
edwardmack Sep 21, 2024
5fa4faf
implement executor params check for pvf_precheck
edwardmack Sep 23, 2024
2971807
add maybeCompressedBlobDecompress function and test
edwardmack Sep 25, 2024
f579c25
Merge branch 'feat/parachain' into ed/feat/candidate_validation_precheck
edwardmack Oct 1, 2024
0522c16
add host precheck function
edwardmack Oct 1, 2024
f47b9f8
add timeout checks to precheck
edwardmack Oct 2, 2024
359ae93
Merge branch 'feat/parachain' into ed/feat/candidate_validation_precheck
edwardmack Oct 2, 2024
2b84a67
cleanup remove un-used code
edwardmack Oct 3, 2024
99ee0ca
add comments
edwardmack Oct 3, 2024
93936cc
regenerate mocks
edwardmack Oct 3, 2024
7f80875
Merge branch 'feat/parachain' into ed/feat/candidate_validation_precheck
edwardmack Oct 4, 2024
f8ec291
address PR comments
edwardmack Oct 7, 2024
d1a825c
address PR comments, update handling of errors.
edwardmack Oct 8, 2024
4796d38
add check for error in test
edwardmack Oct 9, 2024
3375ab1
Merge branch 'feat/parachain' into ed/feat/candidate_validation_prech…
edwardmack Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions dot/parachain/availability-store/mock_runtime_instance_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

32 changes: 2 additions & 30 deletions dot/parachain/backing/per_relay_parent_state.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@ import (
provisionermessages "github.com/ChainSafe/gossamer/dot/parachain/provisioner/messages"
statementedistributionmessages "github.com/ChainSafe/gossamer/dot/parachain/statement-distribution/messages"
parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
"github.com/ChainSafe/gossamer/lib/runtime"
wazero_runtime "github.com/ChainSafe/gossamer/lib/runtime/wazero"

"github.com/ChainSafe/gossamer/dot/parachain/util"
"github.com/ChainSafe/gossamer/lib/common"
)

Expand Down Expand Up @@ -288,7 +286,7 @@ func (rpState *perRelayParentState) validateAndMakeAvailable(
return fmt.Errorf("getting validation code by hash: %w", err)
}

executorParams, err := executorParamsAtRelayParent(rt, relayParent)
executorParams, err := util.ExecutorParamsAtRelayParent(rt, relayParent)
if err != nil {
return fmt.Errorf("getting executor params for relay parent %s: %w", relayParent, err)
}
Expand Down Expand Up @@ -383,32 +381,6 @@ func (rpState *perRelayParentState) validateAndMakeAvailable(
return nil
}

func executorParamsAtRelayParent(rt runtime.Instance, relayParent common.Hash,
) (*parachaintypes.ExecutorParams, error) {
sessionIndex, err := rt.ParachainHostSessionIndexForChild()
if err != nil {
return nil, fmt.Errorf("getting session index for relay parent %s: %w", relayParent, err)
}

executorParams, err := rt.ParachainHostSessionExecutorParams(sessionIndex)
if err != nil {
if errors.Is(err, wazero_runtime.ErrExportFunctionNotFound) {
// Runtime doesn't yet support the api requested,
// should execute anyway with default set of parameters.
defaultExecutorParams := parachaintypes.NewExecutorParams()
return &defaultExecutorParams, nil
}
return nil, err
}

if executorParams == nil {
// should never happen
return nil, fmt.Errorf("executor params for relay parent %s is nil", relayParent)
}

return executorParams, nil
}

func getPovFromValidator(
subSystemToOverseer chan<- any,
chRelayParentAndCommand chan relayParentAndCommand,
Expand Down
82 changes: 81 additions & 1 deletion dot/parachain/candidate-validation/candidate_validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@ import (
"context"
"errors"
"fmt"
"time"

parachainruntime "github.com/ChainSafe/gossamer/dot/parachain/runtime"
parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
"github.com/ChainSafe/gossamer/dot/parachain/util"
"github.com/ChainSafe/gossamer/lib/common"
"github.com/ChainSafe/gossamer/lib/runtime"
)
Expand Down Expand Up @@ -99,7 +101,12 @@ func (cv *CandidateValidation) processMessage(msg any) {
}

case PreCheck:
panic("TODO: implement functionality to handle PreCheck, see issue #3921")
outcome, err := cv.precheckPvF(msg.RelayParent, msg.ValidationCodeHash)
if err != nil {
logger.Errorf("failed to precheck: %w", err)
}
logger.Debugf("Precheck outcome: %v", outcome)
msg.ResponseSender <- outcome

case parachaintypes.ActiveLeavesUpdateSignal:
_ = cv.ProcessActiveLeavesUpdateSignal(msg)
Expand Down Expand Up @@ -226,3 +233,76 @@ func (cv *CandidateValidation) validateFromChainState(msg ValidateFromChainState
Data: *result,
}
}

// precheckPvF prechecks the parachain validation function by retrieving the validation code from the runtime instance
// and calling the precheck method on the pvf host. It returns the precheck outcome.
func (cv *CandidateValidation) precheckPvF(relayParent common.Hash, validationCodeHash parachaintypes.
ValidationCodeHash) (PreCheckOutcome, error) {
runtimeInstance, err := cv.BlockState.GetRuntime(relayParent)
if err != nil {
return PreCheckOutcomeFailed, fmt.Errorf("failed to get runtime instance: %w", err)
}

code, err := runtimeInstance.ParachainHostValidationCodeByHash(common.Hash(validationCodeHash))
if err != nil {
return PreCheckOutcomeFailed, fmt.Errorf("failed to get validation code by hash: %w", err)
}

executorParams, err := util.ExecutorParamsAtRelayParent(runtimeInstance, relayParent)
if err != nil {
return PreCheckOutcomeInvalid, fmt.Errorf("failed to acquire params for the session, thus voting against: %w", err)
}

kind := parachaintypes.NewPvfPrepTimeoutKind()
err = kind.SetValue(parachaintypes.Precheck{})
if err != nil {
return PreCheckOutcomeFailed, fmt.Errorf("failed to set value: %w", err)
}

prepTimeout := pvfPrepTimeout(*executorParams, kind)

pvf := PvFPrepData{
code: *code,
codeHash: validationCodeHash,
executorParams: *executorParams,
prepTimeout: prepTimeout,
prepKind: kind,
}
err = cv.pvfHost.precheck(pvf)
if err != nil {
return PreCheckOutcomeFailed, fmt.Errorf("failed to precheck: %w", err)
}
return PreCheckOutcomeValid, nil
}

// pvfPrepTimeout To determine the amount of timeout time for the pvf execution.
//
// The time period after which the preparation worker is considered
//
// unresponsive and will be killed.
func pvfPrepTimeout(params parachaintypes.ExecutorParams, kind parachaintypes.PvfPrepTimeoutKind) time.Duration {
for _, param := range params {
val, err := param.Value()
if err != nil {
logger.Errorf("determining parameter values %w", err)
}
switch val := val.(type) {
case parachaintypes.PvfPrepTimeout:
// convert milliseconds to nanoseconds and cast to time.Duration.
return time.Duration(val.Millisec * 1000000)
edwardmack marked this conversation as resolved.
Show resolved Hide resolved
}
}

timeoutKind, err := kind.Value()
if err != nil {
return time.Second * 2
}
switch timeoutKind.(type) {
case parachaintypes.Precheck:
return time.Second * 2
case parachaintypes.Lenient:
return time.Second * 10
default:
return time.Second * 2
}
}
123 changes: 123 additions & 0 deletions dot/parachain/candidate-validation/candidate_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package candidatevalidation

import (
"context"
"fmt"
"os"
"testing"

Expand Down Expand Up @@ -508,3 +509,125 @@ func TestCandidateValidation_processMessageValidateFromChainState(t *testing.T)
})
}
}

func Test_precheckPvF(t *testing.T) {
t.Parallel()
ctrl := gomock.NewController(t)
t.Cleanup(ctrl.Finish)
candidate, validationCode := createTestCandidateReceiptAndValidationCodeWParaId(t, 1000)

mockInstance := NewMockInstance(ctrl)
mockInstance.EXPECT().ParachainHostValidationCodeByHash(common.Hash(candidate.Descriptor.ValidationCodeHash)).
Return(&validationCode, nil)
mockInstance.EXPECT().ParachainHostSessionIndexForChild().Return(parachaintypes.SessionIndex(1), nil)

executionParams := parachaintypes.ExecutorParams{}
timeout := parachaintypes.PvfPrepTimeout{
PvfPrepTimeoutKind: func() parachaintypes.PvfPrepTimeoutKind {
kind := parachaintypes.NewPvfPrepTimeoutKind()
if err := kind.SetValue(parachaintypes.Precheck{}); err != nil {
panic(err)
}
return kind
}(),
Millisec: 1000,
}
timeoutParam := parachaintypes.NewExecutorParam()
err := timeoutParam.SetValue(timeout)
require.NoError(t, err)
executionParams = append(executionParams, timeoutParam)
mockInstance.EXPECT().ParachainHostSessionExecutorParams(parachaintypes.SessionIndex(1)).Return(&executionParams,
nil)

mockInstanceExecutorError := NewMockInstance(ctrl)
mockInstanceExecutorError.EXPECT().ParachainHostValidationCodeByHash(common.MustHexToHash("0x04")).Return(
&parachaintypes.ValidationCode{}, nil)
mockInstanceExecutorError.EXPECT().ParachainHostSessionIndexForChild().Return(parachaintypes.SessionIndex(2), nil)
mockInstanceExecutorError.EXPECT().ParachainHostSessionExecutorParams(parachaintypes.SessionIndex(2)).Return(
nil, fmt.Errorf("executor params not found"))

mockInstanceShortTimeout := NewMockInstance(ctrl)
mockInstanceShortTimeout.EXPECT().ParachainHostValidationCodeByHash(common.MustHexToHash("0x0404")).Return(
&validationCode, nil)
mockInstanceShortTimeout.EXPECT().ParachainHostSessionIndexForChild().Return(parachaintypes.SessionIndex(3), nil)
executionParamsShortTimeout := parachaintypes.ExecutorParams{}
timeoutShort := parachaintypes.PvfPrepTimeout{
PvfPrepTimeoutKind: func() parachaintypes.PvfPrepTimeoutKind {
kind := parachaintypes.NewPvfPrepTimeoutKind()
if err := kind.SetValue(parachaintypes.Precheck{}); err != nil {
panic(err)
}
return kind
}(),
Millisec: 1,
}
timeoutShortParam := parachaintypes.NewExecutorParam()
err = timeoutShortParam.SetValue(timeoutShort)
require.NoError(t, err)
executionParamsShortTimeout = append(executionParamsShortTimeout, timeoutShortParam)
mockInstanceShortTimeout.EXPECT().ParachainHostSessionExecutorParams(parachaintypes.SessionIndex(3)).Return(
&executionParamsShortTimeout, nil)

mockBlockState := NewMockBlockState(ctrl)
mockBlockState.EXPECT().GetRuntime(common.MustHexToHash("0x01")).Return(nil, fmt.Errorf("runtime not found"))
mockBlockState.EXPECT().GetRuntime(common.MustHexToHash("0x02")).Return(mockInstance, nil)
mockBlockState.EXPECT().GetRuntime(common.MustHexToHash("0x03")).Return(mockInstanceExecutorError, nil)
mockBlockState.EXPECT().GetRuntime(common.MustHexToHash("0x04")).Return(mockInstanceShortTimeout, nil)

tests := map[string]struct {
msg PreCheck
expectedResult PreCheckOutcome
expectedError error
}{
"validation_code_not_found": {
msg: PreCheck{
RelayParent: common.MustHexToHash("0x01"),
},
expectedResult: PreCheckOutcomeFailed,
expectedError: fmt.Errorf("failed to get runtime instance: runtime not found"),
},
"invalid_executor_params": {
msg: PreCheck{
RelayParent: common.MustHexToHash("0x03"),
ValidationCodeHash: parachaintypes.ValidationCodeHash(common.MustHexToHash("0x04")),
},
expectedResult: PreCheckOutcomeInvalid,
expectedError: fmt.Errorf("failed to acquire params for the session, thus voting against: " +
"executor params not found"),
},
"precheck_timeout": {
msg: PreCheck{
RelayParent: common.MustHexToHash("0x04"),
ValidationCodeHash: parachaintypes.ValidationCodeHash(common.MustHexToHash("0x0404")),
},
expectedResult: PreCheckOutcomeFailed,
expectedError: fmt.Errorf("failed to precheck: failed to create a new worker: precheck timed out"),
},
"happy_path": {
msg: PreCheck{
RelayParent: common.MustHexToHash("0x02"),
ValidationCodeHash: candidate.Descriptor.ValidationCodeHash,
},
expectedResult: PreCheckOutcomeValid,
},
}
for name, tt := range tests {
tt := tt

t.Run(name, func(t *testing.T) {
t.Parallel()

candidateValidationSubsystem := CandidateValidation{
pvfHost: newValidationHost(),
BlockState: mockBlockState,
}
result, err := candidateValidationSubsystem.precheckPvF(tt.msg.RelayParent, tt.msg.ValidationCodeHash)
require.Equal(t, tt.expectedResult, result)
if tt.expectedError != nil {
require.EqualError(t, err, tt.expectedError.Error())
} else {
require.NoError(t, err)
}
edwardmack marked this conversation as resolved.
Show resolved Hide resolved
})
}
}
3 changes: 3 additions & 0 deletions dot/parachain/candidate-validation/host.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ func newValidationHost() *host {
workerPool: newWorkerPool(),
}
}
func (v *host) precheck(data PvFPrepData) error {
return v.workerPool.handlePrecheckPvF(data)
}

func (v *host) validate(msg *ValidationTask) (*ValidationResult, error) {
validationCodeHash := msg.ValidationCode.Hash()
Expand Down
2 changes: 1 addition & 1 deletion dot/parachain/candidate-validation/messages.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ type ValidateFromExhaustive struct {
type PreCheck struct {
RelayParent common.Hash
ValidationCodeHash parachaintypes.ValidationCodeHash
PreCheckOutcome chan PreCheckOutcome
ResponseSender chan PreCheckOutcome
}

// PreCheckOutcome represents the outcome of the candidate-validation pre-check request
Expand Down
Loading
Loading