fix SamplingDecodeTest and SamplingDecodeTest2 unittest failure

[zhyncs]

Motivation

fix SamplingDecodeTest and SamplingDecodeTest2 unittest failure

mkdir bld && cd bld && bash ../debug.sh && ninja -j48 && ./bin/unittest

Hi @lzhangzz @irexyc @lvhan028 May you help review this pr? Thanks.

Modification

as titled

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[zhyncs]

before the fix

[  FAILED  ] 28 tests, listed below:
[  FAILED  ] SamplingDecodeTest/0.TopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.BatchTopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.TopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.BatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.TopKTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.BatchTopKTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.TopKBatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.BatchTopKBatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsZeroTopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsZeroTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsZeroTopKTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsZeroBatchTopKTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsZeroTopKBatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsBatchTopKContainZero, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsBatchTopPContainZero, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsBatchTopKTopPContainZero, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsTopKBatchTopPContainZero, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.InvalidArgsBatchTopKBatchTopPContainZero, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.LocalBatchBatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest/0.LocalBatchBatchTopKBatchTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessSingleRandTopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessSingleRandTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessBatchRandTopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessBatchRandTopP, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessBatchRandTopKLocalBatch, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessBatchRandTopPLocalBatch, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessCumLogProbTopK, where TypeParam = float
[  FAILED  ] SamplingDecodeTest2/0.CorrectnessCumLogProbTopP, where TypeParam = float

after

[==========] 111 tests from 14 test suites ran. (32080 ms total)
[  PASSED  ] 111 tests.

[irexyc]

lmdeploy/tests/csrc/unittests/test_sampling_layer.cu

Lines 1035 to 1041 in 12e8609

	// Prepare decoding arguments
	const size_t random_seed_size = use_single_random_seed ? 1 : batch_size;
	const size_t period_size = 3;
	unsigned long long* random_seed = new unsigned long long[random_seed_size];
	for (size_t i = 0; i < random_seed_size; ++i) {
	random_seed[i] = i / period_size;
	}

Different with fastertransformer, we initialize d_curand_state there. So in unittests, we should call invokeCurandBatchInitialize manually.

[zhyncs]

we should call invokeCurandBatchInitialize manually

ok

[zhyncs]

So in unittests, we should call invokeCurandBatchInitialize manually.

Hi @irexyc Could you please review the latest code? Thanks.