[Bug]: [benchmark] there is mmap memory usage on queryNode when mmap is disabled #36188

wangting0128 · 2024-09-11T09:14:15Z

Is there an existing issue for this?

I have searched the existing issues

Environment

- Milvus version: master-20240910-f4d0c589-amd64
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka): pulsar   
- SDK version(e.g. pymilvus v2.0.0rc2):2.4.5rc7
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

argo task: fouramf-bitmap-scenes-fdgrx
test case name: test_bitmap_locust_dql_ddl_cluster

server:

NAME                                                              READY   STATUS             RESTARTS          AGE     IP              NODE         NOMINATED NODE   READINESS GATES
fouramf-bitmap-scenes-fdgrx-8-etcd-0                              1/1     Running            0                 3h51m   10.104.24.160   4am-node29   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-etcd-1                              1/1     Running            0                 3h51m   10.104.30.81    4am-node38   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-etcd-2                              1/1     Running            0                 3h51m   10.104.18.127   4am-node25   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-datanode-54b6cbd87c-gtwdk    1/1     Running            2 (3h46m ago)     3h51m   10.104.6.177    4am-node13   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-indexnode-84549ff77d-h8wqv   1/1     Running            1 (3h51m ago)     3h51m   10.104.25.236   4am-node30   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-indexnode-84549ff77d-rbg5t   1/1     Running            1 (3h51m ago)     3h51m   10.104.17.192   4am-node23   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-mixcoord-8cd7b7c55-kpqcg     1/1     Running            2 (3h46m ago)     3h51m   10.104.4.148    4am-node11   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-proxy-6579b76787-lkkr2       1/1     Running            2 (3h46m ago)     3h51m   10.104.6.175    4am-node13   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-querynode-6557d69bd4-6hcwf   1/1     Running            1 (3h51m ago)     3h51m   10.104.4.149    4am-node11   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-milvus-querynode-6557d69bd4-b9xzp   1/1     Running            2 (3h50m ago)     3h51m   10.104.15.17    4am-node20   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-minio-0                             1/1     Running            0                 3h51m   10.104.24.159   4am-node29   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-minio-1                             1/1     Running            0                 3h51m   10.104.34.121   4am-node37   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-minio-2                             1/1     Running            0                 3h51m   10.104.30.84    4am-node38   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-minio-3                             1/1     Running            0                 3h51m   10.104.18.129   4am-node25   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-bookie-0                     1/1     Running            0                 3h51m   10.104.24.161   4am-node29   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-bookie-1                     1/1     Running            0                 3h51m   10.104.30.83    4am-node38   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-bookie-2                     1/1     Running            0                 3h51m   10.104.16.98    4am-node21   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-bookie-init-pnnf4            0/1     Completed          0                 3h51m   10.104.6.176    4am-node13   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-broker-0                     1/1     Running            0                 3h51m   10.104.34.117   4am-node37   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-proxy-0                      1/1     Running            0                 3h51m   10.104.23.229   4am-node27   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-pulsar-init-5tr52            0/1     Completed          0                 3h51m   10.104.4.147    4am-node11   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-recovery-0                   1/1     Running            0                 3h51m   10.104.13.222   4am-node16   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-zookeeper-0                  1/1     Running            0                 3h51m   10.104.15.19    4am-node20   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-zookeeper-1                  1/1     Running            0                 3h50m   10.104.18.137   4am-node25   <none>           <none>
fouramf-bitmap-scenes-fdgrx-8-pulsar-zookeeper-2                  1/1     Running            0                 3h50m   10.104.32.252   4am-node39   <none>           <none>

test result:

2024-09-10 07:48:12,076 -  INFO - fouram]: Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: --------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|----------- (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     hybrid_search                                                                   2637     0(0.00%) |   6010    1759   24552   5400 |    0.24        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     query                                                                           2612     0(0.00%) |    797      70   17935    210 |    0.24        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     scene_hybrid_search_test                                                        2605     0(0.00%) |  25675    9852  116995  22000 |    0.24        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     scene_search_test                                                               2651     0(0.00%) |  17115    7748   85467  15000 |    0.25        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     scene_test                                                                      2748     0(0.00%) |  66986   63224   96813  66000 |    0.25        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: grpc     search                                                                          2623     0(0.00%) |   3176    1412   19506   2600 |    0.24        0.00 (stats.py:789)
[2024-09-10 07:48:12,076 -  INFO - fouram]: --------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|----------- (stats.py:789)
[2024-09-10 07:48:12,077 -  INFO - fouram]:          Aggregated                                                                     15876     0(0.00%) |  20319      70  116995  10000 |    1.47        0.00 (stats.py:789)
[2024-09-10 07:48:12,077 -  INFO - fouram]:  (stats.py:790)
[2024-09-10 07:48:12,082 -  INFO - fouram]: [PerfTemplate] Report data: 
{'server': {'deploy_tool': 'helm',
            'deploy_mode': 'cluster',
            'config_name': 'cluster_8c16m',
            'config': {'queryNode': {'resources': {'limits': {'cpu': '8.0', 'memory': '16Gi'}, 'requests': {'cpu': '5.0', 'memory': '9Gi'}}, 'replicas': 2},
                       'indexNode': {'resources': {'limits': {'cpu': '8.0', 'memory': '8Gi'}, 'requests': {'cpu': '5.0', 'memory': '5Gi'}}, 'replicas': 2},
                       'dataNode': {'resources': {'limits': {'cpu': '8.0', 'memory': '16Gi'}, 'requests': {'cpu': '5.0', 'memory': '9Gi'}}},
                       'cluster': {'enabled': True},
                       'pulsar': {},
                       'kafka': {},
                       'minio': {'metrics': {'podMonitor': {'enabled': True}}},
                       'etcd': {'metrics': {'enabled': True, 'podMonitor': {'enabled': True}}},
                       'metrics': {'serviceMonitor': {'enabled': True}},
                       'log': {'level': 'debug'},
                       'image': {'all': {'repository': 'harbor.milvus.io/milvus/milvus', 'tag': 'master-20240910-f4d0c589-amd64'}}},
            'host': 'fouramf-bitmap-scenes-fdgrx-8-milvus.qa-milvus.svc.cluster.local',
            'port': '19530',
            'uri': ''},
 'client': {'test_case_type': 'ConcurrentClientBase',
            'test_case_name': 'test_bitmap_locust_dql_ddl_cluster',
            'test_case_params': {'dataset_params': {'metric_type': 'L2',
                                                    'dim': 128,
                                                    'max_length': 100,
                                                    'scalars_index': {'id': {'index_type': 'INVERTED'},
                                                                      'int8_1': {'index_type': 'BITMAP'},
                                                                      'int16_1': {'index_type': 'BITMAP'},
                                                                      'int32_1': {'index_type': 'BITMAP'},
                                                                      'int64_1': {'index_type': 'BITMAP'},
                                                                      'varchar_1': {'index_type': 'BITMAP'},
                                                                      'bool_1': {'index_type': 'BITMAP'},
                                                                      'array_int8_1': {'index_type': 'BITMAP'},
                                                                      'array_int16_1': {'index_type': 'BITMAP'},
                                                                      'array_int32_1': {'index_type': 'BITMAP'},
                                                                      'array_int64_1': {'index_type': 'BITMAP'},
                                                                      'array_varchar_1': {'index_type': 'BITMAP'},
                                                                      'array_bool_1': {'index_type': 'BITMAP'}},
                                                    'vectors_index': {'float_vector_1': {'index_type': 'IVF_SQ8',
                                                                                         'index_param': {'nlist': 1024},
                                                                                         'metric_type': 'L2'}},
                                                    'scalars_params': {'array_int8_1': {'params': {'max_capacity': 13},
                                                                                        'other_params': {'dataset': 'random_algorithm',
                                                                                                         'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                              'specify_range': [-128, 128],
                                                                                                                              'max_capacity': 13}}},
                                                                       'array_int16_1': {'params': {'max_capacity': 13},
                                                                                         'other_params': {'dataset': 'random_algorithm',
                                                                                                          'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                               'specify_range': [-200, 200],
                                                                                                                               'max_capacity': 13}}},
                                                                       'array_int32_1': {'params': {'max_capacity': 13},
                                                                                         'other_params': {'dataset': 'random_algorithm',
                                                                                                          'algorithm_params': {'algorithm_name': 'specify_scope',
                                                                                                                               'specify_range': [-300, 300],
                                                                                                                               'max_capacity': 13}}},
                                                                       'array_int64_1': {'params': {'max_capacity': 13},
                                                                                         'other_params': {'dataset': 'random_algorithm',
                                                                                                          'algorithm_params': {'algorithm_name': 'fixed_value_range',
                                                                                                                               'specify_range': [-400, 432],
                                                                                                                               'batch': 50,
                                                                                                                               'max_capacity': 13}}},
                                                                       'array_varchar_1': {'params': {'max_capacity': 13},
                                                                                           'other_params': {'dataset': 'random_algorithm',
                                                                                                            'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                                 'specify_range': [-1500, 1500],
                                                                                                                                 'max_capacity': 13}}},
                                                                       'array_bool_1': {'params': {'max_capacity': 13}},
                                                                       'int8_1': {'other_params': {'dataset': 'random_algorithm',
                                                                                                   'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                        'specify_range': [-128, 128],
                                                                                                                        'max_capacity': 13}}},
                                                                       'int16_1': {'other_params': {'dataset': 'random_algorithm',
                                                                                                    'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                         'specify_range': [-200, 200],
                                                                                                                         'max_capacity': 13}}},
                                                                       'int32_1': {'other_params': {'dataset': 'random_algorithm',
                                                                                                    'algorithm_params': {'algorithm_name': 'specify_scope',
                                                                                                                         'specify_range': [-300, 300],
                                                                                                                         'max_capacity': 13}}},
                                                                       'int64_1': {'other_params': {'dataset': 'random_algorithm',
                                                                                                    'algorithm_params': {'algorithm_name': 'fixed_value_range',
                                                                                                                         'specify_range': [-400, 432],
                                                                                                                         'batch': 50,
                                                                                                                         'max_capacity': 13}}},
                                                                       'varchar_1': {'other_params': {'dataset': 'random_algorithm',
                                                                                                      'algorithm_params': {'algorithm_name': 'random_range',
                                                                                                                           'specify_range': [-1500, 1500],
                                                                                                                           'max_capacity': 13}}}},
                                                    'dataset_name': 'sift',
                                                    'dataset_size': 10000000,
                                                    'ni_per': 5000},
                                 'collection_params': {'other_fields': ['float_vector_1', 'int8_1', 'int16_1', 'int32_1', 'int64_1', 'varchar_1', 'bool_1',
                                                                        'array_int8_1', 'array_int16_1', 'array_int32_1', 'array_int64_1', 'array_varchar_1',
                                                                        'array_bool_1'],
                                                       'shards_num': 2},
                                 'resource_groups_params': {'reset': False},
                                 'database_user_params': {'reset_rbac': False, 'reset_db': False},
                                 'index_params': {'index_type': 'HNSW', 'index_param': {'M': 8, 'efConstruction': 200}},
                                 'concurrent_params': {'concurrent_number': 30, 'during_time': '3h', 'interval': 20, 'spawn_rate': None},
                                 'concurrent_tasks': [{'type': 'search',
                                                       'weight': 1,
                                                       'params': {'nq': 1000,
                                                                  'top_k': 10,
                                                                  'search_param': {'nprobe': 16},
                                                                  'expr': 'int8_1 == 100',
                                                                  'guarantee_timestamp': None,
                                                                  'partition_names': None,
                                                                  'output_fields': ['id', 'float_vector', 'int64_1'],
                                                                  'ignore_growing': False,
                                                                  'group_by_field': None,
                                                                  'timeout': 60,
                                                                  'random_data': True,
                                                                  'check_task': 'check_search_output',
                                                                  'check_items': {'nq': 1000}}},
                                                      {'type': 'query',
                                                       'weight': 1,
                                                       'params': {'ids': None,
                                                                  'expr': 'int64_1 > -1',
                                                                  'output_fields': ['*'],
                                                                  'offset': None,
                                                                  'limit': 10,
                                                                  'ignore_growing': False,
                                                                  'partition_names': None,
                                                                  'timeout': 60,
                                                                  'consistency_level': None,
                                                                  'random_data': False,
                                                                  'random_count': 0,
                                                                  'random_range': [0, 1],
                                                                  'field_name': 'id',
                                                                  'field_type': 'int64',
                                                                  'check_task': 'check_query_output',
                                                                  'check_items': {'expect_length': 10}}},
                                                      {'type': 'hybrid_search',
                                                       'weight': 1,
                                                       'params': {'nq': 10,
                                                                  'top_k': 10,
                                                                  'reqs': [{'search_param': {'ef': 32},
                                                                            'anns_field': 'float_vector',
                                                                            'expr': '(array_contains_any(array_int32_1, [0]) || array_contains(array_int64_1, '
                                                                                    '1)) || ((varchar_1 like "1%") and (bool_1 == True))',
                                                                            'top_k': 30},
                                                                           {'search_param': {'nprobe': 64},
                                                                            'anns_field': 'float_vector_1',
                                                                            'expr': 'not (int16_1 == int8_1) && ARRAY_CONTAINS_ANY(array_int64_1, [-1, 0, '
                                                                                    '1])'}],
                                                                  'rerank': {'RRFRanker': []},
                                                                  'output_fields': ['*'],
                                                                  'ignore_growing': False,
                                                                  'guarantee_timestamp': None,
                                                                  'partition_names': None,
                                                                  'timeout': 60,
                                                                  'random_data': True,
                                                                  'check_task': 'check_search_output',
                                                                  'check_items': {'output_fields': ['float_vector_1', 'int8_1', 'int16_1', 'int32_1', 'int64_1',
                                                                                                    'varchar_1', 'bool_1', 'array_int8_1', 'array_int16_1',
                                                                                                    'array_int32_1', 'array_int64_1', 'array_varchar_1',
                                                                                                    'array_bool_1', 'id', 'float_vector'],
                                                                                  'nq': 10}}},
                                                      {'type': 'scene_test',
                                                       'weight': 1,
                                                       'params': {'dim': 128,
                                                                  'data_size': 3000,
                                                                  'nb': 3000,
                                                                  'index_type': 'IVF_SQ8',
                                                                  'index_param': {'nlist': 2048},
                                                                  'metric_type': 'L2',
                                                                  'other_fields': [],
                                                                  'scalars_params': {},
                                                                  'scalars_index': {},
                                                                  'vectors_index': {}}},
                                                      {'type': 'scene_search_test',
                                                       'weight': 1,
                                                       'params': {'dataset': 'local',
                                                                  'dim': 128,
                                                                  'shards_num': 2,
                                                                  'data_size': 3000,
                                                                  'nb': 3000,
                                                                  'index_type': 'IVF_SQ8',
                                                                  'index_param': {'nlist': 2048},
                                                                  'metric_type': 'L2',
                                                                  'other_fields': ['array_int64_1', 'array_bool_1', 'array_varchar_1'],
                                                                  'replica_number': 1,
                                                                  'nq': 1,
                                                                  'top_k': 10,
                                                                  'search_param': {'nprobe': 16},
                                                                  'search_counts': 10,
                                                                  'scalars_params': {},
                                                                  'scalars_index': {'array_int64_1': {'index_type': 'BITMAP'},
                                                                                    'array_bool_1': {'index_type': 'BITMAP'},
                                                                                    'array_varchar_1': {'index_type': 'BITMAP'}},
                                                                  'vectors_index': {},
                                                                  'prepare_before_insert': False,
                                                                  'new_connect': False,
                                                                  'new_user': False}},
                                                      {'type': 'scene_hybrid_search_test',
                                                       'weight': 1,
                                                       'params': {'nq': 1,
                                                                  'top_k': 1,
                                                                  'reqs': [{'search_param': {'nprobe': 128},
                                                                            'anns_field': 'float_vector',
                                                                            'expr': 'bool_1 == True',
                                                                            'top_k': 100},
                                                                           {'search_param': {'nprobe': 32},
                                                                            'anns_field': 'binary_vector_scene_hybrid_search_test_1',
                                                                            'expr': 'bool_1 != True',
                                                                            'top_k': 10},
                                                                           {'search_param': {'search_list': 30},
                                                                            'anns_field': 'float16_vector_scene_hybrid_search_test_2',
                                                                            'expr': 'int64_1 >= 1500',
                                                                            'top_k': 5},
                                                                           {'search_param': {'drop_ratio_search': 0.1},
                                                                            'anns_field': 'sparse_float_vector_scene_hybrid_search_test_3',
                                                                            'expr': 'varchar_1 like "1%"',
                                                                            'top_k': 10}],
                                                                  'rerank': {'RRFRanker': []},
                                                                  'output_fields': ['*'],
                                                                  'ignore_growing': False,
                                                                  'guarantee_timestamp': None,
                                                                  'partition_names': None,
                                                                  'timeout': 600,
                                                                  'random_data': True,
                                                                  'dataset': 'local',
                                                                  'dim': 128,
                                                                  'shards_num': 2,
                                                                  'data_size': 3000,
                                                                  'nb': 3000,
                                                                  'index_type': 'IVF_SQ8',
                                                                  'index_param': {'nlist': 2048},
                                                                  'metric_type': 'L2',
                                                                  'other_fields': ['binary_vector_scene_hybrid_search_test_1',
                                                                                   'float16_vector_scene_hybrid_search_test_2',
                                                                                   'sparse_float_vector_scene_hybrid_search_test_3', 'int64_1', 'bool_1',
                                                                                   'varchar_1'],
                                                                  'replica_number': 1,
                                                                  'scalars_params': {'binary_vector_scene_hybrid_search_test_1': {'params': {'dim': 512},
                                                                                                                                  'other_params': {'dataset': 'binary'}},
                                                                                     'float16_vector_scene_hybrid_search_test_2': {'params': {'dim': 64}}},
                                                                  'scalars_index': {'int64_1': {},
                                                                                    'bool_1': {'index_type': 'BITMAP'},
                                                                                    'varchar_1': {'index_type': 'BITMAP'}},
                                                                  'vectors_index': {'binary_vector_scene_hybrid_search_test_1': {'index_type': 'BIN_IVF_FLAT',
                                                                                                                                 'index_param': {'nlist': 2048},
                                                                                                                                 'metric_type': 'JACCARD'},
                                                                                    'float16_vector_scene_hybrid_search_test_2': {'index_type': 'DISKANN',
                                                                                                                                  'index_param': {},
                                                                                                                                  'metric_type': 'IP'},
                                                                                    'sparse_float_vector_scene_hybrid_search_test_3': {'index_type': 'SPARSE_WAND',
                                                                                                                                       'index_param': {'drop_ratio_build': 0.2},
                                                                                                                                       'metric_type': 'IP'}},
                                                                  'prepare_before_insert': False,
                                                                  'hybrid_search_counts': 10,
                                                                  'new_connect': False,
                                                                  'new_user': False}}]},
            'run_id': 2024091006321793,
            'datetime': '2024-09-10 03:57:12.172241',
            'client_version': '2.5.0'},
 'result': {'test_result': {'index': {'RT': 706.6355,
                                      'float_vector_1': {'RT': 196.4834},
                                      'id': {'RT': 70.1798},
                                      'int8_1': {'RT': 74.8061},
                                      'int16_1': {'RT': 3.5692},
                                      'int32_1': {'RT': 0.5236},
                                      'int64_1': {'RT': 0.5248},
                                      'varchar_1': {'RT': 0.5262},
                                      'bool_1': {'RT': 0.5247},
                                      'array_int8_1': {'RT': 0.5231},
                                      'array_int16_1': {'RT': 0.5664},
                                      'array_int32_1': {'RT': 0.5391},
                                      'array_int64_1': {'RT': 0.5344},
                                      'array_varchar_1': {'RT': 0.5301},
                                      'array_bool_1': {'RT': 0.5233}},
                            'insert': {'total_time': 1224.924, 'VPS': 8163.7718, 'batch_time': 0.6125, 'batch': 5000},
                            'flush': {'RT': 3.0263},
                            'load': {'RT': 14.4123},
                            'Locust': {'Aggregated': {'Requests': 15876,
                                                      'Fails': 0,
                                                      'RPS': 1.47,
                                                      'fail_s': 0.0,
                                                      'RT_max': 116995.74,
                                                      'RT_avg': 20320.0,
                                                      'TP50': 10000.0,
                                                      'TP99': 74000.0},
                                       'hybrid_search': {'Requests': 2637,
                                                         'Fails': 0,
                                                         'RPS': 0.24,
                                                         'fail_s': 0.0,
                                                         'RT_max': 24552.9,
                                                         'RT_avg': 6010.01,
                                                         'TP50': 5400.0,
                                                         'TP99': 15000.0},
                                       'query': {'Requests': 2612,
                                                 'Fails': 0,
                                                 'RPS': 0.24,
                                                 'fail_s': 0.0,
                                                 'RT_max': 17935.36,
                                                 'RT_avg': 797.56,
                                                 'TP50': 210.0,
                                                 'TP99': 5700.0},
                                       'scene_hybrid_search_test': {'Requests': 2605,
                                                                    'Fails': 0,
                                                                    'RPS': 0.24,
                                                                    'fail_s': 0.0,
                                                                    'RT_max': 116995.74,
                                                                    'RT_avg': 25675.92,
                                                                    'TP50': 22000.0,
                                                                    'TP99': 68000.0},
                                       'scene_search_test': {'Requests': 2651,
                                                             'Fails': 0,
                                                             'RPS': 0.25,
                                                             'fail_s': 0.0,
                                                             'RT_max': 85467.48,
                                                             'RT_avg': 17115.05,
                                                             'TP50': 15000.0,
                                                             'TP99': 45000.0},
                                       'scene_test': {'Requests': 2748,
                                                      'Fails': 0,
                                                      'RPS': 0.25,
                                                      'fail_s': 0.0,
                                                      'RT_max': 96813.54,
                                                      'RT_avg': 66986.79,
                                                      'TP50': 66000.0,
                                                      'TP99': 79000.0},
                                       'search': {'Requests': 2623,
                                                  'Fails': 0,
                                                  'RPS': 0.24,
                                                  'fail_s': 0.0,
                                                  'RT_max': 19506.77,
                                                  'RT_avg': 3176.19,
                                                  'TP50': 2600.0,
                                                  'TP99': 8500.0}}}}}

Expected Behavior

No response

Steps To Reproduce

concurrent test and calculation of RT and QPS

        :purpose:  `primary key: INT64`
            1. building `BITMAP` index on all supported 12 scalar fields, `INVERTED` index on pk field
            2. 2 fields of different vector types
            3. verify DQL & DML requests

        :test steps:
            1. create collection with fields:
                'float_vector': 128dim
                'float_vector_1': 768dim
                'id': primary key type is INT64

                all scalar fields: varchar max_length=100, array max_capacity=13
            2. build indexes:
                HNSW: 'float_vector'
                IVF_SQ8: 'float_vector_1'

                BITMAP: all scalar fields
                INVERTED: 'id' prmary key field
            3. insert 10 million data
            4. flush collection
            5. build indexes again using the same params
            6. load collection
            7. concurrent request:
                - search
                - query
                - hybrid_search
                - scene_test
                    (collection: create->insert->flush->index->drop)
                - scene_search_test
                    (collection: create->insert->flush->index->load->search->drop)
                - scene_hybrid_search_test: 4 vector fields, 3 scalar fields
                    (collection: create->insert->flush->index->load->hybrid_search->drop)

Milvus Log

No response

Anything else?

No response

The text was updated successfully, but these errors were encountered:

sunby · 2024-09-12T02:47:49Z

@wangting0128 reading from chunk cache always enable mmap.

wangting0128 · 2024-09-14T09:08:36Z

After turning off vector retrieve, there is no additional mmap memory usage

verification is passed

argo task: fouramf-bitmap-perf-alter-8f7ss
image: master-20240912-5e5e87cc-amd64

wangting0128 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. test/benchmark benchmark test labels Sep 11, 2024

wangting0128 added this to the 2.5.0 milestone Sep 11, 2024

wangting0128 assigned sunby and yanliang567 Sep 11, 2024

yanliang567 removed their assignment Sep 12, 2024

yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Sep 12, 2024

wangting0128 closed this as completed Sep 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: [benchmark] there is mmap memory usage on queryNode when mmap is disabled #36188

[Bug]: [benchmark] there is mmap memory usage on queryNode when mmap is disabled #36188

wangting0128 commented Sep 11, 2024

sunby commented Sep 12, 2024

wangting0128 commented Sep 14, 2024

[Bug]: [benchmark] there is mmap memory usage on queryNode when mmap is disabled #36188

[Bug]: [benchmark] there is mmap memory usage on queryNode when mmap is disabled #36188

Comments

wangting0128 commented Sep 11, 2024

Is there an existing issue for this?

Environment

Current Behavior

Expected Behavior

Steps To Reproduce

Milvus Log

Anything else?

sunby commented Sep 12, 2024

wangting0128 commented Sep 14, 2024

verification is passed