Overlord can support many autoscalers of different categories

[VladimirIordanov]

Motivation

Since #7066 Druid supports different categories of tasks which run on dedicated instance of MiddleManager. But Overlord still can spawn only one instance of MiddleManager via Autoscaler. That is not flexible when Druid should support different categories of tasks and cluster parameter for the each of the category is different. For instance realtime and batch tasks will require clusters of different configurations: workers count; CPU, memory, per each worker, etc. In that case overlord could provision dedicated cluster per each task category.

Proposed changes

Create CategoriedWorkerBehaviorConfig to support declaration of several autoscalers. Each of the autoscalers should have "category" property which should link it to according "category" of Task based on CategoryConfig.

Change all current implementations of ProvisioningStrategy interface to select appropriate AutoScaler from CategoriedWorkerBehaviorConfig during "provisioning" or "terminating". In that way overlord can spawn/terminate a corresponding cluster for a submitted task. If there is no matched autoscaler found in the config then the default one will be used. If there is no declaration of default autoscaler then there will be no autoscaling and tasks will be in pending state.

Rationale

Structure of the CategoriedWorkerBehaviorConfig will be like this:

{
  "selectStrategy": {
    "type": "fillCapacity",
    ...
  },
  "autoScalers": [
    {
        "type": "ec2",
        "minNumWorkers": 0,
        "maxNumWorkers: 5,
        "envConfig": {
           "availabilityZone": "us-east-1a"
           ...
        }
        ...
    },
    {
        "type": "ec2",
        "category": "_category_1_",
        "minNumWorkers": 2,
        "maxNumWorkers: 10,
        "envConfig": {
           "availabilityZone": "us-west-2a"
           ...
        }
        ...
    },
    {
        "type": "ec2",
        "category": "_category_1_",
        "minNumWorkers": 1,
        "maxNumWorkers: 3,
        "envConfig": {
           "availabilityZone": "us-east-2a"
           ...
        }
        ...
    }
  ]
}

The "autoScalers" section will contain definitions of all supported autoscalers as array.
Where "category_1" and "category_2" are category names according to supported types of tasks (and middleManagers). Default autoscaler will not have category name and its declaration is optional. If it was declared then any task which doesn't have autoscaler for its category will be served by default autoscaler.

Operational impact

No impact because this PR will not change the existing config files.

[fxpz]

Any update on this feature?