OmicronZoneType variants don't describe all the addresses in use by those zones

[jgallagher]

OmicronZoneType::InternalDns contains two socket addresses because it's effectively running two services (a dropshot server for administration, and a DNS server):

omicron/nexus-sled-agent-shared/src/inventory.rs

Lines 232 to 233 in f3ab338

	http_address: SocketAddrV6,
	dns_address: SocketAddrV6,

We have several other variants that are running multiple services but which only contain a single socket address:

• OmicronZoneType::Cockroach has the address for cockroach itself, but does not describe the address for the cockroach-admin server running alongside
• Similarly, OmicronZoneType::{Clickhouse,ClickhouseServer,ClickhouseKeeper} have the address for clickhouse itself, but do not describe the address for the clickhouse-admin server

sled-agent (and DNS, probably?) is forced to combine the IP from one address (e.g., Cockroach.address) with a hard-coded port number (COCKROACH_ADMIN_PORT) in order to configure these supplementary services.

This doesn't seem particularly urgent or terrible, as constant ports are all we use today, but is somewhat inconsistent. #6794 removed OmicronZoneConfig::underlay_address (the parent struct of OmicronZoneType) to remove some duplication; I think there are at least two possible ways to address this issue (one of which would undo #6794, and both of which would require some care to support compatibility across upgrades):

• Add new SocketAddrV6 fields to the relevant variants for their supplementary services
• Restore OmicronZoneConfig::underlay_address, and replace the existing OmicronZoneType SocketAddrV6 fields with just port numbers (and add port numbers for the missing services while we're there)

These aren't equivalent: the first (which matches how our DNS zone types are described) allows different services in the zone to run on different IPs; the second would require all services to use the same IP. We've sprinkled assumptions that a given Omicron service zone only has one underlay IP throughout the codebase, but I'm not sure whether that was an intentional design choice or something we may want to change in the future.

[andrewjstone]

Is it a problem to have all zone services listen on the same IP? I think that's simplest and seems to be what we do everywhere. I'm having trouble coming up with a reason why we'd need to use more than one IP. The only one I could think of is if we run multiple instances of the same service and want to use the same hardcoded port.

[jgallagher]

Is it a problem to have all zone services listen on the same IP?

Not that I know of! But I wanted to point out that the two cleanup options are different here, and ask whether we care.