[Observability Onboarding] Determine which dashboard to show #186653
Assignees
Labels
Comments
thomheymann
added
Team:obs-ux-logs
|
Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs) |
|
After speaking to Fleet team, we can hard code the dashboard IDs |
flash1293
added a commit
that referenced
this issue
Jul 17, 2024
Closes #186653 This PR does two things: Show the auto-detect flow in the list: <img width="1219" alt="Screenshot 2024-07-11 at 11 28 00" src="https://github.com/elastic/kibana/assets/1508364/1f7845cc-dec1-45b8-8c8c-4f86de389773"> Only render the main overview dashboard for all the integrations: <img width="959" alt="Screenshot 2024-07-12 at 17 02 35" src="https://github.com/user-attachments/assets/bad4fbde-836c-4d56-9d56-cb03bac860f6"> It's sharing the same component with the kubernetes flow: <img width="555" alt="Screenshot 2024-07-12 at 17 15 54" src="https://github.com/user-attachments/assets/757bd37e-2a40-4430-9d0f-9da967a3629d"> ## Considerations * We discussed removing the custom log files flow form the prominently shown tiles, but I think we should keep it there, mostly because there is no better one to show - we could show nginx, but the auto-detect flow is taking care of that, so not sure what would be a better fit than the "stream log files" one * This is not building back the old system logs flow, let's split this out to a separate PR * I removed the "logs" prefix from "get started with", as it's not always about logs. * Designs show sometimes two dashboards, but I think for the integrations we support, one of them is clearly preferable and I would like to focus on those <details> <summary>Dummy data to show the success message right away</summary> ``` const mockedProgressData = { progress: { 'logs-detect': { status: 'complete', message: '', }, 'install-integrations': { status: 'complete', payload: [ { installSource: 'registry', pkgName: 'nginx', pkgVersion: '1.22.0', title: 'Nginx', inputs: [ { id: 'logfile-nginx', type: 'logfile', streams: [ { id: 'logfile-nginx.access', data_stream: { type: 'logs', dataset: 'nginx.access', }, ignore_older: '72h', paths: ['/var/log/nginx/access.log*'], tags: ['nginx-access'], exclude_files: ['.gz$'], processors: [ { add_locale: null, }, ], }, { id: 'logfile-nginx.error', data_stream: { type: 'logs', dataset: 'nginx.error', }, ignore_older: '72h', paths: ['/var/log/nginx/error.log*'], tags: ['nginx-error'], exclude_files: ['.gz$'], multiline: { pattern: '^\\d{4}\\/\\d{2}\\/\\d{2} ', negate: true, match: 'after', }, processors: [ { add_locale: null, }, ], }, ], }, { id: 'nginx/metrics-nginx', type: 'nginx/metrics', streams: [ { id: 'nginx/metrics-nginx.stubstatus', data_stream: { type: 'metrics', dataset: 'nginx.stubstatus', }, metricsets: ['stubstatus'], hosts: ['http://127.0.0.1:80'], tags: ['nginx-stubstatus'], period: '10s', server_status_path: '/nginx_status', }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'nginx.access', }, { type: 'logs', dataset: 'nginx.error', }, { type: 'metrics', dataset: 'nginx.stubstatus', }, ], kibanaAssets: [ { id: 'nginx-023d2930-f1a5-11e7-a9ef-93c69af7b129', type: 'dashboard', }, { id: 'nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519', type: 'dashboard', }, { id: 'nginx-55a9e6e0-a29e-11e7-928f-5dbe6f6f5519', type: 'dashboard', }, { id: 'nginx-Logs-ml', type: 'ml-module', }, ], }, { installSource: 'registry', pkgName: 'apache', pkgVersion: '1.17.2', title: 'Apache HTTP Server', inputs: [ { id: 'logfile-apache', type: 'logfile', streams: [ { id: 'logfile-apache.access', data_stream: { type: 'logs', dataset: 'apache.access', }, paths: [ '/var/log/apache2/access.log*', '/var/log/apache2/other_vhosts_access.log*', '/var/log/httpd/access_log*', ], tags: ['apache-access'], exclude_files: ['.gz$'], }, { id: 'logfile-apache.error', data_stream: { type: 'logs', dataset: 'apache.error', }, paths: ['/var/log/apache2/error.log*', '/var/log/httpd/error_log*'], exclude_files: ['.gz$'], tags: ['apache-error'], processors: [ { add_locale: null, }, ], }, ], }, { id: 'apache/metrics-apache', type: 'apache/metrics', streams: [ { id: 'apache/metrics-apache.status', data_stream: { type: 'metrics', dataset: 'apache.status', }, metricsets: ['status'], hosts: ['http://127.0.0.1'], period: '30s', server_status_path: '/server-status', }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'apache.access', }, { type: 'logs', dataset: 'apache.error', }, { type: 'metrics', dataset: 'apache.status', }, ], kibanaAssets: [ { id: 'apache-Logs-Apache-Dashboard', type: 'dashboard', }, { id: 'apache-Metrics-Apache-HTTPD-server-status', type: 'dashboard', }, { id: 'apache-22057f20-3a12-11eb-8946-296aab7b13db', type: 'visualization', }, { id: 'apache-errors-log', type: 'search', }, { id: 'apache-72259620-e236-11ec-baf0-970634a1784d', type: 'map', }, { id: 'apache-Logs-ml', type: 'ml-module', }, ], }, { installSource: 'registry', pkgName: 'docker', pkgVersion: '2.10.0', title: 'Docker', inputs: [ { id: 'docker/metrics-docker', type: 'docker/metrics', streams: [ { id: 'docker/metrics-docker.container', data_stream: { type: 'metrics', dataset: 'docker.container', }, metricsets: ['container'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.cpu', data_stream: { type: 'metrics', dataset: 'docker.cpu', }, metricsets: ['cpu'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.diskio', data_stream: { type: 'metrics', dataset: 'docker.diskio', }, metricsets: ['diskio'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, skip_major: [9, 253], }, { id: 'docker/metrics-docker.event', data_stream: { type: 'metrics', dataset: 'docker.event', }, metricsets: ['event'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.healthcheck', data_stream: { type: 'metrics', dataset: 'docker.healthcheck', }, metricsets: ['healthcheck'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.image', data_stream: { type: 'metrics', dataset: 'docker.image', }, metricsets: ['image'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.info', data_stream: { type: 'metrics', dataset: 'docker.info', }, metricsets: ['info'], hosts: ['unix:///var/run/docker.sock'], period: '10s', }, { id: 'docker/metrics-docker.memory', data_stream: { type: 'metrics', dataset: 'docker.memory', }, metricsets: ['memory'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, { id: 'docker/metrics-docker.network', data_stream: { type: 'metrics', dataset: 'docker.network', }, metricsets: ['network'], hosts: ['unix:///var/run/docker.sock'], period: '10s', 'labels.dedot': true, }, ], }, { id: 'filestream-docker', type: 'filestream', streams: [ { id: 'docker-container-logs-${docker.container.name}-${docker.container.id}', data_stream: { type: 'logs', dataset: 'docker.container_logs', elasticsearch: { dynamic_dataset: true, dynamic_namespace: true, }, }, paths: ['/var/lib/docker/containers/${docker.container.id}/*-json.log'], parsers: [ { container: { stream: 'all', format: 'docker', }, }, ], }, ], }, ], dataStreams: [ { type: 'metrics', dataset: 'docker.container', }, { type: 'logs', dataset: 'docker.container_logs', }, { type: 'metrics', dataset: 'docker.cpu', }, { type: 'metrics', dataset: 'docker.diskio', }, { type: 'metrics', dataset: 'docker.event', }, { type: 'metrics', dataset: 'docker.healthcheck', }, { type: 'metrics', dataset: 'docker.image', }, { type: 'metrics', dataset: 'docker.info', }, { type: 'metrics', dataset: 'docker.memory', }, { type: 'metrics', dataset: 'docker.network', }, ], kibanaAssets: [ { id: 'docker-AV4REOpp5NkDleZmzKkE', type: 'dashboard', }, { id: 'docker-Metrics-Docker', type: 'search', }, ], }, { installSource: 'registry', pkgName: 'system', pkgVersion: '1.59.0', title: 'System', inputs: [ { id: 'logfile-system', type: 'logfile', streams: [ { id: 'logfile-system.auth', data_stream: { type: 'logs', dataset: 'system.auth', }, ignore_older: '72h', paths: ['/var/log/auth.log*', '/var/log/secure*'], exclude_files: ['\\.gz$'], multiline: { pattern: '^\\s', match: 'after', }, tags: ['system-auth'], processors: [ { add_locale: null, }, { rename: { fields: [ { from: 'message', to: 'event.original', }, ], ignore_missing: true, fail_on_error: false, }, }, { syslog: { field: 'event.original', ignore_missing: true, ignore_failure: true, }, }, ], }, { id: 'logfile-system.syslog', data_stream: { type: 'logs', dataset: 'system.syslog', }, paths: ['/var/log/messages*', '/var/log/syslog*', '/var/log/system*'], exclude_files: ['\\.gz$'], multiline: { pattern: '^\\s', match: 'after', }, processors: [ { add_locale: null, }, ], tags: null, ignore_older: '72h', }, ], }, { id: 'winlog-system', type: 'winlog', streams: [ { id: 'winlog-system.application', data_stream: { type: 'logs', dataset: 'system.application', }, name: 'Application', condition: "${host.platform} == 'windows'", ignore_older: '72h', }, { id: 'winlog-system.security', data_stream: { type: 'logs', dataset: 'system.security', }, name: 'Security', condition: "${host.platform} == 'windows'", ignore_older: '72h', }, { id: 'winlog-system.system', data_stream: { type: 'logs', dataset: 'system.system', }, name: 'System', condition: "${host.platform} == 'windows'", ignore_older: '72h', }, ], }, { id: 'system/metrics-system', type: 'system/metrics', streams: [ { id: 'system/metrics-system.core', data_stream: { type: 'metrics', dataset: 'system.core', }, metricsets: ['core'], 'core.metrics': ['percentages'], }, { id: 'system/metrics-system.cpu', data_stream: { type: 'metrics', dataset: 'system.cpu', }, metricsets: ['cpu'], 'cpu.metrics': ['percentages', 'normalized_percentages'], period: '10s', }, { id: 'system/metrics-system.diskio', data_stream: { type: 'metrics', dataset: 'system.diskio', }, metricsets: ['diskio'], 'diskio.include_devices': null, period: '10s', }, { id: 'system/metrics-system.filesystem', data_stream: { type: 'metrics', dataset: 'system.filesystem', }, metricsets: ['filesystem'], period: '1m', processors: [ { 'drop_event.when.regexp': { 'system.filesystem.mount_point': '^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)', }, }, ], }, { id: 'system/metrics-system.fsstat', data_stream: { type: 'metrics', dataset: 'system.fsstat', }, metricsets: ['fsstat'], period: '1m', processors: [ { 'drop_event.when.regexp': { 'system.fsstat.mount_point': '^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)', }, }, ], }, { id: 'system/metrics-system.load', data_stream: { type: 'metrics', dataset: 'system.load', }, metricsets: ['load'], condition: "${host.platform} != 'windows'", period: '10s', }, { id: 'system/metrics-system.memory', data_stream: { type: 'metrics', dataset: 'system.memory', }, metricsets: ['memory'], period: '10s', }, { id: 'system/metrics-system.network', data_stream: { type: 'metrics', dataset: 'system.network', }, metricsets: ['network'], period: '10s', 'network.interfaces': null, }, { id: 'system/metrics-system.process', data_stream: { type: 'metrics', dataset: 'system.process', }, metricsets: ['process'], period: '10s', 'process.include_top_n.by_cpu': 5, 'process.include_top_n.by_memory': 5, 'process.cmdline.cache.enabled': true, 'process.cgroups.enabled': false, 'process.include_cpu_ticks': false, processes: ['.*'], }, { id: 'system/metrics-system.process.summary', data_stream: { type: 'metrics', dataset: 'system.process.summary', }, metricsets: ['process_summary'], period: '10s', }, { id: 'system/metrics-system.socket_summary', data_stream: { type: 'metrics', dataset: 'system.socket_summary', }, metricsets: ['socket_summary'], period: '10s', }, { id: 'system/metrics-system.uptime', data_stream: { type: 'metrics', dataset: 'system.uptime', }, metricsets: ['uptime'], period: '10s', }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'system.application', }, { type: 'logs', dataset: 'system.auth', }, { type: 'metrics', dataset: 'system.core', }, { type: 'metrics', dataset: 'system.cpu', }, { type: 'metrics', dataset: 'system.diskio', }, { type: 'metrics', dataset: 'system.filesystem', }, { type: 'metrics', dataset: 'system.fsstat', }, { type: 'metrics', dataset: 'system.load', }, { type: 'metrics', dataset: 'system.memory', }, { type: 'metrics', dataset: 'system.network', }, { type: 'metrics', dataset: 'system.process', }, { type: 'metrics', dataset: 'system.process.summary', }, { type: 'logs', dataset: 'system.security', }, { type: 'metrics', dataset: 'system.socket_summary', }, { type: 'logs', dataset: 'system.syslog', }, { type: 'logs', dataset: 'system.system', }, { type: 'metrics', dataset: 'system.uptime', }, ], kibanaAssets: [ { id: 'system-0d3f2380-fa78-11e6-ae9b-81e5311e8cab', type: 'dashboard', }, { id: 'system-277876d0-fa2c-11e6-bbd3-29c986c96e5a', type: 'dashboard', }, { id: 'system-5517a150-f9ce-11e6-8115-a7c18106d86a', type: 'dashboard', }, { id: 'system-71f720f0-ff18-11e9-8405-516218e3d268', type: 'dashboard', }, { id: 'system-79ffd6e0-faa0-11e6-947f-177f697178b8', type: 'dashboard', }, { id: 'system-Logs-syslog-dashboard', type: 'dashboard', }, { id: 'system-Metrics-system-overview', type: 'dashboard', }, { id: 'system-Windows-Dashboard', type: 'dashboard', }, { id: 'system-bae11b00-9bfc-11ea-87e4-49f31ec44891', type: 'dashboard', }, { id: 'system-bb858830-f412-11e9-8405-516218e3d268', type: 'dashboard', }, { id: 'system-d401ef40-a7d5-11e9-a422-d144027429da', type: 'dashboard', }, ], }, { installSource: 'custom', pkgName: 'var_log_lightdm', pkgVersion: '1.0.0', title: 'var_log_lightdm', inputs: [ { id: 'filestream-var_log_lightdm', type: 'filestream', streams: [ { id: 'filestream-var_log_lightdm', data_stream: { type: 'logs', dataset: 'var_log_lightdm', }, paths: ['/var/log/lightdm/*.log'], }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'var_log_lightdm', }, ], kibanaAssets: [], }, { installSource: 'custom', pkgName: 'var_log', pkgVersion: '1.0.0', title: 'var_log', inputs: [ { id: 'filestream-var_log', type: 'filestream', streams: [ { id: 'filestream-var_log', data_stream: { type: 'logs', dataset: 'var_log', }, paths: ['/var/log/*.log'], }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'var_log', }, ], kibanaAssets: [], }, { installSource: 'custom', pkgName: 'var_log_postgresql', pkgVersion: '1.0.0', title: 'var_log_postgresql', inputs: [ { id: 'filestream-var_log_postgresql', type: 'filestream', streams: [ { id: 'filestream-var_log_postgresql', data_stream: { type: 'logs', dataset: 'var_log_postgresql', }, paths: ['/var/log/postgresql/*.log'], }, ], }, ], dataStreams: [ { type: 'logs', dataset: 'var_log_postgresql', }, ], kibanaAssets: [], }, ], }, 'ea-download': { status: 'complete', message: '', }, 'ea-extract': { status: 'complete', message: '', }, 'ea-install': { status: 'complete', message: '', }, 'ea-config': { status: 'complete', message: '', }, 'ea-status': { status: 'complete', message: '', payload: { agentId: '2c22b333-224e-4c26-ae5a-0da2e2eb11ba', }, }, 'logs-ingest': { status: 'complete', }, }, }; const status = getOnboardingStatus(mockedProgressData); const installedIntegrations = getInstalledIntegrations(mockedProgressData); ``` </details> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When the auto-detect flow succeeds we currently show a list of all dashboards returned for each integration.
This includes dashboards which are not relevant (include no data) like the dashboards for Windows when installing System logs integration on a Linux host.
The list can also be quite long and overwhelming so we should see if we can pick (hard code) a dashboard as the main dashboard for each integration or if that's not possible whether we can determine that based off the title or another attribute in a more generic way.
The text was updated successfully, but these errors were encountered: