Perfetto plugin - Update Perfetto plugin docs

PerfettoCds/Architecture.md

@@ -0,0 +1,28 @@
+# Perfetto Plugin architecture
+
+The WPA Perfetto plugin follows the standard Microsoft Performance Toolkit SDK [architecture](https://github.com/microsoft/microsoft-performance-toolkit-sdk/tree/main/documentation/Architecture)
+
+![PerfettoPluginArchitecture](../Images/PerfettoPluginArchitecture.png)
+
+Here is a closer look at the bottom half of that diagram, the data cooking pipeline:
+
+![PerfettoCookerPipeline](../Images/PerfettoCookerPipeline.png)
+
+PerfettoSourceParser will start the trace_processor_shell.exe process and load the Perfetto trace file. Trace_processor_shell will process the trace file and will load all the trace data into SQL tables (such as slice, process, thread). SQL Queries will be made over HTTP/RPC with Protobuf objects. One query will be made per SQL table. Each query will produce events of the corresponding type (PerfettoSliceEvent). 
+
+Source cookers are registered to receive their own specific events and will process and store them as soon as they are created. Composite cookers are registered to receive events from a specific set of source cookers and they will query those source cookers once they all have finished their processing. Composite cookers will create the final output events, which can then be used by the client (such as WPA).
+
+For example, for the slice table, the process goes like this:
+
+1. Perform a SQL query into the Perfetto trace through trace_processor_shell.exe ("select * from slice"). This returns a QueryResult protobuf object. The QueryResult object is parsed and objects of type PerfettoSliceEvent are returned. One event for each row in the SQL table.
+2. PerfettoSliceCooker will process/store all the PerfettoSliceEvents as they are created.
+3. Once all the other tables have also finished, PerfettoGenericEventCooker will gather all the events from each cooker and do a join on them to create complete PerfettoGenericEvent objects.
+
+## Rationale
+
+The reason for performing individual queries of each table through trace_processor_shell.exe is due to performance. In order to display useful output data like GenericEvent charts or CPU scheduler view, we need data from multiple tables. Performing those multi-table queries over trace_processor_shell.exe proved to be slower than performing those multi-table queries in C# with LINQ.
+
+## Trace Processor Shell
+Trace_processor_shell.exe is built from the [Perfetto GitHub repo](https://github.com/google/perfetto). To build this for Windows, follow the instructions from their [documentation](https://perfetto.dev/docs/contributing/build-instructions#building-on-windows). Prebuilt binaries can also be found in their [releases](https://github.com/google/perfetto/releases/).
+
+

PerfettoCds/ReadMe.md

@@ -2,24 +2,101 @@
 
 The PerfettoCds project adds a pipeline for displaying data from Perfetto traces using Windows Performance Toolkit SDK. The PerfettoProcessor project contains the logic for processing Perfetto traces.
 
-Data is gathered from the Perfetto trace through trace_processor_shell.exe. Trace_processor_shell.exe opens an interface that allows for making SQL queries over HTTP/RPC to localhost. All the trace data is retrieved with SQL queries. Data is serialized over the HTTP interface with Protobuf objects. The original TraceProcessor protobuf object (trace_processor.proto) and the C# conversions are included in the project.
+Data is gathered from the Perfetto trace through trace_processor_shell.exe. Trace_processor_shell.exe opens an interface that allows for making SQL queries over HTTP/RPC to localhost. All the trace data is retrieved through SQL queries and is processed through the data cooker pipeline. 
 
-PerfettoSourceParser will start the trace_processor_shell.exe process. SQL Queries will be made over HTTP and the protobuf output will be converted to objects of type PerfettoSqlEvent. One query will be made per SQL table. Each query will produce events of the same type and will be processed by their own individual source cooker. For a complete generic event, we need data from 5 tables: slice, args, thread_track, thread, and process. A composite cooker will then take all the data gathered from each individual source cooker and join them to create the final generic event.
+See the [Architecture](Architecture.md) document for more information.
 
-For example, for the slice table, the process goes like this:
+## Trace config files
+See Perfetto documentation for more information on how to create [config files](https://perfetto.dev/docs/concepts/config) and [collect traces](https://perfetto.dev/docs/quickstart/android-tracing). 
 
-1. Perform a SQL query into the Perfetto trace through trace_processor_shell.exe ("select * from slice"). This returns a QueryResult protobuf object. The QueryResult object is parsed and objects of type PerfettoSliceEvent are returned. One event for each row in the SQL table.
-2. PerfettoSliceCooker will process/store all the PerfettoSliceEvents.
-3. Once all the other tables have also finished, PerfettoGenericEventCooker will gather all the events from each cooker and do a join on them to create complete PerfettoGenericEvent objects.
-4. Create WPA table of PerfettoGenericEvents
+See the ["Record new trace"](https://ui.perfetto.dev/#!/record) menu for a helpful guide on creating custom config files.
 
-## Trace Processor Shell
-Trace_processor_shell.exe is built from the [Perfetto GitHub repo](https://github.com/google/perfetto). To build this for Windows, follow the instructions from their [documentation](https://perfetto.dev/docs/contributing/build-instructions#building-on-windows) 
+### Some config options for some event types we support
+* Generic Events
+  * `data_sources: {
+  config {
+    name: "My.Trace.Event"
+  }
+}`
+* CPU Counters (coarse)
+  * `data_sources: {
+    config {
+        name: "linux.sys_stats"
+        sys_stats_config {
+            stat_period_ms: 1000
+            stat_counters: STAT_CPU_TIMES
+            stat_counters: STAT_FORK_COUNT
+        }
+    }
+}`
+* CPU Frequency Scaling
+  * `data_sources: {
+    config {
+        name: "linux.ftrace"
+        ftrace_config {
+            ftrace_events: "power/cpu_frequency"
+            ftrace_events: "power/cpu_idle"
+            ftrace_events: "power/suspend_resume"
+        }
+    }
+}`
+* CPU Scheduler
+  * `data_sources: {
+    config {
+        name: "linux.process_stats"
+        target_buffer: 1
+        process_stats_config {
+            scan_all_processes_on_start: true
+        }
+    }
+}
+data_sources: {
+    config {
+        name: "linux.ftrace"
+        ftrace_config {
+            ftrace_events: "sched/sched_switch"
+            ftrace_events: "power/suspend_resume"
+            ftrace_events: "sched/sched_wakeup"
+            ftrace_events: "sched/sched_wakeup_new"
+            ftrace_events: "sched/sched_waking"
+            ftrace_events: "sched/sched_process_exit"
+            ftrace_events: "sched/sched_process_free"
+            ftrace_events: "task/task_newtask"
+            ftrace_events: "task/task_rename"
+        }
+    }
+}`
+* Perfetto Process Memory
+  * `data_sources: {
+    config {
+        name: "linux.process_stats"
+        target_buffer: 1
+        process_stats_config {
+            proc_stats_poll_ms: 1000
+        }
+    }
+}`
+* Perfetto System Memory
+  * `data_sources: {
+    config {
+        name: "linux.sys_stats"
+        sys_stats_config {
+            meminfo_period_ms: 1000
+            meminfo_counters: MEMINFO_ACTIVE
+            meminfo_counters: MEMINFO_ACTIVE_FILE
+            meminfo_counters: MEMINFO_CACHED
+        }
+    }
+}`
 
-## Provider-GUID Mappings
+
+## Additional Features
+
+### Provider-GUID Mappings
 Perfetto does not surface the name of an event's provider. As a workaround, the Perfetto plugin has support to load a ProviderGuid XML mapping file that maps GUIDs to Provider Names. Using this mapping, the plugin will search each GenericEvent's debug annotations for a specific key that holds a GUID value, and it will convert that GUID to the ProviderName in the mapping. The GenericEvent table will then display an extra column with that Provider name.
 
-To use:
+#### To use:
+
 1. Your GenericEvents should be instrumented to output debug annotations where the argument key is the string "ProviderGuid" and the argument value is the provider GUID string.
 2. Add an XML file with the exact name "ProviderMapping.xml" to the same directory that contains the Perfetto plugin binaries. That directory is the one containing "PerfettoCds.dll". The XML schema should look like the following. The "DebugAnnotationKey" attribute is optional. That specifies which debug annotation key to look for. It defaults to "ProviderGuid". Add an EventProvider node for each Provider you want mapped.