Initial Perfetto plugin checkin#21
Conversation
|
Also had MIT license header to each file (see other source files in repo) |
|
Great summary in the PR description of what this does and how it works. Add this info to a readme.md or similar |
| join thread in threadData on threadTrack.Utid equals thread.Utid | ||
| join process in processData on thread.Upid equals process.Upid | ||
| select new { slice, args, threadTrack, thread, process }; | ||
|
|
There was a problem hiding this comment.
What's your thoughts on just having this cooker receive the events instead of querying all the other cookers?
There was a problem hiding this comment.
I see why this approach was taken, each event type requires a specific SQL query.
Is there a "generic" query that could be performed across all data in Perfetto?
Wondering if a general query should be done at the trace processor level, then each cooker would compose the events
There was a problem hiding this comment.
@trgibeau and I were talking. His suggestion in English (in my words) is to do a basic trace load/query on trace load for data that every table needs. Then all the tables will have on-demand sql queries as they load. In this manner the trace loads faster, and some data in the trace is available to view. Only visible mini-graphs would attempt to load in this manner. Or if someone used a profile with say 2 tables only those would load not all the other tables. This can be saved for another PR
There was a problem hiding this comment.
I'll explain my cooker architecture and rationale a little.
In order to get all the data needed for a complete "Generic Event", I need to join data across multiple SQL tables (5 tables for GenericEvents). This join can happen in 2 places: across the trace_processor_shell interface or in the cookers. These SQL tables also contain data that is needed for other WPA tables that I will be adding in the future, like CPU scheduling, memory usage, etc.
So I was trying to follow the cooker principle that data should be processed as few times as possible and can be re-used by other composite cookers. Now if any WPA table needs data that comes from a SQL "slices" table, they can get that data from the Slice source cooker and do the join in LINQ. As opposed to making multiple join queries across the trace_processor_shell interface, where we have to query from the SQL slice table multiple times.
That was my thinking anyway and I haven't tested at scale yet so I don't know how this holds up.
Once I start adding more tables there might be some bad delay and we might need to add some intelligence to which tables I query from and when. I'll schedule a meeting with the 3 of you to discuss the best cooker architecture once I start adding more WPA tables.
| { | ||
| // Use DateTime.Now as the wall clock time. This doesn't matter for displaying events on a relative timescale | ||
| // TODO Actual wall clock time needs to be gathered from SQL somehow | ||
| this.dataSourceInfo = new DataSourceInfo(traceStartTime.Value.ToNanoseconds, traceEndTime.Value.ToNanoseconds, DateTime.Now.ToUniversalTime()); |
There was a problem hiding this comment.
The absolute time is incorrect for the trace and just is whenever the trace was opened. Replace "DateTime.Now.ToUniversalTime()" with the actual trace start time and verify event absolute timestamps in UTC match what other Perfetto tools provide
There was a problem hiding this comment.
I still haven't figured out how to get the absolute time the trace was taken from Perfetto. It is also unclear what the relative timestamps are relative to. I have a future task to figure out the timings. But for now the relative timestamps align correctly within the same trace.
This PR adds a plugin for displaying generic events from Perfetto traces. Added a new "PerfettoCds" project to the solution. Keeping this in a separate "perfetto" feature branch while I work on it.
Data is gathered from the Perfetto trace through trace_processor_shell.exe. Trace_processor_shell.exe opens an interface that allows for making SQL queries over HTTP/RPC to localhost. All the trace data is retrieved with SQL queries. Data is serialized over the HTTP interface with Protobuf objects. The original TraceProcessor protobuf object (trace_processor.proto) and the C# conversions are included in the project.
PerfettoSourceParser will start the trace_processor_shell.exe process. SQL Queries will be made over HTTP and the protobuf output will be converted to objects of type PerfettoSqlEvent. One query will be made per SQL table. Each query will produce events of the same type and will be processed by their own individual source cooker. For a complete generic event, we need data from 5 tables: slice, args, thread_track, thread, and process. A composite cooker will then take all the data gathered from each individual source cooker and join them to create the final generic event.
For example, for the slice table, the process goes like this: