Yeah, I think for intra prediction it will be necessary to create a map of the dependencies of all the blocks that need predicting for each frame, and to schedule the GPU tasks accordingly. Sticking with P-frames for now is probably a good plan.

Running the chroma and luma tasks in parallel can be done for all stages of the pipeline, although presumably only the loop filter can be run in U+V lock-step.

Yes it should be possible, but then we would have to guess the values (each between 0 and 255) of :
– at best, one edge, 4 pixels
– at worst, three edges and a corner, 13 pixels

And that is definitly hard to do.

Very useful comment indeed thank you, I hope you don’t mind if I answer here.
I took the same the same approach than you, start at the top and replace CPU operations by GPU ones. Color space convertions are done and I am working on the loop filter, and I’ll finish by inter-predictions. That should be enough to beat a purely CPU based decoder performance wise.
For the intra-predictions it might be easier to do the ones in the middle of inter-predicted pictures, the dependencies on adjacent blocks could be easily solved if these blocks are inter-predicted and thus already available, and otherwise it should be at least easier to find new and independant entry points to start more intra-prediction processes in parallel.
For the loop filter your idea is great I was looking to do something similar. Plus, for the normal loop filter we can do the 3 planes at the same time, so with a 4:2:0 subsampling we can complete both chroma planes in less than half the amount of time required to do the luma plane.

Thanks !
For blockcoeff.png, I actually made this one with libreoffice calc 😉 It’s just a 3d graph using 4 series of data. I emailed you my template, hope that helps.