Issue seems to be around new instances spawned to handle load having to create their own in memory page cache. If you are unfortunate enough to get a new instance to service your request you have to wait whist it builds and caches pages to answer you.

Suggested investigation of use of NDB Cloud Datastore to create a single page cache to be shared by all instances.