Reland "Reland "Add support for drawImageRect-based directly composit…

…ed images""

This is a reland of 0de4d37

relanding (again) now that perf regressions are understood.

There are cases where images are more frequently directly
composited which leads to more memory usage. However there are
Also some perf improvements, and no memory regressions on Android
so we're going to reland this as-is.

Original change's description:
> Reland "Add support for drawImageRect-based directly composited images"
>
> Reland now that M83 has forked. Also adjust DCHECK in
> picture_layer_impl.cc. crbug.com/1063358 points out that even if the
> aspect ratio matches, we can still end up with a scaled tile that does not
> exactly match the layer's bounds because of the raster scale adjustments
> that kick in inside RecalculateRasterScales when the desired scale is
> adjusted based on the ideal scale.
>
> > Original change's description:
> > > Add support for drawImageRect-based directly composited images
> > >
> > > Directly composited images is a feature that rasterizes images at their
> > > intrinsic size, when an image is the only thing drawn into a layer.
> > > However, this determination is today made prior to painting the image
> > > in blink, which can't be done with CompositeAfterPaint enabled.
> > >
> > > This CL adds support for detecting PictureLayers that qualify as directly
> > > composited images, by analyzing the paint ops of the display item list.
> > > If a single drawImageRect paint op is detected (with some facilities for
> > > detecting images for which orientation is respected), then we'll
> > > propagate the image's intrinsic size over to PictureLayerImpl on commit.
> > > PictureLayerImpl then uses that information to calculate a raster scale
> > > that results in the image being rasterized at its intrinsic size.
> > >
> > > There are a few additional changes needed outside that core calculation.
> > > If the aspect ratio of the layer bounds does not match the image's
> > > aspect ratio, we won't raster the image at the intrinsic size, but
> > > instead will choose the scale from one dimension. In these cases, we will
> > > choose to re-raster when the layer is resized until the layer bounds
> > > aspect ratio matches that of the image.
> > > If we do adjust the raster scale and add tilings to the active tree
> > > (as is seen in the default configuration for web_tests), we should drop
> > > the previous tilings (which are now non-ideal) since the max raster scale
> > > tile will be chosen when the SharedQuadState is created during
> > > AppendQuads.
> > > We also need to tell the LayerImpl if nearest neighbor should be used
> > > when the TileDrawQuad is scaled in the display compositor to preserve
> > > the same output of the original drawImageRect.
> > >
> > > Bug: 875110
> > >
> > > Change-Id: I2cda143f483c2aa30a6ae1fb850e1c7ef1227882
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2036542
> > > Reviewed-by: vmpstr <vmpstr@chromium.org>
> > > Reviewed-by: Philip Rogers <pdr@chromium.org>
> > > Commit-Queue: Daniel Libby <dlibby@microsoft.com>
> > > Cr-Commit-Position: refs/heads/master@{#751891}
>
> TBR=vmpstr@chromium.org,pdr@chromium.org,dlibby@microsoft.com
>
> # Not skipping CQ checks because original CL landed > 1 day ago.
>
> Bug: 875110
> Change-Id: I485e939ba08575a3e2af5a821558e54ce6132cc2
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2144398
> Reviewed-by: Daniel Libby <dlibby@microsoft.com>
> Reviewed-by: Philip Rogers <pdr@chromium.org>
> Reviewed-by: vmpstr <vmpstr@chromium.org>
> Commit-Queue: Daniel Libby <dlibby@microsoft.com>
> Cr-Commit-Position: refs/heads/master@{#758935}

TBR=vmpstr@chromium.org,pdr@chromium.org

Bug: 875110
Change-Id: Ide8ed7f61b045ad42862713597f2b59306e32d4f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2167175
Commit-Queue: Daniel Libby <dlibby@microsoft.com>
Reviewed-by: Daniel Libby <dlibby@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#762682}

cc/layers/picture_layer.cc

@@ -140,6 +140,26 @@ bool PictureLayer::Update() {
     picture_layer_inputs_.display_list =
         picture_layer_inputs_.client->PaintContentsToDisplayList(
             ContentLayerClient::PAINTING_BEHAVIOR_NORMAL);
+
+    // Clear out previous directly composited image state - if the layer
+    // qualifies we'll set up the state below.
+    picture_layer_inputs_.directly_composited_image_size = base::nullopt;
+    picture_layer_inputs_.nearest_neighbor = false;
+    if (QualifiesForDirectlyCompositedImage()) {
+      base::Optional<DisplayItemList::DirectlyCompositedImageResult> result =
+          picture_layer_inputs_.display_list->GetDirectlyCompositedImageResult(
+              bounds());
+      if (result) {
+        // Directly composited images are not guaranteed to fully cover every
+        // pixel in the layer due to ceiling when calculating the tile content
+        // rect from the layer bounds.
+        recording_source_->SetRequiresClear(true);
+        picture_layer_inputs_.directly_composited_image_size =
+            result->intrinsic_image_size;
+        picture_layer_inputs_.nearest_neighbor = result->nearest_neighbor;
+      }
+    }
+
     picture_layer_inputs_.painter_reported_memory_usage =
         picture_layer_inputs_.client->GetApproximateUnsharedMemoryUsage();
     recording_source_->UpdateDisplayItemList(
@@ -158,6 +178,21 @@ bool PictureLayer::Update() {
   return updated;
 }
 
+bool PictureLayer::QualifiesForDirectlyCompositedImage() const {
+  // Filters and backdrop-filters disqualify a layer from being a directly
+  // composited image.
+  // TODO(dlibby): crbug.com/875110 - remove this in an upcoming change and
+  // update baselines.
+  const EffectNode* effect_node =
+      layer_tree_host()->property_trees()->effect_tree.Node(
+          effect_tree_index());
+  if (!effect_node)
+    return true;
+
+  return (effect_node->filters.IsEmpty() &&
+          effect_node->backdrop_filters.IsEmpty());
+}
+
 sk_sp<SkPicture> PictureLayer::GetPicture() const {
   if (!DrawsContent() || bounds().IsEmpty())
     return nullptr;

cc/layers/picture_layer.h

@@ -93,6 +93,8 @@ class CC_EXPORT PictureLayer : public Layer {
 
   bool ShouldUseTransformedRasterization() const;
 
+  bool QualifiesForDirectlyCompositedImage() const;
+
   std::unique_ptr<RecordingSource> recording_source_;
   devtools_instrumentation::
       ScopedLayerObjectTracker instrumentation_object_tracker_;

cc/layers/picture_layer_impl.cc

@@ -99,6 +99,7 @@ PictureLayerImpl::PictureLayerImpl(LayerTreeImpl* tree_impl, int id)
       use_transformed_rasterization_(false),
       lcd_text_disallowed_reason_(LCDTextDisallowedReason::kNone),
       directly_composited_image_size_(base::nullopt),
+      directly_composited_image_raster_aspect_ratio_(0.f),
       tile_size_calculator_(this) {
   layer_tree_impl()->RegisterPictureLayerImpl(this);
 }
@@ -242,6 +243,41 @@ void PictureLayerImpl::AppendQuads(viz::RenderPass* render_pass,
       tilings_->num_tilings() ? MaximumTilingContentsScale() : 1.f;
   PopulateScaledSharedQuadState(shared_quad_state, max_contents_scale,
                                 contents_opaque());
+
+  if (directly_composited_image_size_) {
+    // Directly composited images should be clipped to the layer's content rect.
+    // When a PictureLayerTiling is created for a directly composited image, the
+    // layer bounds are multiplied by the raster scale in order to compute the
+    // tile size. If the aspect ratio of the layer doesn't match that of the
+    // image, it's possible that one of the dimensions of the resulting size
+    // (layer bounds * raster scale) is a fractional number, as raster scale
+    // does not scale x and y independently.
+    // When this happens, the ToEnclosingRect() operation in
+    // |PictureLayerTiling::EnclosingContentsRectFromLayer()| will
+    // create a tiling that, when scaled by |max_contents_scale| above, is
+    // larger than the layer bounds by a fraction of a pixel.
+    gfx::Rect clip_rect = draw_properties().drawable_content_rect;
+    if (shared_quad_state->is_clipped)
+      clip_rect.Intersect(shared_quad_state->clip_rect);
+
+    shared_quad_state->is_clipped = true;
+    shared_quad_state->clip_rect = clip_rect;
+
+#if DCHECK_IS_ON()
+    // Validate that the tile and bounds size are always within one pixel.
+    PictureLayerTiling* high_res =
+        tilings_->FindTilingWithResolution(HIGH_RESOLUTION);
+    if (high_res) {
+      const float epsilon = 1.f;
+      gfx::SizeF scaled_tiling_size(high_res->tiling_size());
+      scaled_tiling_size.Scale(1 / raster_contents_scale_);
+      DCHECK(std::abs(bounds().width() - scaled_tiling_size.width()) < epsilon);
+      DCHECK(std::abs(bounds().height() - scaled_tiling_size.height()) <
+             epsilon);
+    }
+#endif
+  }
+
   Occlusion scaled_occlusion =
       draw_properties()
           .occlusion_in_content_space.GetOcclusionWithGivenDrawTransform(
@@ -1035,17 +1071,50 @@ void PictureLayerImpl::SetDirectlyCompositedImageSize(
   if (directly_composited_image_size_ == size)
     return;
 
-  directly_composited_image_size_ = size;
+  directly_composited_image_size_ =
+      ShouldDirectlyCompositeImage(size) ? size : base::nullopt;
   NoteLayerPropertyChanged();
 }
 
-float PictureLayerImpl::GetDirectlyCompositedImageRasterScale() const {
-  float x = static_cast<float>(directly_composited_image_size_->width()) /
+bool PictureLayerImpl::ShouldDirectlyCompositeImage(
+    base::Optional<gfx::Size> size) const {
+  if (!size)
+    return false;
+
+  // If the results of scaling the bounds by the expected raster scale
+  // would end up with a content rect whose width/height are more than one
+  // pixel different from the layer bounds, don't directly composite the image
+  // to avoid incorrect rendering.
+  float raster_scale = GetDirectlyCompositedImageRasterScale(size.value());
+  gfx::SizeF layer_bounds(bounds());
+  gfx::RectF scaled_bounds_rect(layer_bounds);
+  scaled_bounds_rect.Scale(raster_scale);
+
+  // Take the scaled bounds, get the enclosing rect then scale it back down -
+  // this is the same set of operations that will happen when using the tiling
+  // at that raster scale.
+  gfx::RectF content_rect(gfx::ToEnclosingRect(scaled_bounds_rect));
+  content_rect.Scale(1 / raster_scale);
+
+  return std::abs(layer_bounds.width() - content_rect.width()) < 1.f &&
+         std::abs(layer_bounds.height() - content_rect.height()) < 1.f;
+}
+
+bool PictureLayerImpl::IsDirectlyCompositedImageRasteredAtIntrinsicRatio()
+    const {
+  DCHECK(directly_composited_image_size_.has_value());
+  return MathUtil::IsWithinEpsilon(
+      directly_composited_image_raster_aspect_ratio_,
+      static_cast<float>(directly_composited_image_size_->width()) /
+          directly_composited_image_size_->height());
+}
+
+float PictureLayerImpl::GetDirectlyCompositedImageRasterScale(
+    gfx::Size directly_composited_image_size) const {
+  float x = static_cast<float>(directly_composited_image_size.width()) /
             bounds().width();
-  float y = static_cast<float>(directly_composited_image_size_->height()) /
+  float y = static_cast<float>(directly_composited_image_size.height()) /
             bounds().height();
-  DCHECK_EQ(x, 1.f);
-  DCHECK_EQ(y, 1.f);
   return GetPreferredRasterScale(gfx::Vector2dF(x, y));
 }
 
@@ -1096,9 +1165,18 @@ void PictureLayerImpl::AddTilingsForRasterScale() {
   }
   high_res->set_resolution(HIGH_RESOLUTION);
 
-  if (layer_tree_impl()->IsPendingTree()) {
+  if (layer_tree_impl()->IsPendingTree() ||
+      (layer_tree_impl()->settings().commit_to_active_tree &&
+       directly_composited_image_size_.has_value())) {
     // On the pending tree, drop any tilings that are non-ideal since we don't
     // need them to activate anyway.
+
+    // For DirectlyCompositedImages, if we recomputed a new raster scale, we
+    // should drop the non-ideal ones if we're committing to the active tree.
+    // Otherwise a non-ideal scale that is _larger_ than the HIGH_RESOLUTION
+    // tile will be used as the coverage scale, and we'll produce a slightly
+    // different rendering. We don't drop the tilings on the active tree if
+    // we're not committing to the active tree to prevent checkerboarding.
     tilings_->RemoveNonIdealTilings();
   }
 
@@ -1107,7 +1185,21 @@ void PictureLayerImpl::AddTilingsForRasterScale() {
 
 bool PictureLayerImpl::ShouldAdjustRasterScale() const {
   if (directly_composited_image_size_) {
-    float desired_raster_scale = GetDirectlyCompositedImageRasterScale();
+    // Since the raster scale is only expressed as a single dimension,
+    // we may end up rasterizing a directly composited image at an aspect-ratio
+    // that doesn't match the intrinsic size. In these cases we will re-raster
+    // as the layers bounds change the aspect ratio, until we raster at the
+    // intrinsic size (or some multiple of it as enforced by the clamping and
+    // adjustments in |RecalculateRasterScales()|.
+    const bool layer_aspect_ratio_changed = !MathUtil::IsWithinEpsilon(
+        directly_composited_image_raster_aspect_ratio_,
+        static_cast<float>(bounds().width()) / bounds().height());
+    if (layer_aspect_ratio_changed &&
+        !IsDirectlyCompositedImageRasteredAtIntrinsicRatio())
+      return true;
+
+    float desired_raster_scale = GetDirectlyCompositedImageRasterScale(
+        directly_composited_image_size_.value());
     float max_scale = std::max(desired_raster_scale, MinimumContentsScale());
     if (raster_source_scale_ < std::min(ideal_source_scale_, max_scale))
       return true;
@@ -1196,11 +1288,12 @@ void PictureLayerImpl::AddLowResolutionTilingIfNeeded() {
 
 void PictureLayerImpl::RecalculateRasterScales() {
   if (directly_composited_image_size_) {
+    float desired_raster_scale = GetDirectlyCompositedImageRasterScale(
+        directly_composited_image_size_.value());
     if (!raster_source_scale_)
-      raster_source_scale_ = GetDirectlyCompositedImageRasterScale();
+      raster_source_scale_ = desired_raster_scale;
 
     float min_scale = MinimumContentsScale();
-    float desired_raster_scale = GetDirectlyCompositedImageRasterScale();
     float max_scale = std::max(desired_raster_scale, MinimumContentsScale());
     float clamped_ideal_source_scale =
         base::ClampToRange(ideal_source_scale_, min_scale, max_scale);
@@ -1217,6 +1310,9 @@ void PictureLayerImpl::RecalculateRasterScales() {
     raster_device_scale_ = 1.f;
     raster_contents_scale_ = raster_source_scale_;
     low_res_raster_contents_scale_ = raster_contents_scale_;
+
+    directly_composited_image_raster_aspect_ratio_ =
+        static_cast<float>(bounds().width()) / bounds().height();
     return;
   }
 
@@ -1421,7 +1517,13 @@ float PictureLayerImpl::MinimumContentsScale() const {
   if (!min_dimension)
     return setting_min;
 
-  return std::max(1.f / min_dimension, setting_min);
+  // Directly composited images may result in contents scales that are
+  // less than the configured setting. We allow this lower scale so that we
+  // can raster at the intrinsic image size.
+  const float inverse_min_dimension = 1.f / min_dimension;
+  return (directly_composited_image_size_.has_value())
+             ? inverse_min_dimension
+             : std::max(inverse_min_dimension, setting_min);
 }
 
 float PictureLayerImpl::MaximumContentsScale() const {

cc/layers/picture_layer_impl.h

@@ -182,7 +182,9 @@ class CC_EXPORT PictureLayerImpl
   float MaximumContentsScale() const;
   void UpdateViewportRectForTilePriorityInContentSpace();
   PictureLayerImpl* GetRecycledTwinLayer() const;
-  float GetDirectlyCompositedImageRasterScale() const;
+  bool ShouldDirectlyCompositeImage(base::Optional<gfx::Size> size) const;
+  bool IsDirectlyCompositedImageRasteredAtIntrinsicRatio() const;
+  float GetDirectlyCompositedImageRasterScale(gfx::Size) const;
 
   void SanityCheckTilingState() const;
 
@@ -242,8 +244,17 @@ class CC_EXPORT PictureLayerImpl
 
   LCDTextDisallowedReason lcd_text_disallowed_reason_;
 
+  // The intrinsic size of the directly composited image. A directly composited
+  // image is an image which is the only thing drawn into a layer. In these
+  // cases we attempt to raster the image at its intrinsic size.
   base::Optional<gfx::Size> directly_composited_image_size_;
 
+  // If |directly_composited_image_size_| is set, this is the aspect ratio of
+  // the *layer* (and thus the rasterized contents) when the
+  // raster_contents_scale_ was last calculated. See comments in
+  // |ShouldAdjustRasterScale| for an explanation of how this is used.
+  float directly_composited_image_raster_aspect_ratio_;
+
   // Use this instead of |visible_layer_rect()| for tiling calculations. This
   // takes external viewport and transform for tile priority into account.
   gfx::Rect viewport_rect_for_tile_priority_in_content_space_;