Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Commit 49643c4

Browse files
chinmaygardechinmaygarde
committed
[Impeller] Wire up hardware buffer backed swapchains on Android.
This wires up Android Hardware Buffer backed swapchains on platform that support it (Android >= 29) with a KHR swapchain fallback (which can be removed later to save on some binary size if needed). Some salient features of these swapchains and their differences with the KHR variant: * All swapchain images are guaranteed to R8G8B8A8. This could potentially allow for earlier creation of the context and the PSO libraries. * All swapchain allocations are lazy. This should greatly reduce the time it takes to create and resize a swapchain. However, swapchain image acquisition may take longer for the first few frame where there is no pre-pooled image available. Resizes should be similarly faster since the swapchain images for the intermediate sizes are never created. * Swapchain transients allocations (the attachments for the root render target) are also lazily allocated. * Swapchain images are pool and reused. The size of the pool is user specified (currently 2). If an image in the pool ages past a user supplied value (currently 1 second), it is collected as well. Applications that don't render frames for a long period of time should see less memory use because of swapchain image allocations. * The present mode of AHB swapchains behave similar to KHR swapchains but with VK_PRESENT_MODE_MAILBOX_KHR. In cases where there is no application managed frame pipelining, this might cause images to never be presented if a newer image is available. This wasted work can only be avoided by application provided pipelining. * There are no client side waits during image presentation. Instead, a new type of fence is wired up that exports its state as a sync file descriptor. The fence signal operation is enqueued on the client side and the buffer is set on the surface control. The presentation engine then performs the wait. * On Qualcomm devices, Chromium seems to be setting vendor specified flags for opting the hardware buffers into using UBWC. AFAICT, this is similar to AFBC (and NOT AFRC) on ARM Mali. This has not been wired up since I don't have a Qualcomm device at the moment and cant verify bandwidth use using GPU counters. I would also like to verify that UBWC is safe to use to images that can be used as input attachments.
1 parent 424a963 commit 49643c4

31 files changed

+1181
-79
lines changed

‎ci/licenses_golden/licenses_flutter‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40731,6 +40731,15 @@ ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/shared_object_vk.cc +
4073140731
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/shared_object_vk.h + ../../../flutter/LICENSE
4073240732
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/surface_context_vk.cc + ../../../flutter/LICENSE
4073340733
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/surface_context_vk.h + ../../../flutter/LICENSE
40734+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_formats.h + ../../../flutter/LICENSE
40735+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_impl_vk.cc + ../../../flutter/LICENSE
40736+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_impl_vk.h + ../../../flutter/LICENSE
40737+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_vk.cc + ../../../flutter/LICENSE
40738+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_vk.h + ../../../flutter/LICENSE
40739+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_texture_pool_vk.cc + ../../../flutter/LICENSE
40740+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_texture_pool_vk.h + ../../../flutter/LICENSE
40741+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/external_fence_vk.cc + ../../../flutter/LICENSE
40742+
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/external_fence_vk.h + ../../../flutter/LICENSE
4073440743
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_image_vk.cc + ../../../flutter/LICENSE
4073540744
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_image_vk.h + ../../../flutter/LICENSE
4073640745
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_impl_vk.cc + ../../../flutter/LICENSE
@@ -43609,6 +43618,15 @@ FILE: ../../../flutter/impeller/renderer/backend/vulkan/shared_object_vk.cc
4360943618
FILE: ../../../flutter/impeller/renderer/backend/vulkan/shared_object_vk.h
4361043619
FILE: ../../../flutter/impeller/renderer/backend/vulkan/surface_context_vk.cc
4361143620
FILE: ../../../flutter/impeller/renderer/backend/vulkan/surface_context_vk.h
43621+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_formats.h
43622+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_impl_vk.cc
43623+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_impl_vk.h
43624+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_vk.cc
43625+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_swapchain_vk.h
43626+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_texture_pool_vk.cc
43627+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/ahb_texture_pool_vk.h
43628+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/external_fence_vk.cc
43629+
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/ahb/external_fence_vk.h
4361243630
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_image_vk.cc
4361343631
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_image_vk.h
4361443632
FILE: ../../../flutter/impeller/renderer/backend/vulkan/swapchain/khr/khr_swapchain_impl_vk.cc

‎fml/closure.h‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,7 @@ class ScopedCleanupClosure final {
4545
explicit ScopedCleanupClosure(const fml::closure& closure)
4646
: closure_(closure) {}
4747

48-
~ScopedCleanupClosure() {
49-
if (closure_) {
50-
closure_();
51-
}
52-
}
48+
~ScopedCleanupClosure() { Reset(); }
5349

5450
fml::closure SetClosure(const fml::closure& closure) {
5551
auto old_closure = closure_;
@@ -63,6 +59,13 @@ class ScopedCleanupClosure final {
6359
return closure;
6460
}
6561

62+
void Reset() {
63+
if (closure_) {
64+
closure_();
65+
closure_ = nullptr;
66+
}
67+
}
68+
6669
private:
6770
fml::closure closure_;
6871

‎impeller/renderer/backend/vulkan/BUILD.gn‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,15 @@ impeller_component("vulkan") {
133133
sources += [
134134
"android/ahb_texture_source_vk.cc",
135135
"android/ahb_texture_source_vk.h",
136+
"swapchain/ahb/ahb_formats.h",
137+
"swapchain/ahb/ahb_swapchain_impl_vk.cc",
138+
"swapchain/ahb/ahb_swapchain_impl_vk.h",
139+
"swapchain/ahb/ahb_swapchain_vk.cc",
140+
"swapchain/ahb/ahb_swapchain_vk.h",
141+
"swapchain/ahb/ahb_texture_pool_vk.cc",
142+
"swapchain/ahb/ahb_texture_pool_vk.h",
143+
"swapchain/ahb/external_fence_vk.cc",
144+
"swapchain/ahb/external_fence_vk.h",
136145
]
137146
}
138147

@@ -144,4 +153,8 @@ impeller_component("vulkan") {
144153
"//flutter/third_party/vulkan-deps/vulkan-headers/src:vulkan_headers",
145154
"//flutter/third_party/vulkan_memory_allocator",
146155
]
156+
157+
if (is_android) {
158+
public_deps += [ "../../../toolkit/android" ]
159+
}
147160
}

‎impeller/renderer/backend/vulkan/android/ahb_texture_source_vk.cc‎

Lines changed: 32 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,11 @@ static vk::UniqueImage CreateVKImageWrapperForAndroidHarwareBuffer(
4040
if (ahb_desc.usage & AHARDWAREBUFFER_USAGE_GPU_FRAMEBUFFER) {
4141
image_usage_flags |= vk::ImageUsageFlagBits::eColorAttachment;
4242
}
43+
if (ahb_desc.usage & AHARDWAREBUFFER_USAGE_COMPOSER_OVERLAY) {
44+
image_usage_flags |= vk::ImageUsageFlagBits::eColorAttachment;
45+
image_usage_flags |= vk::ImageUsageFlagBits::eInputAttachment;
46+
image_usage_flags |= vk::ImageUsageFlagBits::eTransferDst;
47+
}
4348

4449
vk::ImageCreateFlags image_create_flags;
4550
if (ahb_desc.usage & AHARDWAREBUFFER_USAGE_PROTECTED_CONTENT) {
@@ -280,17 +285,18 @@ static TextureDescriptor ToTextureDescriptor(
280285
}
281286

282287
AHBTextureSourceVK::AHBTextureSourceVK(
283-
const std::shared_ptr<ContextVK>& context,
288+
const std::shared_ptr<Context>& p_context,
284289
struct AHardwareBuffer* ahb,
285290
const AHardwareBuffer_Desc& ahb_desc)
286291
: TextureSourceVK(ToTextureDescriptor(ahb_desc)) {
287-
if (!context) {
288-
VALIDATION_LOG << "Invalid context.";
292+
if (!p_context) {
289293
return;
290294
}
291295

292-
const auto& device = context->GetDevice();
293-
const auto& physical_device = context->GetPhysicalDevice();
296+
const auto& context = ContextVK::Cast(*p_context);
297+
298+
const auto& device = context.GetDevice();
299+
const auto& physical_device = context.GetPhysicalDevice();
294300

295301
AHBProperties ahb_props;
296302

@@ -327,7 +333,7 @@ AHBTextureSourceVK::AHBTextureSourceVK(
327333
}
328334

329335
// Figure out how to perform YUV conversions.
330-
auto yuv_conversion = CreateYUVConversion(*context, ahb_props);
336+
auto yuv_conversion = CreateYUVConversion(context, ahb_props);
331337
if (!yuv_conversion || !yuv_conversion->IsValid()) {
332338
return;
333339
}
@@ -350,15 +356,26 @@ AHBTextureSourceVK::AHBTextureSourceVK(
350356
image_view_ = std::move(image_view);
351357

352358
#ifdef IMPELLER_DEBUG
353-
context->SetDebugName(device_memory_.get(), "AHB Device Memory");
354-
context->SetDebugName(image_.get(), "AHB Image");
355-
context->SetDebugName(yuv_conversion_->GetConversion(), "AHB YUV Conversion");
356-
context->SetDebugName(image_view_.get(), "AHB ImageView");
359+
context.SetDebugName(device_memory_.get(), "AHB Device Memory");
360+
context.SetDebugName(image_.get(), "AHB Image");
361+
context.SetDebugName(yuv_conversion_->GetConversion(), "AHB YUV Conversion");
362+
context.SetDebugName(image_view_.get(), "AHB ImageView");
357363
#endif // IMPELLER_DEBUG
358364

359365
is_valid_ = true;
360366
}
361367

368+
AHBTextureSourceVK::AHBTextureSourceVK(
369+
const std::shared_ptr<Context>& context,
370+
std::unique_ptr<android::HardwareBuffer> backing_store,
371+
bool is_swapchain_image)
372+
: AHBTextureSourceVK(context,
373+
backing_store->GetHandle(),
374+
backing_store->GetAndroidDescriptor()) {
375+
backing_store_ = std::move(backing_store);
376+
is_swapchain_image_ = is_swapchain_image;
377+
}
378+
362379
// |TextureSourceVK|
363380
AHBTextureSourceVK::~AHBTextureSourceVK() = default;
364381

@@ -383,12 +400,16 @@ vk::ImageView AHBTextureSourceVK::GetRenderTargetView() const {
383400

384401
// |TextureSourceVK|
385402
bool AHBTextureSourceVK::IsSwapchainImage() const {
386-
return false;
403+
return is_swapchain_image_;
387404
}
388405

389406
// |TextureSourceVK|
390407
std::shared_ptr<YUVConversionVK> AHBTextureSourceVK::GetYUVConversion() const {
391408
return needs_yuv_conversion_ ? yuv_conversion_ : nullptr;
392409
}
393410

411+
const android::HardwareBuffer* AHBTextureSourceVK::GetBackingStore() const {
412+
return backing_store_.get();
413+
}
414+
394415
} // namespace impeller

‎impeller/renderer/backend/vulkan/android/ahb_texture_source_vk.h‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
#include "impeller/renderer/backend/vulkan/texture_source_vk.h"
1212
#include "impeller/renderer/backend/vulkan/vk.h"
1313
#include "impeller/renderer/backend/vulkan/yuv_conversion_vk.h"
14+
#include "impeller/toolkit/android/hardware_buffer.h"
1415

1516
#include <android/hardware_buffer.h>
1617
#include <android/hardware_buffer_jni.h>
@@ -33,10 +34,14 @@ class ContextVK;
3334
///
3435
class AHBTextureSourceVK final : public TextureSourceVK {
3536
public:
36-
AHBTextureSourceVK(const std::shared_ptr<ContextVK>& context,
37+
AHBTextureSourceVK(const std::shared_ptr<Context>& context,
3738
struct AHardwareBuffer* hardware_buffer,
3839
const AHardwareBuffer_Desc& hardware_buffer_desc);
3940

41+
AHBTextureSourceVK(const std::shared_ptr<Context>& context,
42+
std::unique_ptr<android::HardwareBuffer> backing_store,
43+
bool is_swapchain_image);
44+
4045
// |TextureSourceVK|
4146
~AHBTextureSourceVK() override;
4247

@@ -57,12 +62,16 @@ class AHBTextureSourceVK final : public TextureSourceVK {
5762
// |TextureSourceVK|
5863
std::shared_ptr<YUVConversionVK> GetYUVConversion() const override;
5964

65+
const android::HardwareBuffer* GetBackingStore() const;
66+
6067
private:
68+
std::unique_ptr<android::HardwareBuffer> backing_store_;
6169
vk::UniqueDeviceMemory device_memory_ = {};
6270
vk::UniqueImage image_ = {};
6371
vk::UniqueImageView image_view_ = {};
6472
std::shared_ptr<YUVConversionVK> yuv_conversion_ = {};
6573
bool needs_yuv_conversion_ = false;
74+
bool is_swapchain_image_ = false;
6675
bool is_valid_ = false;
6776

6877
AHBTextureSourceVK(const AHBTextureSourceVK&) = delete;

‎impeller/renderer/backend/vulkan/capabilities_vk.cc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,10 @@ static const char* GetExtensionName(RequiredAndroidDeviceExtensionVK ext) {
179179
return VK_EXT_QUEUE_FAMILY_FOREIGN_EXTENSION_NAME;
180180
case RequiredAndroidDeviceExtensionVK::kKHRDedicatedAllocation:
181181
return VK_KHR_DEDICATED_ALLOCATION_EXTENSION_NAME;
182+
case RequiredAndroidDeviceExtensionVK::kKHRExternalFenceFd:
183+
return VK_KHR_EXTERNAL_FENCE_FD_EXTENSION_NAME;
184+
case RequiredAndroidDeviceExtensionVK::kKHRExternalFence:
185+
return VK_KHR_EXTERNAL_FENCE_EXTENSION_NAME;
182186
case RequiredAndroidDeviceExtensionVK::kLast:
183187
return "Unknown";
184188
}

‎impeller/renderer/backend/vulkan/capabilities_vk.h‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,20 @@ enum class RequiredAndroidDeviceExtensionVK : uint32_t {
7878
///
7979
kKHRDedicatedAllocation,
8080

81+
//----------------------------------------------------------------------------
82+
/// For exporting file descriptors from fences to interact with platform APIs.
83+
///
84+
/// https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_external_fence_fd.html
85+
///
86+
kKHRExternalFenceFd,
87+
88+
//----------------------------------------------------------------------------
89+
/// Dependency of kKHRExternalFenceFd.
90+
///
91+
/// https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_external_fence.html
92+
///
93+
kKHRExternalFence,
94+
8195
kLast,
8296
};
8397

‎impeller/renderer/backend/vulkan/driver_info_vk_unittests.cc‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ INSTANTIATE_VULKAN_PLAYGROUND_SUITE(DriverInfoVKTest);
1414
TEST_P(DriverInfoVKTest, CanQueryDriverInfo) {
1515
ASSERT_TRUE(GetContext());
1616
const auto& driver_info =
17-
SurfaceContextVK::Cast(*GetContext()).GetParent().GetDriverInfo();
17+
SurfaceContextVK::Cast(*GetContext()).GetParent()->GetDriverInfo();
1818
ASSERT_NE(driver_info, nullptr);
1919
// 1.1 is the base Impeller version. The driver can't be lower than that.
2020
ASSERT_TRUE(driver_info->GetAPIVersion().IsAtLeast(Version{1, 1, 0}));
@@ -26,7 +26,7 @@ TEST_P(DriverInfoVKTest, CanQueryDriverInfo) {
2626
TEST_P(DriverInfoVKTest, CanDumpToLog) {
2727
ASSERT_TRUE(GetContext());
2828
const auto& driver_info =
29-
SurfaceContextVK::Cast(*GetContext()).GetParent().GetDriverInfo();
29+
SurfaceContextVK::Cast(*GetContext()).GetParent()->GetDriverInfo();
3030
ASSERT_NE(driver_info, nullptr);
3131
fml::testing::LogCapture log;
3232
driver_info->DumpToLog();

‎impeller/renderer/backend/vulkan/queue_vk.cc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,11 @@ vk::Result QueueVK::Submit(const vk::SubmitInfo& submit_info,
2323
return queue_.submit(submit_info, fence);
2424
}
2525

26+
vk::Result QueueVK::Submit(const vk::Fence& fence) const {
27+
Lock lock(queue_mutex_);
28+
return queue_.submit({}, fence);
29+
}
30+
2631
vk::Result QueueVK::Present(const vk::PresentInfoKHR& present_info) {
2732
Lock lock(queue_mutex_);
2833
return queue_.presentKHR(present_info);

‎impeller/renderer/backend/vulkan/queue_vk.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ class QueueVK {
3939
vk::Result Submit(const vk::SubmitInfo& submit_info,
4040
const vk::Fence& fence) const;
4141

42+
vk::Result Submit(const vk::Fence& fence) const;
43+
4244
vk::Result Present(const vk::PresentInfoKHR& present_info);
4345

4446
void InsertDebugMarker(std::string_view label) const;

0 commit comments

Comments
 (0)