Future of OpenGL/OpenGLCompute needs deciding

[steven-johnson]

Per various offline discussions (and some online, eg #5474), our OpenGL backend has many limitations/problems and is arguably of dubious value even if implementation issues are fixed. We should really decide whether we want to improve it or officially deprecate/remove them.

Strawman proposal to start a discussion:

• Designate these backends as 'deprecated' as of Halide 11
• Remove them entirely in Halide 12
• Take the work-in-progress Vulkan backend and devote resources to it with the intent of getting it useful for Halide 12

[shoaibkamil]

The OpenGL (non-compute) backend is extremely limited and probably not worth keeping. However, I think OpenGLCompute is worth saving and will definitely help us get Vulkan going (since they are semantically very similar). The major limitation of OpenGLCompute AFAIK is poor support for anything except 32-bit types; we will need to solve this for Vulkan as well. So I propose:

• mark OpenGL deprecated in Halide 11 and remove in Halide 12
• move type legalization from D3D12 into a reusable pass that can be used by Vulkan, OpenGLCompute, etc
• get OpenGLCompute working with all GPU tests (except those that do e.g. workgroup operations)
• finish WIP Vulkan backend (@slomp and I are planning on tackling this starting late Dec/early Jan)

[slomp]

On that note, I'm almost inclined in creating a general gpu back-end run-time subsystem, and there is a lot of code in the gpu back-ends that could be shared among each other.

[alexreinking]

We need to decide on this. I think @shoaibkamil's proposal to deprecate in Halide 11 and remove in Halide 12 is a good one. We need to support this with loud warning messages when using the OpenGL backend.

[alexreinking]

I wonder whether #5545 should influence our thinking on this. I think it would be good to get it reviewed and merged for Halide 11 but still move forward with deprecation. The current model of "let OpenGL bit rot until someone fixes it" isn't really sustainable.

[steven-johnson]

I'm all in favor of landing #5545 if it passes review.

[zvookin]

The issue with OpenGL is not so much bit rot but that it never worked very well in the first place. Some of this is the compiler implementation, but some of it is fundamental limitations in OpenGL itself. (As well as the practical realities of the ecosystem around it.) We can certainly improve the compiler, but if every use case has it dominated by another approach, it seems wasteful to push on this.

[alexreinking]

From our discussion today, it sounds like we will deprecate OpenGL in Halide 11, which means we need to add a warning.

Then we will outright remove the code from master once it's released.

[steven-johnson]

I'll add warning(s)