1
$\begingroup$

I recently ran across the paper RAISR: Rapid and Accurate Image Super Resolution and thought it seemed like an interesting technique for scaling up images. However, I haven't seen any implementation of it, and I would like to see what it would be like to use it for scaling up a video in realtime, if that is possible. So I'm thinking of implementing my own version of it, but I will need to use GPU acceleration in some way if I want to at least give it a chance to run in realtime. Ultimately, I would like to see if I could successfully implemented this in an open source media player.

So, what library should I use to implement this algorithm? DirectX and CUDA are out of the question, since I want to make it run under any (major) OS and on any graphics card (well, on most of the popular ones, at least), so I'm thinking that my only options are

  1. OpenGL (ES) + GLSL (ES), and

  2. OpenCL.

(Vulkan also exists but I don't know whether this API is mature enough yet.)

If I choose OpenGL + GLSL, is this approach flexible enough to allow this algorithm to be implemented? Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?

$\endgroup$
  • $\begingroup$ "If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?" That's just as true of OpenGL. The only difference being that GL typically comes with graphics drivers, while CL typically requires an explicit download. $\endgroup$ – Nicol Bolas Feb 3 '17 at 1:54
  • $\begingroup$ Why was my question downvoted? $\endgroup$ – HelloGoodbye Feb 3 '17 at 18:27
  • $\begingroup$ @NicolBolas That may be true. So what frameworks do media players and other video software usually use to accelerate video rendering? $\endgroup$ – HelloGoodbye Feb 3 '17 at 18:34
  • $\begingroup$ "Vulkan also exists but I don't know whether this API is mature enough yet." It's mature but Apple doesn't support it, so you won't meet your goal of running on any OS. $\endgroup$ – Dan Hulme Nov 7 '17 at 14:28
  • $\begingroup$ I implemented a CUDA version of RAISR and am working on an OpenGL implemention. The hashing step isn't much of a problem -- just cache things in shared memory. You'll get garbage performance on older hardware, but on newer low-end cards, it's totally fine. The tricky part is the filter lookup, and there's quite a few methods available to deal with that problem (shared memory, some compression tricks, etc). But you'll probably get decent performance if you just store the filters in an RGBA texture. $\endgroup$ – AnimatedRNG Dec 12 '18 at 20:35
2
$\begingroup$

Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

Yes. Generally scatter-gather algorithms work poorly in OpenGL as the data locality is poor. When you have to grab bits from all over the input image for each output pixel, it's going to be slow or nearly impossible to implement in a typical GL shader. Likewise, when a calculation on an input pixel results in data that needs to be spread out to several output pixels, it's a poor fit.

It's hard to tell just from the abstract, but it looks to me like this wouldn't be a great fit for a typical OpenGL shader. I think you'll end up needing to sample an area of the input, classify it, then sample from one or more of a large set of small exemplars.

I mainly work on iOS and macOS which both come with OpenGL and OpenCL. Not sure how Windows and Linux handle that.

$\endgroup$
  • $\begingroup$ Thanks. So do you mean that scatter-gather algorithms work better in OpenCL than in OpenGL? $\endgroup$ – HelloGoodbye Feb 6 '17 at 7:42
  • $\begingroup$ Which part of the algorithm don't you think would be well suited for an OpenGL shader? The algorithm uses interpolation by combining the pixels in a small neighborhood linearly with coefficients that it gets from a lookup table. The table key is obtained by looking at the strength, angle and spread respectively of the gradients in a local neighborhood and combining the quantizations of those values. It also calculates the local binary patterns feature. $\endgroup$ – HelloGoodbye Feb 6 '17 at 7:58
  • $\begingroup$ The part you described sounds like it involves sampling in an area around the pixel currently being operated on to calculate a value to then use in a lookup table. So you your "gathering" by sampling in an area around the current pixel. Then your doing a dependent texture lookup. Both of those can be expensive depending on the hardware you're running on. If the area sampled in both the input texture and the lookup table is small enough, it might not be a bottleneck, but I suspect that's where it would happen. It seems OpenCL might be a better fit for that reason (to me). $\endgroup$ – user1118321 Feb 6 '17 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.