1
$\begingroup$

I recently ran across the paper RAISR: Rapid and Accurate Image Super Resolution and thought it seemed like an interesting technique for scaling up images. However, I haven't seen any implementation of it, and I would like to see what it would be like to use it for scaling up a video in realtime, if that is possible. So I'm thinking of implementing my own version of it, but I will need to use GPU acceleration in some way if I want to at least give it a chance to run in realtime. Ultimately, I would like to see if I could successfully implemented this in an open source media player.

So, what library should I use to implement this algorithm? DirectX and CUDA are out of the question, since I want to make it run under any (major) OS and on any graphics card (well, on most of the popular ones, at least), so I'm thinking that my only options are

  1. OpenGL (ES) + GLSL (ES), and

  2. OpenCL.

(Vulkan also exists but I don't know whether this API is mature enough yet.)

If I choose OpenGL + GLSL, is this approach flexible enough to allow this algorithm to be implemented? Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?

Improve this question
$\endgroup$
5
  • $\begingroup$ "If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?" That's just as true of OpenGL. The only difference being that GL typically comes with graphics drivers, while CL typically requires an explicit download. $\endgroup$
    – Nicol Bolas
    Commented Feb 3, 2017 at 1:54
  • $\begingroup$ Why was my question downvoted? $\endgroup$
    – HelloGoodbye
    Commented Feb 3, 2017 at 18:27
  • $\begingroup$ @NicolBolas That may be true. So what frameworks do media players and other video software usually use to accelerate video rendering? $\endgroup$
    – HelloGoodbye
    Commented Feb 3, 2017 at 18:34
  • $\begingroup$ "Vulkan also exists but I don't know whether this API is mature enough yet." It's mature but Apple doesn't support it, so you won't meet your goal of running on any OS. $\endgroup$
    – Dan Hulme
    Commented Nov 7, 2017 at 14:28
  • $\begingroup$ I implemented a CUDA version of RAISR and am working on an OpenGL implemention. The hashing step isn't much of a problem -- just cache things in shared memory. You'll get garbage performance on older hardware, but on newer low-end cards, it's totally fine. The tricky part is the filter lookup, and there's quite a few methods available to deal with that problem (shared memory, some compression tricks, etc). But you'll probably get decent performance if you just store the filters in an RGBA texture. $\endgroup$
    – AnimatedRNG
    Commented Dec 12, 2018 at 20:35

1 Answer 1

Reset to default
2
$\begingroup$

Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

Yes. Generally scatter-gather algorithms work poorly in OpenGL as the data locality is poor. When you have to grab bits from all over the input image for each output pixel, it's going to be slow or nearly impossible to implement in a typical GL shader. Likewise, when a calculation on an input pixel results in data that needs to be spread out to several output pixels, it's a poor fit.

It's hard to tell just from the abstract, but it looks to me like this wouldn't be a great fit for a typical OpenGL shader. I think you'll end up needing to sample an area of the input, classify it, then sample from one or more of a large set of small exemplars.

I mainly work on iOS and macOS which both come with OpenGL and OpenCL. Not sure how Windows and Linux handle that.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks. So do you mean that scatter-gather algorithms work better in OpenCL than in OpenGL? $\endgroup$
    – HelloGoodbye
    Commented Feb 6, 2017 at 7:42
  • $\begingroup$ Which part of the algorithm don't you think would be well suited for an OpenGL shader? The algorithm uses interpolation by combining the pixels in a small neighborhood linearly with coefficients that it gets from a lookup table. The table key is obtained by looking at the strength, angle and spread respectively of the gradients in a local neighborhood and combining the quantizations of those values. It also calculates the local binary patterns feature. $\endgroup$
    – HelloGoodbye
    Commented Feb 6, 2017 at 7:58
  • $\begingroup$ The part you described sounds like it involves sampling in an area around the pixel currently being operated on to calculate a value to then use in a lookup table. So you your "gathering" by sampling in an area around the current pixel. Then your doing a dependent texture lookup. Both of those can be expensive depending on the hardware you're running on. If the area sampled in both the input texture and the lookup table is small enough, it might not be a bottleneck, but I suspect that's where it would happen. It seems OpenCL might be a better fit for that reason (to me). $\endgroup$
    – user1118321
    Commented Feb 6, 2017 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.