10
$\begingroup$

I'd like to "hook" into a running game, say Mario Bros, and capture each frame rendered... saving that frame to an image file. A good example of something similar is FRAPS. --Note: I don't want to capture the whole screen/desktop. I want to capture a targeted window.

I have had a look at OBS (Open Broadcasting Software) but it is not particularly fast. Don't get me wrong, it's great software, but unfortunately there is no/poor documentation, making a massive project written in c and c++ nearly inaccessible for a new-to-c++ programmer.

I've also had a look at GamingAnywhere, but unfortunately, I am unable to get it to work, there is very little/no documentation, runs in VS2010 only and is messy (with poor variable naming). However, it is a research project and so it is understandably undocumented and messy.

I know that this can be done with OpenGL, GDI and with Direct3D, but I am unable to find some good examples on the net.

I read that glReadlPixels (using OpenGL) can be used and I've read the documentation, but the post mentioned nothing about hooking into a running game/application graphics.

Questions:

  1. Can I hook into the graphics of a game developed with OpenGL, using, say, Direct3D? Does the library used for hooking have to be the same as the one used by the game?

  2. How do I hook into the game's rendered frames so that I can output those frames to image files or to a video file? (Just some links or brief explanation of what I need to do would be great)

  3. BackBuffer - I read that it is very fast accessing the BackBuffer to retrieve the frames. Does someone have an example for me on how to do this with the latest libraries? I have found that most examples are out of date.

  4. For my purposes, is there any clearly "this is faster than that"? What I mean is, would, say, OpenGL, be faster for my purposes?

If someone knows of an open-source project (that does essentially what I need) that is actively developed and well documented, I'd love to know about it.

$\endgroup$
1
  • $\begingroup$ Now this appears to be Mac only, which may not work for you with Direct3D, but Syphon Inject which OBS uses on Mac can capture game windows that use OpenGL much faster than standard video capture. Syphon "allows applications to share frames - full frame rate video or stills - with one another in realtime" to quote the website. $\endgroup$
    – Gliderman
    Oct 24, 2015 at 22:45

1 Answer 1

9
$\begingroup$

It is possible to do what you describe, but I'm afraid it is not a trivial process. Actually, this will be very tied to the Operating Systems you are targeting and possibly also requiring specific tweaks for the given game/application.

I would start looking into DLL injection techniques, which should allow, for instance, intercepting calls to the Direct3D (Windows) or OpenGL APIs, which you can then use to copy de framebuffers of the application. A quick Internet search turns up this and this with detailed explanation.

I once wrote a small-scale OpenGL interceptor on MacOS by injecting a custom shared module, but it was under very controlled circumstances and making a lot of assumptions, such as having root privileges in the system, so for a production tool, it would be a lot more complicated. You can find another very interesting project here about intercepting the D3D API to install an automated AI bot on Starcraft, which should touch on very similar concepts to what you are looking for.

Once you manage to intercept the underlying rendering API, you could then perhaps hook your code into each SwapBuffers or equivalent call to just copy the previous framebuffer before the swap, then save it (this is where something like glReadPixels would come into play).

Copying and saving the framebuffer in an efficient manner is also challenging. The easy way is to just dump each frame as an uncompressed RGB image, but then you'll wind up with hundreds of gigabytes of data for just a couple of minutes of gameplay, so unless you have a nice HDD array sitting at the corner of your table ;), you'll need to look into compressing the frames somehow.

The downside of compressing the frames now is that it takes a lot of processing, so a naive approach might turn the once interactive application you are trying to record into an interactive slide-show :(. So at this stage you'd have to start thinking about optimizations.

Don't know of any projects out there providing a library for efficient frame capture for games, but this would certainly be something nice to have! I think one thing that might be holding such project back is, like I mentioned at the beginning, that this is a very system dependent thing, so cases of use will most likely be limited to a single platform or OS.

$\endgroup$
6
  • $\begingroup$ Thanks for the answer. I think that ultimately, I will instead of saving images, stream them to a remote machine, so copying the framebuffer may be the way to go. You seem to have a fair bit of experience with this stuff. Are you able to expand on any of the questions that I posed? Thanks for the help! $\endgroup$
    – pookie
    Oct 25, 2015 at 10:10
  • 1
    $\begingroup$ @pookie Hey! Sure, I can expand this. Is there anything specifically you'd like me to comment on? For your questions, I think 1&2 can be done with the DLL hooks, so be sure to read the links I've included (specially the one about starcraft). 3&4 will be API specific, but on OpenGL, you can start with glReadPixels and work from there if you find it to be too slow. My experience is from recording frames from my own games plus some DLL hacking done in the past, but I've never combined both in the attempt of recording frames from a third party app ;) $\endgroup$
    – glampert
    Oct 25, 2015 at 17:13
  • $\begingroup$ Thank you! I've checked the links - windows DLL injection and starcraft link (pretty awesome) seem to be good starting points. I will take a deapre look into glReadPixels, too. In your opinion, which did you find easier to work with and which did you find faster: DirectX or OpenGL? $\endgroup$
    – pookie
    Oct 25, 2015 at 17:23
  • $\begingroup$ @pookie, if you have some programming experience, I think Direct3D will be much easier to use, the libraries are better designed. OpenGL has too may "rough edges" in my opinion (this coming from someone that uses OpenGL more than D3D). D3D is Windows only, so that might be a negative for you, don't know. Performance-wise, I don't think there's going to make much of a difference which one you use. The best advice I can probably give you is: first figure out how to get it working, then worry about optimizations. $\endgroup$
    – glampert
    Oct 25, 2015 at 18:08
  • 1
    $\begingroup$ Ah, ok, if you're familiar with OOP programming, then D3D will definitely be easier. OpenGL is all procedural and uses a bunch of global state, that's mostly what I meant about rough edges. Best of luck to you! $\endgroup$
    – glampert
    Oct 25, 2015 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.