0
$\begingroup$

A friend of mine offers a live-stream via TCP. I can view the live-stream in a browser by fetching it via web-socket. What I want to achieve is to segregate the video into frames. Furthermore, the frames have to be saved to disk as images (e.g. png). I don't want to use a screen recording tool, though.

My idea was to access the graphics card buffer via OpenGL or any other library and get every frame of the video live stream there. I just don't know if this is possible and where to start. Of course I know that every more or less modern graphics card has its own frame buffer. Is it possible to access this buffer and get every frame during the live stream is played in the browser?

$\endgroup$
  • $\begingroup$ It sounds like you're asking, "I don't want to use a screen recording tool, I want to make my own." $\endgroup$ – Dan Hulme Aug 7 at 7:31
  • $\begingroup$ @Dan Hulme I would like to process the frames further and keep them in-memory. Afterwards I would like to send them via TCP to another client. A screen recording tool seems unappropriate for this use case. $\endgroup$ – enne87 Aug 7 at 20:33
  • $\begingroup$ Processing the frames, buffering them, and sending them over the network is exactly what a screen capture tool like OBS does. It's open-source and has a plugin architecture, so you can use its capture architecture and write whatever code you like to consume the capture frames. $\endgroup$ – Dan Hulme Aug 7 at 21:17
  • $\begingroup$ @Dan Hulme Aha very interesting, I didn't know that. Would you mind writing a short answer so I can accept it? Maybe with a link to a description of OBS' plugin architecture? Thank you so much! $\endgroup$ – enne87 Aug 8 at 16:37
1
$\begingroup$

I wouldn't recommend trying to write this code yourself. It's not a question of using into OpenGL - which won't get you another process' framebuffer - but of asking the OS. This is not at all easy to do, and it's work you don't need to do yourself.

Depending on what the streaming protocol is, the easiest way might be just to write or use a client for that protocol directly instead of hooking another application. Failing that, a screen capture tool such as OBS can already do what you want. It can capture another application's frames or your desktop with relative efficiency. You can set up overlays or write a plugin to manipulate the video the way you want. If you access the stream through a web browser, you might not even need to capture a window: it has the built-in feature of being able to render into your video a web browser component it runs itself (you tell it what URL to point to) without having to run a separate web browser and capture it. This feature is how most streamers get things like a Twitch chat view, score board, or timings onto their video.

OBS has a C API for writing fast, native plugins as well as Python and Lua interfaces. And it's open-source, so in the unlikely event you can't write a plugin to do what you want, you could hack it into a private fork directly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.