Ye Olde GDI

So, the switch to LuaJIT has been going well.  I was having one challenge with trying to get a User32 based window up on the screen, then I thought, I’ll just use GLFW because they’ve already figured this out across platforms.  And it’s working like a charm!

So, what can I do?  I have a window, it has a GL Context associated with it, and other than the core GLFW, everything is actually Lua.  That’s the power of simple interop.  There are other things I want to interop with though.  Some number of years ago, I wrote this screen capture code.  If you go digging around my C# archives, you can find the Snapper example.  It’s a very simple application that takes a snapshot of the screen.  It uses the underlying GDI stuff that’s been in Windows since time immemorial.  I wrote it about 5 years ago I think, to teach people how to do some rudimentary programming, with interop, on Windows.

It’s a good example case of how to do some simple things in Windows.  It involves GDI DibSections, Device Context, BitBLT.  And that’s just to get a copy of the screen.  I want to go a step further though.  I want to be able to share my screen (collaboration).  So, there are two things that need to be done.  First, I simply need to be able to display what I’ve captured into the GL Window.  That’s a relatively easy task now.  I first capture into the DIB Section.  Then I copy those bits to a GLTexture object, which is displayed by applying the texture to a quad in GL, and there you have it.  Without doing any sort of compression tricks, I can get about 24 frames/second capturing at 1920×1080 on my monitor.  That’s pretty darned good considering how slow this was when I first did it 5 years ago.

But, let’s say you want to do something more interesting that mirroring between two monitors on your desktop.  Let’s say you want to share your screen with someone, or multiple someones, in a classroom, or on iPads, across the internet, etc.  Well, now you have the basics.  But, the math isn’t looking too good.  At 1920×1080, that’s 2073600 pixels you have to shove down the pipe.  Given the typical RGB, that’s 6220800 bytes, and at 24 frames per second, that’s 149299200 (roughly 145Mb), or 1Gbit/second.  No problem, as long as you’re connected to a hard wired router that handles that kind of speed.

But, I want to reduce that somewhat.  The first thing I want to do is switch from RGB to YUV (4:2:0).  the YUV color model is such that luminance gets more bits than chrominance.  In this particular configuration, I can take a 4×4 pixel area, and instead of using 12 bytes, as I would for RGB, I can use only 6 bytes, and not really lose too much in terms of image quality.  That’s a 2:1 compression right there.  Another trick I can do is to reduce the size of the image by simple decimation.  In this case, I can reduce by half in each direction, and still be happy with the outcome.  So, down to 960×540 = 518400 bytes per frame, or 12441600 bytes per second at 24 fps (roughly  12Mb), or 120Mbits per second.  Divide by half for switching to YUV, and you get a bitrate of roughly 60mbits.  That’s much more reasonable for hard wired networks running at 100Mbit.

Now that I’m in a more reasonable realm, it’s time to do some serious compression.  H.264 seems to be the compression technique of the day (wave to WebM and others).  As it is designed to deliver the likes of YouTube and others, it is certainly adequate to the task of sharing desktop snapshots.  So, back to LuaJIT.

Next task is to write the FFI for x264.  That should be straight forward as it has a nice clean interface to play with.  If I can run x264 in realtime on a quad i7, then I think I have a chance of sharing my desktop out in realtime over wireless networks.  So, that’s my next task.

Using LuaJIT makes this stuff really easy.  It’s fairly hard to deal with all those various libraries.  Mostly dealing with memory management as is typical with non GC based languages.  As well, dealing with differing interfaces such as C vs C++, exceptions, etc.  LuaJIT provides a fairly streamlined view of all those as the FFI layer kind of deals with a lot of it for you.  Of particular usefulness is the FFI figures out stdcall vs fastcall vs whatever, and that can be a pain in and of itself.

So, along with my desktop, I’ll be hooking up the webcam programs like Camera Serve.  That’s a nice little program that takes the stream of pictures from whatever webcam you specify, and streams it out so that others can view.

None of this is rocket science or new.  These are merely exercises to play with the LuaJIT/FFI interfaces.  But, along the way, I’m finding that the code is becoming much simpler, smaller, and it’s just so darned quick to throw things together, once you’ve worked out some basics.

I would put up a picture of my shared screen, but without actually seeing it shared across the monitors, that’s kind of pointless.  So, it will have to wait until I get x264 working so I can make a movie of the experience.

---