BanateCoreWin32, is the barest most minimal amount of code required to interop with Windows, to get a window up on the screen, and do some bare metal graphics.  It includes various amounts of:

user32, gdi32, kernel32, opengl, avicaps32

A very large part of interacting with things on Windows has to do with interop through COM.  Well, that’s a whole other beast in and of itself.  I could use LuaCom, but that code seems a bit out of date.  What I really want access to is the USB port, to get at things like Kinect and web cameras.  Ideally, I’ll just read the USB port directly, totally avoiding things like DirectShow and the myriad other layers of left behind APIs that are in Windows.  But, this will take some time.

In the meanwhile, if what you want to do is put a window up on the screen using LuaJit code from the beginning, then this library will help you do it.

The great benefit is that it’s cross platform.  But, the downside is a 46K glfw.dll file that I have to carry around.  Well, what is GLFW in the first place?  It gives me a nice easy way to get a window, with a OpenGL device context already made and ready to go.  It also handles mouse and keyboard actions in a coherent way.  Besides that, it does a little bit of text handling, and can load Targa images.  By design, that’s about all it does.

For the most part, it’s wrapping functions that are already in the underlying OS.  So, I thought, why not just write those wrappers in Lua itself?  I’m already doing wrappers for GDI calls, so User32 is not any harder.  It was kicking my behind though, and I was having problems getting CreateWindowEx to work correctly.  Here’s one of those places where the “magic of Windows” kind of screws with you.  On Windows, when you make a call like ‘CreateWindowEx’, you don’t actually call a function by that name.  The wisdom of Windows will determine whether you need to be calling the ASCII version (CreateWindowA) or the Unicode version (CreateWindowW).  As long as you’re using Microsoft environments, this magic is figured out automatically.  When you’re on the frontier, doing your own interop, you have to be explicit about what you’re calling.  Well, I finally got all that straightened out, and I can finally put a window up on the screen:

Now that I can actually display a window, it’s a small step to be able to create my own GL context within that Window as well, all from Lua.  The amount of wrapper code can be counted in the hundreds of lines of Lua code.  The FFI of LuaJit makes this a relatively painless process.  Just grab sections out of a header file, put them into a ffi.cdef[[]] block, and start calling your functions as usual.

Having used this LuaJit FFI stuff, I really can’t see using any other mechanism.  The previous forms of interop to C from Lua were very challenging, and I’m very relieved that FFI reduces the challenge to simple copy/paste.

At any rate, that’s a savings of 46K for the release.  Now I can go spend that on some other new features.

cat file | more

Much later, C# continued the theme with streams.  Not quite at the process level, but within programs themselves.  Streams are great because they provide a simple common interface to just about anything:

stream.WriteByte(abyte); stream.Write(somebytes); stream.WriteTo(stream)

Well, this is great.  A nice abstraction for packaging things up and sending them somewhere else.  Although Lua has a simple io library, which has a nice enough interface, it’s missing a couple of things.  At the very lowest level, I want a stream on a chunk of memory.  In the .net frameworks, that’s provided by the MemoryStream class.  You tell it how much memory, and then you can do Read(), Write() to that memory as if it were any other stream.

This is a pattern that shows up a lot in network programming as well.  I’ve got data, and I need to encapsulate it into a chunk of memory, and then send it out over a connection of some sort.  It sure would be nice to have that MemoryStream in Lua.

So, I created a MemoryStream object.  But, fefore I could create that, I had to first create the ByteArray object.  It’s nothing more than a length, and ‘unsigned char *’, for the actual data.  You can allocate one like this:

ba = BufferArray():new(256)

This is possible with LuaJit.  You can do it in regular Lua, but not so easily, and not without writing a bit of C code I believe.  The interface to this thing is simple, you just access the data directly:

ba.data[0] = 23

under the covers, my allocation code is doing this:

self.data = ffi.new(‘unsigned char[?]‘, size)

Through the magic of LuaJit, this will allocate exactly 256 bytes.  If I were using a regular Lua table and did this, it would allocate 8 times the amount, as lua numbers are actually doubles (8 bytes each).

Now, armed with this simple ByteArray, I can build the MemoryStream:

ms = MemoryStream(256)

ms:WriteByte(35);

ms:WriteByte(66);

ms:WriteByte(67);

To write ‘A’,'B’,'C’ into the memory buffer.  But, if I’m just going to write characters, that would be of limited usefulness.  I really want to deal with integers, and fractions thereof.  So, I can also do this:

ms:WriteInt32(527)

ms:WriteFloat(37.5)

And likewise for all the other numerical types.  Well, now we’re talking!  The MemoryStream will automatically write the value as the appropriate number of bytes in network byte order (big endian).  That’s useful because now I can write out the attached ByteArray as a single chunk, using whatever socket library I happen to have.

It’s just one of those fundamental building block sorts of things for me.  A ByteArray, a streaming interface, that makes it relatively easy to do something like chaining streams together, just like with UNIX pipes.  Now that I have an easily managed chunk of memory, I can go back and look at the PixelBuffer class I created, and refactor it to just use a ByteArray at its base.  That means eliminating more code!

Meanwhile, back at the graphics ranch, I managed to squeeze 1200 bytes out of my glsl.lua code.  Basically just refactoring the routines to utilize a couple of common base routines.  That’s 1200 bytes I can now allocate to improving the ray tracer, or adding in some other feature.  I’m thinking the ray tracer will be the best place to spend these newfound bites.

I’ve also decided that I don’t need to have the ability to read all file formats known to man for text maps.  I’ll have Targa built in, because I wrote code for that a long time ago, and not only is it an easy format to deal with, but the run length codec it uses can be repurposed, which saves more code.

With that, the only real hard dependency I have, beyond LuaJit itself, is the glfw framework/library, and LuaSocket.  For now, I’ll stick with both of these as they are cross platform, and not too onerous.  With all this work, surely a new release is in the offing!

So, what can I do?  I have a window, it has a GL Context associated with it, and other than the core GLFW, everything is actually Lua.  That’s the power of simple interop.  There are other things I want to interop with though.  Some number of years ago, I wrote this screen capture code.  If you go digging around my C# archives, you can find the Snapper example.  It’s a very simple application that takes a snapshot of the screen.  It uses the underlying GDI stuff that’s been in Windows since time immemorial.  I wrote it about 5 years ago I think, to teach people how to do some rudimentary programming, with interop, on Windows.

It’s a good example case of how to do some simple things in Windows.  It involves GDI DibSections, Device Context, BitBLT.  And that’s just to get a copy of the screen.  I want to go a step further though.  I want to be able to share my screen (collaboration).  So, there are two things that need to be done.  First, I simply need to be able to display what I’ve captured into the GL Window.  That’s a relatively easy task now.  I first capture into the DIB Section.  Then I copy those bits to a GLTexture object, which is displayed by applying the texture to a quad in GL, and there you have it.  Without doing any sort of compression tricks, I can get about 24 frames/second capturing at 1920×1080 on my monitor.  That’s pretty darned good considering how slow this was when I first did it 5 years ago.

But, let’s say you want to do something more interesting that mirroring between two monitors on your desktop.  Let’s say you want to share your screen with someone, or multiple someones, in a classroom, or on iPads, across the internet, etc.  Well, now you have the basics.  But, the math isn’t looking too good.  At 1920×1080, that’s 2073600 pixels you have to shove down the pipe.  Given the typical RGB, that’s 6220800 bytes, and at 24 frames per second, that’s 149299200 (roughly 145Mb), or 1Gbit/second.  No problem, as long as you’re connected to a hard wired router that handles that kind of speed.

But, I want to reduce that somewhat.  The first thing I want to do is switch from RGB to YUV (4:2:0).  the YUV color model is such that luminance gets more bits than chrominance.  In this particular configuration, I can take a 4×4 pixel area, and instead of using 12 bytes, as I would for RGB, I can use only 6 bytes, and not really lose too much in terms of image quality.  That’s a 2:1 compression right there.  Another trick I can do is to reduce the size of the image by simple decimation.  In this case, I can reduce by half in each direction, and still be happy with the outcome.  So, down to 960×540 = 518400 bytes per frame, or 12441600 bytes per second at 24 fps (roughly  12Mb), or 120Mbits per second.  Divide by half for switching to YUV, and you get a bitrate of roughly 60mbits.  That’s much more reasonable for hard wired networks running at 100Mbit.

Now that I’m in a more reasonable realm, it’s time to do some serious compression.  H.264 seems to be the compression technique of the day (wave to WebM and others).  As it is designed to deliver the likes of YouTube and others, it is certainly adequate to the task of sharing desktop snapshots.  So, back to LuaJIT.

Next task is to write the FFI for x264.  That should be straight forward as it has a nice clean interface to play with.  If I can run x264 in realtime on a quad i7, then I think I have a chance of sharing my desktop out in realtime over wireless networks.  So, that’s my next task.

Using LuaJIT makes this stuff really easy.  It’s fairly hard to deal with all those various libraries.  Mostly dealing with memory management as is typical with non GC based languages.  As well, dealing with differing interfaces such as C vs C++, exceptions, etc.  LuaJIT provides a fairly streamlined view of all those as the FFI layer kind of deals with a lot of it for you.  Of particular usefulness is the FFI figures out stdcall vs fastcall vs whatever, and that can be a pain in and of itself.

So, along with my desktop, I’ll be hooking up the webcam programs like Camera Serve.  That’s a nice little program that takes the stream of pictures from whatever webcam you specify, and streams it out so that others can view.

None of this is rocket science or new.  These are merely exercises to play with the LuaJIT/FFI interfaces.  But, along the way, I’m finding that the code is becoming much simpler, smaller, and it’s just so darned quick to throw things together, once you’ve worked out some basics.

I would put up a picture of my shared screen, but without actually seeing it shared across the monitors, that’s kind of pointless.  So, it will have to wait until I get x264 working so I can make a movie of the experience.

My environment of choice right now is SciTE.  As Integrated Development Environts (IDEs) go, this is not much more than a nice editor with syntax highlighting.  It has some debug capability, but honestly, I haven’t even tried it as yet.  My favorite “Print” debugger works the same as it has for the past 25 years, so I’ll stick with it until absolutely necessary.

Lua does satisfy many of the constraints I’ve given myself in terms of platform availability, succinct code, flexibility, and the like.  Of particular value is the ability to run on really small devices like microcontrollers, cell phones, TVs, and whatnot.

Although I’ve been satisfied to date, speed is also a concern.  Previously, I made a note about using LuaJIT, and that is the path that I am traveling down as well.  It’s like throwing off training wheels.  At the same time that I go down the LuaJIT path, I am contemplating my dependency chain.  The various libraries that I use.  I currently use iup (for basic UI) and IM (for imaging).  I also use LuaSocket for collaboration.  The challenge with iup is that it’s not easily on all platforms.  Works great for Linux and Windows, not so much for Mac.  I’ve also reconsidered my need to use IM.  I don’t need to be able to read every type of image file that exists in the world.  I’m just going to pick a few.

So, I’m changing things up.  I’ve switched to using glfw as my core UI library.  GLFW is the most bare minimum UI framework glue you could hope for.  It combines being able to get a handle on a window with being able to get a GL context.  Add in simple mouse/keyboard/joystick events, and you’re done.  It does not try to solve all programming challenges like larger frameworks.  So, this is very good.  It works on Windows, Mac SO X, and many Unix-like systems using X Windows.  Well, that’s a good start.  Perhaps I can add the Android and iOS portions if they’re not already there, and call it a day.

This is great.  Now that I’m using LuaJIT, and I’m takinig up GLFW, I get full access to the entirety of the OpenGL/ES API.  I found a nice ready made set of FFI based interop stubs in this project UFO.  Dimiter “malkia” stanve has done a great job of going down the path of providing interop to many different libraries, and across multiple platforms, including iOS and Android.  That’s nice.  Now I can truly write once, and run anywhere.  That’s the benefit of the Lua talking.

One of the challenges I was facing when using the more pure lua (not LuaJIT) was the interop story.  It’s extremely hard to write those perfect interop layers.  As such, I was dreading having to write the fully expanded OpenGL API so that I could do some shader programning.  Now, with LuaJIT and UFO, I have ready access to all the nice shader functionality.  That’s really cool.

Now to try things out.  LuaJIT is supposed to be fairly fast.  So, I’m going to test that speed by doing some simple compressed target tests.  That’s an easy task because Targa (.tga files) files use a simple run-length encoding scheme.  So, let’s see, take a snapshot, compress, send over the net, decompress.  Yep, that aught to about do it.

At any rate, you can go an extremely long way by using these modern tools.  I’m weening myself and reeducating my expectations as to what can be done with a “scripting” language.  I have found myself to be much more productive, able to write new bugs faster than ever before!  This is great, and I’m sure as more people head down this path (just like with JavaScript), we’ll see a lot more interesting and dynamic stuff coming online.

What is that?  Well, it’s a bunch of spheres that have been RayTraced in BanateCAD!

I’ve been going on a very interesting trip of late.  What I have been pondering on my walkabout is the following:

Is it possible to support – 3D CSG, Isosurface, Solid Modeling, Perspective, Orthographic, Arbitrary Bump Mapped surfaces, collaboration, and 2D UI, all in a single environment, written almost exclusively in Lua, in less than 256K of source code?

It’s a challenging endeavor.  Every new feature adds code.  I’ve been following a simple principle.  For every expansion of code, there must be an equal and opposite constriction of code.

I am looking at Ray Tracing as a relatively quick way to display a scene.  The first time I even played with ray tracing was probably back in 1994, playing with whatever version of POV-Ray was available at that time.  On my standard issue PC of the day, I would setup a semi-translucent sphere on checkered floor scene, and let it run overnight!  Wow, how times have changed.  Given all the processor performance doubling in the intervening years, machines are certainly capable of rendering much more complex scenes in a fraction of the time.  My thinking is, Ray tracing is the way to go for instant display, and I don’t deal with meshes at all until I absolutely need to.  That might only be at export time in most cases.  At the same time, it will make displaying isosurfaces in general much easier.

When it comes to the simple ‘scenes’ in 3D modeling, a fraction of a second to render an interesting model should not be unheard of.

Ray tracing is interesting.  Similar to 3D modeling, there are some core geometries that you want to deal with.  Ellipsoid, cube, cylinder, etc, and of course some interesting surfaces.  This implies a certain kind of ‘world’ model.  Some sort of data structure that exists so that when it comes time to cast rays, you can walk that structure, calling an “intersect()” function for each object.

Similarly, you have the same thing for 3D modeling, but you’re a bit more concerned with a mesh representation, as you’re going for 2-manifold solids so you can actually print things out.  That implies a representation that favors some mesh like thing.

And finally, there’s the 2D UI world.  Not 2-manifold, so no mesh required, and instead of a “hit”, which is concerned with hit a boundary of an element, you have “contains()”, which is concerned with whether a mouse click is ‘inside’ or ‘outside’ the object in question.

Yep, quite a lot to balance there.

There are essentially three different kinds of renderers, and at least 2 different types of UI interactors (2D vs 3D).

So, some basic things will help here.  First of all, there needs to be a fundamental way of organizing ‘scenes’ that’s universal.  I can default to the 3D representation because I can always assume ‘z==0′ for the 2D world, with some arbitrary non-zero thickness.  That’s nice.  And how about the actual structure itself?  How about a k-d tree?  That seems to be a nice universal structure that can be applied to the 3D scene. It can nicely deal with quickly isolating which objects are in which part of a scene, without having to traverse the whole scene graph.  This is critical for fast rendering updates.  OK, that’s good, and I can reduce a lot of code.

How about basic data structures?  Well, I started out with a ‘vec’ object.  A basic one dimensional array that deals with doing some basic linear algebra.  Then I added Vector3D, Point3D, Normal3D, and Ray3D.  All very similar, but slightly different.  And then, I added the Matrix class, which deals with 1D or 2D arrays (1D being a degenerate case).  Well, there’s really no difference between the Matrix class and the vec class, so I think the vec class can disappear, or rather simply become a wrapper for the Matrix class.  Same goes for the other ones, although, they are presently just wrappers for vec anyway.  So, more reduction.

At present, the ray tracer is so rudimentary it’s not even worth calling it a ray tracer, but it is extremely small (80 lines of code).  As it expands in features, it will more than double, and perhaps triple in size.  Given that I won’t need to introduce any new geometry, this expansion is not that bad.  I’ll pay for the expansion through consolidation of the aforementioned vector/matrix objects, and savings elsewhere.

In other news, I’ve been playing with LuaJIT some more, and have a basic Pixel/PixelBuffer thing working.  It’s fast as can be, and can already be used as an output surface for any of the 3D modeling that I’m currently doing.  This bodes well since I can cut the number of dependencies (CD and IM Libraries), which makes the code that much smaller and portable.

I love C# as a language.  Really I love well done garbage collection, otherwise, it’s just another in a long line of ‘C’ inspired languages.

One thing I find particularly challenging with C# is that there isn’t always an interop library available for the latest and greatest cool thing.  Nowadays, invariably it will show up, but still, not always.  For example, the very first attempts at interop with a Kinect were not using C#.  In addition, C# is limited in that it is not really native to all platforms on the planet.

So, what about Lua?  Well, Lua, although it works fairly nicely with C interop, is a bit much to deal with.  In order to write a simple wrapper might require quite a few likely error prone steps.  The state of things is such that it’s best done using a tool, such as tolua, or tolua++.  These tools will invariably write a bit of C code that wraps the library call in question with some  more goo, dealing with marshalling parameters if necessary, and the like.

Then, along comes LuaJIT.  There are a number of good things about LuaJIT.  One of the most important is that it runs much faster than the standard Lua distribution.  In some cases, the code runs as fast as standard compiled C code does.  At the very least, I believe it runs faster than any of the other scripting language environments, including the V8 processor for JavaScript.

The thing that got me to look at it though was not just the speed, although that’s a great bonus.  The other thing is the easier interop to libraries.  With LuaJIT, there is this FFI technology that allows you to make library calls by doing nothing more than taking the standard C definitions of those functions and putting them within some brackets.  The LuaJIT parser can read the standard C declaratons, and generate the appropriate stub call, do the marshalling of parameters, and all that stuff.

Well gee!  That’s just a godsend.  I was really dreading having to write all that wrapper good, bloating my code in the process.  With LuaJIT, I don’t have to.  I just need to make sure my declarations are exactly right, and then make calls to functions almost as easily as if they were just Lua calls.

Another bonus is the easy treatment of structure types.  You can use the familiar:

typedef struct { int x,y;} POINT;

And that works just fine.  Similarly, if you want to create an array of rgb pixels: typedef struct {byte r,g,b;} pixel_rgb

And you’re all set.  This latter one is useful if you’re doing any rendering, because from Lua I can now do “direct access” to stuff, like I do in C, although there is still managed memory safety.  This reduces the memory footprint of my code, and speeds things up dramatically, particularly when I’m transferring from a pixel buffer to a texture object for display on the screen.

LuaJIT is a departure from the normal Lua distribution, which just recently had a release.  I for one welcome the departure.  It allows for much more flexibility in how to do things.

One thing that I discovered as I was trying out LuaJIT.  The first library I connected to was the gdi32.dll on Windows.  I wanted to get a device context for the screen, and then draw lines, ellipse, and rectangles.  Easy enough.  I did discover something along the way though.  I thought Rectangle() would be really fast, hardware accelerated goodness.  As it turns out, it took many seconds to draw a few hundred on the screen.  I thought that was rather odd, so I tried out RoundedRect() to see how much worse that was going to perform.  Given the same number of rectangles, it displayed in the blink of an eye!  Go figure.  I guess somewhere in the graphics pipeline, rounded rectangles got optimized, and straight ones did not.  that’s a good fact to know.

One thing that makes this interesting to me is the ease with which this little experiment can be achieved.  Since it’s Lua, it’s a small matter of script to get things going.  There is no explicit “compile” phase.  Just write a little script and run it, and see what happens.

Given the speed, the interop capabilities, and the general ease of use of Lua, I’m wondering if I ever really need to use anything else to do system type programming.  I did write all those libraries in C#, but maintaining, and extending them is quite a chore, an art and science, and a generally error prone activity to get them absolutely right.

The next thing I’ll try, after finishing up some more of GDI and User32, is hook up the USB port functions to see how hard it will be to Kinect.

Well, I’m actually in favor of asynchronous processing because that’s the way the world works in general.  Sure, we drive towards things, but more often than not, we are responding to something in our environment.  I figure the same should be true of a program.  Yes, it has some goals in mind, but largely it should be responsive to activities that occur on the periphery.

HeadsUp is the 2D aspect of BanateCAD.  This is where events like mouse and keyboard interaction enter the system.  Up until recently, I dealt with these events in a synchronous sort of way, with a typical event loop.  Now, I do a hybrid.  Yes, there is a fundamental message pump that pulls messages out of the system’s message queue, but then that pump turns around and stuffs them into an asynchronous queue, which is handled by a coroutine (cooperative multitasking).  That’s handy for a couple of reasons.

First of all, the message pump does not have to be completely synchronous with the rest of the program.  It can just pull messages out of the pipe, and stick them into the queue, rinse and repeat.  That makes the message pump very simple, and easily separable from the rest of the program.  Second, the part of the program that deals with processing messages does not have to know anything about the mechanics of the message pump.  All it knows it that it will be notified when there are messages to be processed.  It can then go pull as many messages from the async queue as it likes, and deal with them at its leisure.

Perhaps it’s only a point an asynchronous programmer could love.

Now that there’s a generic queue that can process commands, I want to make everything into commands.  So, here’s an example of a very typical case.  When I want to draw a line, there is typically some interface connected to the graphics API that looks like this:

Graphics:DrawLine(pt1, pt2)

That will talk to the rendering engine directly, and a line will magically appear.  But, what if under the covers, what was really happening was this:


void Graphics:DrawLine(pt1, pt2)
local cmd = Graphics:CreateDrawLine(pt1, pt2)
outboundqueue:Enque(cmd)
end


From the programmer’s perspective, there’s nothing different in their conceptual model.  They call DrawLine(), and that’s the end of it.  They assume a line is drawn, and it will be drawn, eventually.

From the system’s perspective, a whole world of possibilities just opened up.  Now that the command is packaged up and stuck into a queue, it could be immediately removed from the queue, and really executed against the graphics system, or it could be stored off in some persistent store somewhere, possibly on a distant part of the planet.  And, as long as you’ve got commands all packaged up and ready to go, you could ship them to other renderers while you’re at it.  That might be interesting and useful.

At any rate, it’s a fairly fundamental question to deal with.  Should you poll, or be notified, along with, should you call directly, or issue commands.  In the case of BanateCAD, there is a hybrid of poll/notify, and commands are definitely the way to go.

The samples included in the download show how to do various things from animations to simple controls.

Part of doing 3D CAD is being able to create a surface of revolution (revoloid).  Well, in order to create a revoloid, I need to be able to draw a curve, which will then be rotated around an axis.  Well, in order to create a curve, I need to have a nice grid I can use as a guide, and I need control points, and mouse control of those points, etc.

So, I create those various bits and pieces.  I think the grid thing is really cool.  You can specify the distance between the light and dark lines.  You can specify the colors as well.  It becomes useful when you need a backdrop for drawing, or displaying stuff in general to scale.

There have been other graphics along the way as well.  This arrow thing is useful in that the dark gray area can hold other graphics and controls.  Perhaps it doesn’t make sense sitting there on its own, but it does when it’s combined with other things.

Of course, no UI toolkit these days would be complete without having tabbed views.  So, there they are…  Of course, since the tabs are created using the ShapeBuilder object, you can get rounded corners instead of those sharp corners if you like.  Since these are just graphics objects, you can also fairly easily do drop shadows, glowing backgrounds, and the like.  And, since they are “Actor” objects, they can respond to “Update” messages, and change with time, if that’s useful.

I recently purchased a BeagleBone.  A nifty little piece of kit that one.  It uses a TI AM3358 ARM Cortex-A8 based microprocessor.  The board has ethernet and USB built in.  One of the benefits of the little board is that you can run an Android distribution on it.  Also, it has some graphics capabilities, including support for a VNC host, so you can see what’s on it’s “screen”.

I fully intend to put BanateCAD on this little device.  The Beaglebone is good in that it represents a fairly constrained system.  It has a 4Gb micro SD card, and 256Mb of RAM.  It runs at 700Mhz.  I’m thinking this should be a beefy enough spec to run a simple 3D CAD modeling program such as BanateCAD.  Also, if I can run on this little device, then certainly I can run on any Android device.  I’ll see how it goes.

At any rate, things progress.  2D UI, 3D shapes, monkeys on typewriters…  Soon enough something interesting will pop out of this little exercise.