Frame Buffer 101

Have you ever seen examples of ASCII ART?  Probably one of the greatest examples of doing ASCII art can be found in the libcaca project.  I think that project has a most interesting license, and quite a sense of humor about itself.

But what you see here is some test code for the FixedArray2D class in BanateCore.

The challenge I am trying to conquer is simply this.  I need to manage a framebuffer, as in, something that can be drawn into.  I am implementing a version of the Remote Frame Buffer protocol (RFB), which is typically found in VNC products for remote desktop sharing.  Of course there are already plenty of implementations of such around the world, both commercial, and non, but what would be the fun in that.  There is even a Lua based version of a VNC client.  But hay, none of them done in LuaJIT as yet, so this is a perfect chance to exercise my low level 2D graphics libraries.

To implement the ASCII dump above, I start with two instances of the FixedArray2D object.

local framebuffer = FixedArray2D(20, 20, "unsigned char", string.byte("#"))
local window = FixedArray2D(10,10, "unsigned char",string.byte("O"))

As you might imagine, FixedArray2D is an object that manages what appears to be a 2 dimensional array of elements of a specified type.  Any sane person might just call this a matrix, or a pixel buffer, but I tried to be more explicit in what it actually is.

There are a couple of things to note about this simple object.  First, the construction parameters are:

Columns, Rows, element type, initial value

The element type allows you to specify any type at all.  The only caveat is that the type must be something that the LuaJIT FFI knows about.  Since my library includes some more interesting types, I could have just as easily specified “pixel_rgb_b”, to get an array of rgb pixels with 8 bits per component.  Or, if I’m working with some grayscale image: “pixel_lum_b”.  In the case of the ASCII art, I just went with “unsigned char”, which is similar to “pixel_lum_b”.

What I have after that is an object that contains exactly the number of bytes specified.  It is not a conversion of ‘number’ it is actually an array of bytes.

So, I’ve created a “framebuffer”, and a “window”.  My intention is to have the framebuffer represent the entirety of my graphics display.  The “window” is smaller, and can represent an image that I want to copy to the frame buffer.

Before I do the “blit” of the window into the frame buffer, I want to do some drawing into the window.  First I’ll draw a frame:

function frameRect(dst, value)
for i=0,dst.Width-1 do
dst:Set(i,0, string.byte(“-“))
dst:Set(i,dst.Height-1,string.byte(“_”))
end
— Draw vertical sides
for i=0,(dst.Height-1) do
dst:Set(0,i, string.byte(“|”))
dst:Set(dst.Width-1, i, string.byte(“|”))
end
end

That should be easy enough to understand.  First draw the horizontal lines at the top and bottom, then draw the vertical lines on the left and right sides.  The FixedArray2D object has a Set(x, y, value) method.  This is where Lua is very nice and handy.  I do NOT need to create a different variant of this Set() method for every type of array that I create.  In this sense, it is like C++ templates, except there’s not the attendant code bloat and confusing looking syntax.  Just set the value, and move on.

Then I want to draw a diagonal across the window:

-- Draw diagonal line on window
diagonal = EFLA_Iterator(0,0,window.Width-1,window.Width-1)
for x,y in diagonal do
window:Set(x,y,string.byte("\\"))
end

I really like that little snippet because it uses the Extra Fast Line Algorithm as an iterator, and then just sets values in the window array.  This will be great fun later when it comes time to rasterize polygons and such.  Again, the EFLA does not do any drawing itself.  It just iterates, returning the list of coordinates that something interesting should happen with.  It also returns a percent complete (between 0 and 1), which I am not using here.  That can be extremely useful when you’re trying to draw a line where the color is supposed to interpolate from one end to the other.  Again, no need to implement a special version of EFLA.  Just change how you use the results of the iterations.

OK.  So, up to this point, there is a frame buffer, that is filled with the ‘#’ character, and there is this window that is filled with the ‘O’ character, is framed, and has a diagonal drawn across it.

Now I need to execute the copy:

TransferArray2D(framebuffer, window, 5,2)

And that’s it!  The first argument is the destination of the copy, our framebuffer.  The second argument is the source that we want to copy from, our window.  The next two are the destination point within the frame buffer, with a default of {0,0}.

Yipee!  Pop the corks on the champagne!!!  But wait, there’s more.  Why is that function called “TransferArray2D”?, and not simply Blit, or Copy, or DrawBitmap?  Because, this is more explicit.  At the very simplest, the function will simply copy they source to the destination.  If this were truly a graphics system, it would be a SRCCOPY operation.  But, there are other operations.  There are various forms of blending, using transparency, and any manner of things that can occur.  The full flexibility of the function is this:

function TransferArray2D(dst, src,  dstX, dstY, srcBounds, driver, elementOp)

Previously, I just used the dst, src, dstX, dstY parameters.  The others were left as defaults.  The default for the srcBounds (the boundary of the source) will be the entire source.  You can select a subset of the source to copy, thus copying a small chunk of the source, rather than the entire thing.  That’s great when you have a small piece of a window that is changing, and you don’t need to copy the whole thing, just the part that has changed.

The next two are interesting.  The driver is the “inner loop” if you will of the operation.  The guts of the TransferArray2D function looks like this:

driver = driver or CopyRect
local targetFrame, dstFrame, srcRect  = CalculateTargetFrame(   dstX, dstY, dst.Width, dst.Height,   src.Width, src.Height, srcBounds)
driver(dst, src, targetFrame, dstFrame, srcRect, elementOp)

Basically, do the work to setup the destination and source rectangles, then call the driver to actually perform the transfer. As you can see, the default driver is “CopyRect”, which is a routine defined to simply go element by element, performing a copy. Since it can do a copy, it’s actually really fast because it performs a straight memory copy (supported by LuaJIT FFI).

There is another simple driver available though. That is the ComposeRect() driver. And this is where that last parameter of the TransferArray2D object comes in. ComposeRect will go element by element, and call the elementOp parameter if specified. If it’s not specified, then the default will again be a simple copy. But if it is specified, the transferOp function will receive both the source and destination elements, and it simply returns a result, performing whatever operations it wants to.

To demonstrate, here’s another picture, where I’ve utilized a ‘threshold’ operation:

In this case, the TransferArray2D call looks like this:

function threshold(dst, src)  if src >120 then return src end  return nil end

TransferArray2D(framebuffer, window, 5,2,RectI(0,0,window.Width,window.Height),ComposeRect, threshold)

So, I’m using the ComposeRect driver, and this simple threshold function to limit the values of pixels that get passed through.  Returning a nil value will essentially leave the destination pixel in place.  You could easily create a stencil operation like this, or do whatever else you wanted, including procedural textures.

The function does deal with the cases where the source is going off the edge of the framebuffer, or overlapping, or not intersecting at all.  It’s a real work horse of a routine, and not very large at all.  One of the keys buried deep within the implementation is a simple rectangle intersection calculation.  Once you have that, everything else becomes easy.

The FixedArray2D object is very versatile.  It can do anything from represent ascii art, to a bitmap, to a matrix, vector, point, or whatever.  All in one tidy little package.  Since it uses the LuaJIT FFI, you have very precise control of the data types that are stored in the array.  This is great for composability.  Similarly, the graphics system is structured in such a way that there are some core primitives that are meant to be composable to form more complex, or simple interfaces.  Rather than just one massive all singing all dancing graphics API, there are bits and pieces that perform various operations.  These bits and pieces can be composed into graphics APIs that make the most sense for a particular application’s needs.

---