Spelunking Linux – Decomposing systemd

Honestly, I don’t know what all the fuss is about. What is systemd?  It’s that bit of code that gets things going on your Linux machine once the kernel has loaded itself.  You know, dealing with bringing up services, communicating between services, running the udev and dbus stuff, etc.

So, I wrote an ffi wrapper for the libsystemd.so library This has proven to be handy, as usual, I can essentially write what looks like standard C code, but it’s actually LuaJIT goodness.

	Test using SDJournal as a cursor over the journal entries

	In this case, we want to try the various cursor positioning
	operations to ensure the work correctly.
package.path = package.path..";../src/?.lua"

local SDJournal = require("SDJournal")
local sysd = require("systemd_ffi")

local jnl = SDJournal()

-- move forward a few times
for i=1,10 do

-- get the cursor label for this position
local label10 = jnl:positionLabel()
print("Label 10: ", label10)

-- seek to the beginning, print that label
local label1 = jnl:positionLabel();
print("Label 1: ", label1);

-- seek to label 10 again
local label3 = jnl:positionLabel();
print("Label 3: ", label3)
print("label 3 == label 10: ", label3 == label10)

In this case, a simple journal object which makes it relatively easy to browse through the systemd journals that are laying about. That’s handy. Combined with the luafun functions, browsing through journals suddenly becomes a lot easier, with the full power of lua to form very interesting queries, or other operations.

	Test cursoring over journal, turning each entry
	into a lua table to be used with luafun filters and whatnot
package.path = package.path..";../src/?.lua"

local SDJournal = require("SDJournal")
local sysd = require("systemd_ffi")
local fun = require("fun")()

-- Feed this routine a table with the names of the fields
-- you are interested in seeing in the final output table
local function selection(fields, aliases)
	return function(entry)
		local res = {}
		for _, k in ipairs(fields) do
			if entry[k] then
				res[k] = entry[k];
		return res;

local function  printTable(entry)
	each(print, entry)

local function convertCursorToTable(cursor)
	return cursor:currentValue();

local function printJournalFields(selector, flags)
	flags = flags or 0
	local jnl1 = SDJournal();

	if selector then
		each(printTable, map(selector, map(convertCursorToTable, jnl1:entries())))
		each(printTable, map(convertCursorToTable, jnl1:entries()))	

-- print all fields, but filter the kind of journal being looked at
--printJournalFields(nil, sysd.SD_JOURNAL_CURRENT_USER)
--printJournalFields(nil, sysd.SD_JOURNAL_SYSTEM)

-- printing specific fields
--printJournalFields(selection({"_HOSTNAME", "SYSLOG_FACILITY"}));
printJournalFields(selection({"_EXE", "_CMDLINE"}));

-- to print all the fields available per entry

In this case, we have a simple journal printer, which will take a list of fields, as well as a selection of the kinds of journals to look at. That’s quite useful as you can easily generate JSON or XML, or Lua tables on the output end, without much work. You can easily select which fields you want to display, and you could even change the names along the way. You have the full power of lua at your disposal to do whatever you want with the data.

In this case, the SDJournal object is pretty straight forward. It simply wraps the various ‘sd_xxx’ calls within the library to get its work done. What about some other cases? Does the systemd library need to be used for absolutely everything that it does? The answer is ‘no’, you can do a lot of the work yourself, because at the end of the day, the passive part of systemd is just a bunch of file system manipulation.

Here’s where it gets interesting in terms of decomposition.

Within the libsystemd library, there is the sd_get_machine_names() function:

_public_ int sd_get_machine_names(char ***machines) {
        char **l = NULL, **a, **b;
        int r;

        assert_return(machines, -EINVAL);

        r = get_files_in_directory("/run/systemd/machines/", &l);
        if (r < 0)
                return r;

        if (l) {
                r = 0;

                /* Filter out the unit: symlinks */
                for (a = l, b = l; *a; a++) {
                        if (startswith(*a, "unit:") || !machine_name_is_valid(*a))
                        else {
                                *b = *a;

                *b = NULL;

        *machines = l;
        return r;

The lua wrapper for this would simply be:

ffi.cdef("int sd_get_machine_names(char ***machines)")

Great, for those who already know this call, you can allocate a char * array, get the array of string values, and party on. But what about the lifetime of those strings, and if you’re doing it as an iterator, when do you ever free stuff, and isn’t this all wasteful?

So, looking at that code in the library, you might think, ‘wait a minute, I could just replicate that in Lua, and get it done without doing any ffi stuff at all!

local fun = require("fun")

local function isNotUnit(name)
	return not strutil.startswith(name, "unit:")

function SDMachine.machineNames(self)
	return fun.filter(isNotUnit, fsutil.files_in_directory("/run/systemd/machines/"))

OK, that looks simple. But what’s happening with that ‘files_in_directory()’ function? Well, that’s the meat and potatoes of this operation.

local function nil_iter()
    return nil;

-- This is a 'generator' which will continue
-- the iteration over files
local function gen_files(dir, state)
    local de = nil

    while true do
       de = libc.readdir(dir)
        -- if we've run out of entries, then return nil
        if de == nil then return nil end

    -- check the entry to see if it's an actual file, and not
    -- a directory or link
        if dirutil.dirent_is_file(de) then

    local name = ffi.string(de.d_name);

    return de, name

local function files_in_directory(path)
    local dir = libc.opendir(path)

    if not dir==nil then return nil_iter, nil, nil; end

    -- make sure to do the finalizer
    -- for garbage collection
    ffi.gc(dir, libc.closedir);

    return gen_files, dir, initial;

In this case, files_in_directory() takes a string path, like “/run/systemd/machines”, and just iterates over the directory, returning only the files found there. It’s convenient in that it will skip so called ‘hidden’ files, and things that are links. This simple technique/function can be the cornerstone of a lot of things that view files in Linux. The function leverages the libc opendir(), readdir(), functions, so there’s nothing new here, but it wraps it all up in this convenient iterator, which is nice.

systemd is about a whole lot more than just browsing directories, but that’s certain a big part of it. When you break it down like this, you begin to realize that you don’t actually need to use a ton of stuff from the library. In fact, it’s probably better and less resource intensive to just ‘go direct’ where it makes sense. In this case, it was just implementing a few choice routines to make file iteration work the same as it does in systemd. As this binding evolves, I’m sure there is other low lying fruit that I’ll be able to pluck to make it even more interesting, useful, and independent of the libsystemd library.

Spelunking Linux – Yes, the system truly is a database

In this article: Isn’t the whole system just a database? – libdrm, I explored a little bit of the database nature of Linux by using libudev to enumerate and open libdrm devices.  After that, I spent some time bringing up a USB module: LJIT2libusb.  libusb is a useful cross platform library that makes it relatively easy to gain access to the usb functions on multiple platforms.  It can enumerate devices, deal with hot plug notifications, open up, read, write, etc.

At its core, on Linux at least, libusb tries to leverage the uvdev capabilities of the target system, if those capabilities are there.  This means that device enumeration and hot plugging actually use the libuvdev stuff.  In fact, the code for enumerating those usb devices in libusb looks like this:


	udev_enumerate_add_match_subsystem(enumerator, "usb");
	udev_enumerate_add_match_property(enumerator, "DEVTYPE", "usb_device");
	devices = udev_enumerate_get_list_entry(enumerator);

There’s more stuff of course, to turn that into data structures which are appropriate for use within the libusb view of the world. But, here’s the equivalent using LLUI and the previously developed UVDev stuff:

local function isUsbDevice(dev)
	if dev.IsInitialized and dev:getProperty("subsystem") == "usb" and
		dev:getProperty("devtype") == "usb_device" then
		return true;

	return false;

each(print, filter(isUsbDevice, ctxt:devices()))

It’s just illustrative, but it’s fairly simple to understand I think. The ‘ctxt:devices()’ is an iterator over all the devices in the system. The ‘filter’ function is part of the luafun functional programming routines available to Lua. the ‘isUsbDevice’ is a predicate function, which returns ‘true’ when the device in question matches what it believes makes a device a ‘usb’ device. In this case, its the subsystem and dev_type properties which are used.

Being able to easily query devices like this makes life a heck of a lot easier. No funky code polluting my pure application. Just these simple query predicates written in Lua, and I’m all set. So, instead of relying on libusb to enumerate my usb devices, I can just enumerate them directly using uvdev, which is what the library does anyway. Enumeration and hotplug handing is part of the library. The other part is the actual send and receiving of data. For that, the libusb library is still primarily important, as replacing that code will take some time.

Where else can this great query capability be applied? Well, libudev is just a nice wrapper atop sysfs, which is that virtual file system built into Linux for gaining access to device information and control of the same. There’s all sorts of stuff in there. So, let’s say you want to list all the block devices?

local function isBlockDevice(dev)
	if dev.IsInitialized and dev:getProperty("subsystem") == "block" then
		return true;

	return false;

That will get all the devices which are in the subsystem “block”. That includes physical disks, virtual disks, partitions, and the like. If you’re after just the physical ones, then you might use something like this:

local function isPhysicalBlockDevice(dev)
	if dev.IsInitialized and dev:getProperty("subsystem") == "block" and
		dev:getProperty("devtype") == "disk" and
		dev:getProperty("ID_BUS") ~= nil then
		return true;

	return false;

Here, a physical device is indicated by subsystem == ‘block’ and devtype == ‘disk’ and the ‘ID_BUS’ property exists, assuming any physical disk would show up on one of the system’s buses. This won’t catch a SD card though. For that, you’d use the first one, and then look for a property related to being an SD card. Same goes for ‘cd’ vs ramdisk, or whatever. You can make these queries as complex or simple as you want.

Once you have a device, you can simply open it using the “SysName” parameter, handed to an fopen() call.

I find this to be a great way to program. It makes the creation of utilities such as ‘lsblk’ relatively easy. You would just look for all the block devices and their partitions, and put them into a table. Then separately, you would have a display routine, which would consume the table and generate whatever output you want. I find this much better than the typical Linux tools which try to do advanced display using the terminal window. That’s great as far as it goes, but not so great if what you really want is a nice html page generated for some remote viewing.

At any rate, this whole libudev exploration is a great thing. You can list all devices easily, getting every bit of information you care to examine. Since it’s all scriptable, it’s fairly easy to taylor your queries on the fly, looking at, discovering, and the like. I discovered that the thumb print reader in my old laptop was made by Broadcom, and my webcam by 3M? It’s just so much fun.

Well there you have it. The more you spelunk, the more you know, and the more you can fiddle about.

Isn’t the whole system just a database? – libdrm

Do enough programming, and everything looks like a database of one form or another. Case in point, when you want to get keyboard and mouse input, you first have to query the system to see which of the /dev/input/eventxxx devices you want to open for your particular needs. Yes, there are convenient shortcuts, but that’s beside the point.

Same goes with other devices in the system. This time around, I want to find the drm device which represents the graphics card in my system (from the libdrm perspective).

In LJIT2libudev there are already objects that make it convenient to enumerate all the devices in the system using a simple iterator:

local ctxt, err = require("UDVContext")()
for _, dev in ctxt:devices() do

Well, that’s find and all, but let’s get specific. To use the libdrm library, I very specifically need one of the active devices in the ‘drm’ subsystem. I could write this:

local ctxt, err = require("UDVContext")()

local function getActiveDrm()
  local function isActiveDrm(dev)
    if dev.IsInitialized and dev:getProperty("subsystem") == "drm" then
      return true;

    return false;

  for _, dev in ctxt:devices() do
    if isActiveDrm(dev) then
      return dev;

  return nil;

local device = getActiveDrm()

Yah, that would work. Then of course, when I want to change the criteria for finding the device I’m looking for, I would change up this code a bit. The core iterator is the key starting point at least. The ‘isActiveDrm()’ is a function which acts as a predicate to filter through the results, and only return the ones I want.

Since this is Lua though, and since there is a well throught out functional programming library already (luafun), this could be made even easier:

local function isActiveDrm(dev)
  if dev.IsInitialized and dev:getProperty("subsystem") == "drm" then
    return true;

  return false;

local device = head(filter(isActiveDrm, ctxt:devices()))
assert(device, "could not find active drm device")

In this case, we let the luafun ‘filter’ and ‘head’ functions do their job of dealing with the predicate, and taking the first one off the iterator that matches and returning it. Now, changing my criteria is fairly straight forward. Just change out the predicate, and done. This is kind of nice, particularly with Lua, because that predicate is just some code, it could be generated at runtime because we’re in script right?

So, how about this version:

-- File: IsActiveDrmDevice.lua
-- predicate to determine if a device is a DRM device and it's active
return function(dev)
  if dev.IsInitialized and dev:getProperty("subsystem") == "drm" then
    return true;

  return false;

-- File: devices_where.lua
#!/usr/bin/env luajit

-- devices_where.lua
-- print devices in the system, filtered by a supplied predicate
-- generates output which is a valid lua table
package.path = package.path..";../?.lua"

local fun = require("fun")()
local utils = require("utils")

local ctxt, err = require("UDVContext")()
assert(ctxt ~= nil, "Error creating context")

if #arg < 1 then
  error("you must specify a predicate")

local predicate = require(arg[1])

each(utils.printDevice, filter(predicate, ctxt:devices()))


-- Actual usage from the command line
./devices_where.lua isActiveDrmDevice

In this case, the ‘query’ has been generalized enough such that you can pass a predicate as a filename (minus the ‘.lua’). The code for the predicate will be compiled in, and used as the predicate for the filter() function. Well, that’s pretty nifty I think. And since the query itself again is just a bit of code, that can be changed on the fly as well. I can easily see a system where lua is the query language, and the entire machine is the database.

The tarantool database is written in Lua, and I believe the luafun code is used there. Tarantool is not a system database, but the fact that it’s written in Lua itself is interesting, and just proves the case that Lua is a good language for doing some database work.

I have found that tackling the lowest level enumeration by putting a Lua iterator on top of it makes life a whole lot easier. With many of the libraries that you run across, they spend a fair amount of resources/code on trying to make things look like a database. In the case of libudev, there are functions for iterating their internal hash table of values, routines for creating ‘enumerators’ which are essentially queries, routines for getting properties, routines for turning properties into more accessible strings, routines for turning the ‘FLAGS’ property into individual values, and the like, and then there’s the memory management routines (ref, unref). A lot of that stuff either goes away, or is handled much more succinctly when you’re using a language such as Lua, or JavaScript, or Python, Ruby, whatever, as long as it’s modern, dynamic, and has decent enough higher level memory managed libraries.

And thus, the whole system, from log files, to perf counters, to device lists, is a database, waiting to be harvested, and made readily available.

Functional Programming with libevdev

Previously, I wrote about the LuaJIT binding to libevdev: LJIT2libevdev – input device tracking on Linux

In that case, I went over the basics, and it all works out well, and you get get at keyboard and mouse events from Lua, with very little overhead.  Feels natural, light weight iterators, life is good.

Recently, I wanted to go one step further though.  Once you go down the path of using iterators, you begin to think about functional programming.  At least that’s how I’m wired.  So, I’ve pushed it a bit further, and incorporated the fun.lua module to make things a bit more interesting.  fun.lua provides things like each, any, all, map, filter, and other stuff which makes dealing with streams of data really brainlessly simple.

Here’s an example of spewing out the stream of events for a device:


package.path = package.path..";../?.lua"

local EVEvent = require("EVEvent")
local dev = require("EVDevice")(arg[1])

local utils = require("utils")
local fun = require("fun")

-- print out the device particulars before 
-- printing out the stream of events
print("===== ===== =====")

-- perform the actual printing of the event
local function printEvent(ev)
    print(string.format("{'%s', '%s', %d};",ev:typeName(),ev:codeName(),ev:value()));

-- decide whether an event is interesting enough to 
-- print or not
local function isInteresting(ev)
	return ev:typeName() ~= "EV_SYN" and ev:typeName() ~= "EV_MSC"

-- convert from a raw 'struct input_event' to the EVEvent object
local function toEVEvent(rawev)
    return EVEvent(rawev)


The last line there, where it starts with “fun.each”, can be read from the inside out.

dev:rawEvents() – is an iterator, it returns a steady stream of events coming off of the specified device.

fun.map(toEVEvent,…) – the map function will take some input, run it through the provided function, and send that as the output. In functional programming, this would be transformation.

fun.filter(isInteresting, …) – filter takes an input, allplies the specified predicate, and allows that item to pass through or not based on the predicate returning true or false.

fun.each(printEvent, …) – the ‘each’ function takes each of the items coming from the stream of items, and applies the specified function, in this case the printEvent function.

This is a typical pull chain. The events are pulled out of the lowest level iterator as they are needed by subsequent operations in the chain. If the chain stops, then nothing further is pulled.

This is a great way to program because doing different things with the stream of events is simply a matter of replacing items in the chain. For example, if we just want the first 100 items, we could write

        map(toEVEvent, dev:rawEvents()))));

You can create tees, send data elsewhere, construct new streams by combining streams. There are a few key operators, and with them you can do a ton of stuff.

At the very heart, the EVDevice:rawEvents() iterator looks like this:

function EVDevice.rawEvents(self)
	local function iter_gen(params, flags)
		local flags = flags or ffi.C.LIBEVDEV_READ_FLAG_NORMAL;
		local ev = ffi.new("struct input_event");
		--local event = EVEvent(ev);
		local rc = 0;

			rc = evdev.libevdev_next_event(params.Handle, flags, ev);
		until rc ~= -libc.EAGAIN
			return flags, ev;

		return nil, rc;

	return iter_gen, {Handle = self.Handle}, state 

This is better than the previous version where we could pass in a predicate. In this case, we don’t even convert to the EVEvent object at this early low level stage, because we’re not sure what subsequent links in the chain will want to do, so we leave it out. This simplifies the code at the lowest levels, and makes it more composable, which is a desirable effect.

And so it goes. This binding started out as a simple ffi.cdef hack job, then objects were added to encapsulate some stuff, and now the iterators are being used in a functional programming style, which makes the whole thing that much more useful and integratable.

LJIT2pixman – Drawing on Linux

On the Linux OS, libpixman is at the heart of doing some low level drawing. It’s very simple stuff like compositing, rendering of gradients and the like. It takes up the slack where hardware acceleration doesn’t exist. So, graphics libraries such as Cairo leverage libpixman at the bottom to take care of the basics. LJIT2pixman is a little project that delivers a fairly decent LuaJIT binding to that library.

Here’s one demo of what it can do.


Of course, given the title of this post, you know LuaJIT is involved, so you can expect that there’s some way of doing this in LuaJIT.

package.path = package.path..";../?.lua"

local ffi = require("ffi")
local bit = require("bit")
local band = bit.band

local pixman = require("pixman")()
local pixlib = pixman.Lib_pixman;
local ENUM = ffi.C
local utils = require("utils")
local save_image = utils.save_image;

local function D2F(d) return (pixman_double_to_fixed(d)) end

local function main (argc, argv)

	local WIDTH = 400;
	local HEIGHT = 400;
	local TILE_SIZE = 25;

    local trans = ffi.new("pixman_transform_t", { {
	    { D2F (-1.96830), D2F (-1.82250), D2F (512.12250)},
	    { D2F (0.00000), D2F (-7.29000), D2F (1458.00000)},
	    { D2F (0.00000), D2F (-0.00911), D2F (0.59231)},

    local checkerboard = pixlib.pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8,
					     WIDTH, HEIGHT,
					     nil, 0);

    local destination = pixlib.pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8,
					    WIDTH, HEIGHT,
					    nil, 0);

    for i = 0, (HEIGHT / TILE_SIZE)-1  do
		for j = 0, (WIDTH / TILE_SIZE)-1 do
	    	local u = (j + 1) / (WIDTH / TILE_SIZE);
	    	local v = (i + 1) / (HEIGHT / TILE_SIZE);
	    	local black = ffi.new("pixman_color_t", { 0, 0, 0, 0xffff });
	    	local white = ffi.new("pixman_color_t", {
				v * 0xffff,
				u * 0xffff,
				(1 - u) * 0xffff,
				0xffff });
	    	local c = white;

	    	if (band(j, 1) ~= band(i, 1)) then
				c = black;

	    	local fill = pixlib.pixman_image_create_solid_fill (c);

	    	pixlib.pixman_image_composite (ENUM.PIXMAN_OP_SRC, fill, nil, checkerboard,
				    0, 0, 0, 0, j * TILE_SIZE, i * TILE_SIZE,

    pixlib.pixman_image_set_transform (checkerboard, trans);
    pixlib.pixman_image_set_filter (checkerboard, ENUM.PIXMAN_FILTER_BEST, nil, 0);
    pixlib.pixman_image_set_repeat (checkerboard, ENUM.PIXMAN_REPEAT_NONE);

    pixlib.pixman_image_composite (ENUM.PIXMAN_OP_SRC,
			    checkerboard, nil, destination,
			    0, 0, 0, 0, 0, 0,
			    WIDTH, HEIGHT);

	save_image (destination, "checkerboard.ppm");

    return true;

main(#arg, arg)

With a couple of exceptions, the code looks almost exactly like its C based counterpart. I actually think this is a very good thing, because you can rapidly prototype something from a C coding example, but have all the support and protection that a dynamic language such as lua provides.

And here’s another:


In this case is the conical-test.lua demo doing the work.

package.path = package.path..";../?.lua"

local ffi = require("ffi")
local bit = require("bit")
local band = bit.band
local lshift, rshift = bit.lshift, bit.rshift

local pixman = require("pixman")()
local pixlib = pixman.Lib_pixman;
local ENUM = ffi.C
local utils = require("utils")
local save_image = utils.save_image;
local libc = require("libc")

local SIZE = 128
local NUM_GRADIENTS = 35


local function double_to_color(x)
    return (x*65536) - rshift( (x*65536), 16)

local function PIXMAN_STOP(offset,r,g,b,a)		
   return ffi.new("pixman_gradient_stop_t", { pixman_double_to_fixed (offset),		
	    double_to_color (r),		
		double_to_color (g),		
		double_to_color (b),		
		double_to_color (a)		

local stops = ffi.new("pixman_gradient_stop_t[4]",{
    PIXMAN_STOP (0.25,       1, 0, 0, 0.7),
    PIXMAN_STOP (0.5,        1, 1, 0, 0.7),
    PIXMAN_STOP (0.75,       0, 1, 0, 0.7),
    PIXMAN_STOP (1.0,        0, 0, 1, 0.7)

local  NUM_STOPS = (ffi.sizeof (stops) / ffi.sizeof (stops[0]))

local function create_conical (index)
    local c = ffi.new("pixman_point_fixed_t")
    c.x = pixman_double_to_fixed (0);
    c.y = pixman_double_to_fixed (0);

    local angle = (0.5 / NUM_GRADIENTS + index / NUM_GRADIENTS) * 720 - 180;

    return pixlib.pixman_image_create_conical_gradient (c, pixman_double_to_fixed (angle), stops, NUM_STOPS);

local function main (argc, argv)

    local transform = ffi.new("pixman_transform_t");

    local dest_img = pixlib.pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8,
					 nil, 0);
    utils.draw_checkerboard (dest_img, 25, 0xffaaaaaa, 0xff888888);

    pixlib.pixman_transform_init_identity (transform);

    pixlib.pixman_transform_translate (NULL, transform,
				pixman_double_to_fixed (0.5),
				pixman_double_to_fixed (0.5));

    pixlib.pixman_transform_scale (nil, transform,
			    pixman_double_to_fixed (SIZE),
			    pixman_double_to_fixed (SIZE));
    pixlib.pixman_transform_translate (nil, transform,
				pixman_double_to_fixed (0.5),
				pixman_double_to_fixed (0.5));

    for i = 0, NUM_GRADIENTS-1 do
	   local column = i % GRADIENTS_PER_ROW;
	   local row = i / GRADIENTS_PER_ROW;

	   local src_img = create_conical (i); 
	   pixlib.pixman_image_set_repeat (src_img, ENUM.PIXMAN_REPEAT_NORMAL);
	   pixlib.pixman_image_set_transform (src_img, transform);
	   pixlib.pixman_image_composite32 (
	       ENUM.PIXMAN_OP_OVER, src_img, nil,dest_img,
	       0, 0, 0, 0, column * SIZE, row * SIZE,
	       SIZE, SIZE);
	   pixlib.pixman_image_unref (src_img);

    save_image (dest_img, "conical-test.ppm");

    pixlib.pixman_image_unref (dest_img);

    return true;

main(#arg, arg)


Linear Gradient demo


screen demo (transparency).
Perhaps this is the easiest one of all. All the interesting functions are placed into the global namespace, so they can be accessed easily, just like everything in C is globally available.

package.path = package.path..";../?.lua"

local ffi = require("ffi")
local bit = require("bit")
local band = bit.band
local lshift, rshift = bit.lshift, bit.rshift

local pixman = require("pixman")()
local pixlib = pixman.Lib_pixman;
local ENUM = ffi.C
local utils = require("utils")
local save_image = utils.save_image;
local libc = require("libc")

local function main (argc, argv)

    WIDTH = 40
    HEIGHT = 40
    local src1 = ffi.cast("uint32_t *", libc.malloc (WIDTH * HEIGHT * 4));
    local src2 = ffi.cast("uint32_t *", libc.malloc (WIDTH * HEIGHT * 4));
    local src3 = ffi.cast("uint32_t *", libc.malloc (WIDTH * HEIGHT * 4));
    local dest = ffi.cast("uint32_t *", libc.malloc (3 * WIDTH * 2 * HEIGHT * 4));

    for i = 0, (WIDTH * HEIGHT)-1 do
	   src1[i] = 0x7ff00000;
	   src2[i] = 0x7f00ff00;
	   src3[i] = 0x7f0000ff;

    for i = 0, (3 * WIDTH * 2 * HEIGHT)-1 do
	   dest[i] = 0x0;

    local simg1 = pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8, WIDTH, HEIGHT, src1, WIDTH * 4);
    local simg2 = pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8, WIDTH, HEIGHT, src2, WIDTH * 4);
    local simg3 = pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8, WIDTH, HEIGHT, src3, WIDTH * 4);
    local dimg  = pixman_image_create_bits (ENUM.PIXMAN_a8r8g8b8, 3 * WIDTH, 2 * HEIGHT, dest, 3 * WIDTH * 4);

    pixman_image_composite (ENUM.PIXMAN_OP_SCREEN, simg1, NULL, dimg, 0, 0, 0, 0, WIDTH, HEIGHT / 4, WIDTH, HEIGHT);
    pixman_image_composite (ENUM.PIXMAN_OP_SCREEN, simg2, NULL, dimg, 0, 0, 0, 0, (WIDTH/2), HEIGHT / 4 + HEIGHT / 2, WIDTH, HEIGHT);
    pixman_image_composite (ENUM.PIXMAN_OP_SCREEN, simg3, NULL, dimg, 0, 0, 0, 0, (4 * WIDTH) / 3, HEIGHT, WIDTH, HEIGHT);

    save_image (dimg, "screen-test.ppm");
    return true;

main(#arg, arg)

I really like this style of rapid prototyping. The challenge I have otherwise is that it’s just too time consuming to consume things in their raw C form. Things like build systems, compiler versions, and other forms of magic always seem to get in the way. And if it’s not that stuff, it’s memory management, and figuring out the inevitable crashes.

Once you wrap a library up in a bit of lua goodness though, it becomes much more approachable. It may or may not be the most performant thing in the world, but you can worry about that later.

Having this style of rapid prototyping available saves tremendous amounts of time. Since you’re not wasting your time on the mundane (memory management, build system, compiler extensions and the like), you can spend much more time on doing mashups of the various pieces of technology at hand.

In this case it was libpixman. Previously I’ve tackled everything from hot plugging usb devices, to consuming keystrokes, and putting a window on the screen.

What’s next? Well, someone inquired as to whether I would be doing a Wayland binding. My response was essentially, “since I now have all the basics, there’s no need for wayland, I can just call what it was going to call directly.

And so it goes. One more library in the toolbox, more interesting demos generated.

LJIT2libc – LuaJIT IS “batteries included”

I would say one of the most common complaints about any platform, framework, product is a paucity of available stuff to fiddle about with.  I’ve heard this criticism a few times leveled at the lua world in general.  Fatter frameworks are usually “batteries included”.  That’s great, when you get a whole ton of stuff that makes writing all sorts of apps relatively easy to do right off the bat.  The challenge of “batteries included” is that you get a whole ton of stuff, most of which you will never use, and some of which doesn’t have the best implementation.

Recently, I’ve been doing quite a lot off luajit bindings on the Linux platform:

If you were into anything from ASCII art graphics drivers to raw frame buffer management in a Linux system, these might start to look like batteries.  They’re not included with the luajit compiler, but they’re relatively easy to add.

But, there’s another one that I’ve been playing with recently which I think is even better:

Luajit is typically built and compiled against the libc and libm libraries.  As such, being able to access routines within those libraries comes for ‘free’ from within luajit, courtesy of the ffi capabilities.  This is very interesting because it means these routines, which are already on your system, are in fact just a stones throw away from being available within your app.
Let’s imagine you wanted to write some script like this, using the libc provided ‘puts()’ function:
puts("Hello, World!");

Well, the lua language has its own print routine, so this is a bit contrived, but let’s say you wanted to do it anyway. Well, to make this work, you need access to the function signature for the ‘puts’ routine and you need that to be accessible in the global namespace. So, it would actually look like this:

local ffi = require("ffi")

ffi.cdef("int puts(const char *);");
puts = ffi.C.puts;

puts("Hello, World!")

Great. A bit of work, and now I can program as wrecklessly as I could in C with all the batteries included in the libc and libm libraries. But, it gets tedious having to write that ffi stuff all over the place, so this is where the LJIT2libc thing comes in. It basically goes and implements a ton of the required ffi.cdef statements to make all those functions readily available to your script. An easy way to pull it into any script is to do the following:

local init = require("test_setup")()

puts("Hello, World!");

That first line ‘init = …’ in this case pulls in all of the definitions, and puts them into the global namespace, so that you can simply write your program without having to know anything about the ffi, or where functions come from or anything like that. Just start writing your code knowing that the batteries are already included.

Now, this example might seem too trivial for words, but just think about what’s in a typical libc library. It’s all the file system, sockets, system calls, math, random numbers, everything you’re likely to need to create higher level application stuff. It’s a lot of stuff that other systems end up creating from scratch, as part of their ‘batteries’.

Of course this is trivializing what’s happening in those batteries, because what you’re getting here is the raw C implementations of things, with all the headaches and dangers that are typically associated with writing code at that level. But, for those who want to have access to that level of code, and apply the safety net of lua where they see fit, this is quite a useful tool I think.

In short, batteries ARE included with every luajit. Those batteries might be just anode, cathode, and electrolyte, without the shell, but there are the raw ingredients. Having these wrappers available just makes it that much easier to think about and deal with a lot of low level stuff where I might previously had to resort to some sort of ‘package’ to achieve something as simple as traverse the file system.

So there you have it. luajit is a ‘batteries included’ system.


Get every new post delivered to your Inbox.

Join 55 other followers