A New Way of Programming Systems

Just a couple days back, I became aware of this project called LJSyscall.  From it’s GitHub page:

What? An FFI implementation of the Linux kernel ABI for LuaJIT.

For me, this is one of those moments in programming history that when you look back, from a distance of 10 years or so, you say “and that’s when things changed”.  So what is this?  Well, if you’re a Linux programmer, and particularly if you’re not, programming against the low level kernel ABI just became that much easier.  Now granted, most people don’t program against the low level Kernel ABI, but rather stick to various frameworks way above this level.  Most apps I see these days on Linux seem to utilize Node.js, or Python frameworks, or Qt for UI, or whatever, so the amount of low level coding is done by a few, but what do I know really?

What’s truly interests me here is the way in which very challenging things are becoming relatively easy.  Take a simple example, again from the GitHub page:

File descriptors are returned as a type not an integer. This is because they are garbage collected by default, ie if they go out of scope the file is closed. You can get the file descriptor using the fileno field. To disable the garbage collection you can call fd:nogc(), in which case you need to close the descriptors by hand. They also have methods for operations that take an fd, like close, fsync, read. You can use this type where an fd is required, or a numeric fd, or a string like “stderr”.

 It seems like a simple thing, but think about it.  How many bugs and/or concerns crop up in code because somewhere someone didn’t properly shut down a system resource.  Resource leaks, finding and plugging them, is the bain of any system level high performance server’s existance.  So, basically, with LJSyscall, these issues become moot.  It’s just the smallest application of garbage collection, while maintaining what makes sense for the overall system, such that performance is not hurt.

This is not simply a FFI wrapper on the Linux kernel calls (although it is that as well).  Instead, it is a well thought out module that makes programming against the Linux kernel painlessly easy.  I think this bodes well for many projects.  If I wander over to the world of Raspberry Pi, for example, I could easily see playing about with a Pi using LJSyscall, and even showing grade school kids how to do the same.  In the plain C form, I couldn’t imagine trying to teach grade school kinds how to program against the Linux kernel.

All this goodness comes in about 4500 lines of code, which when you think of the many megabytes taken up by the pictures typical on a web page, this is nothing.  Another benefit accrues due to the nature of Lua itself.  Since it’s relatively easy to embed Lua into other languages, including other scripting environments, LJSyscall could define the interface to the Linux Kernel, period.  Why bother writing a bunch of C code wrappers to integrate Linux system calls into Python, or Javascript.  Just embed Lua, and use LJSyscall, and you’re done.  It will be far less code, and likely less buggy, than going any other route.

In that sense, LJSyscall becomes the new interface to the Linux Kernel…

A similar thing could happen to Windows.  There is the Drawbridge project in Microsoft Research.  This project is interesting because it encapsulates the Windows programming API, and slips in a shim that is only 45 system calls.  This ABI of 45 calls essentially defines what it means to be a “Windows” operating environment.  Now, let’s imagine for a moment that these 45 ABI calls were wrapped up in a way similar to what LJSyscall does.  That would essentially define an ABI that could be accessed by anything on any platform that supports LuaJIT.  So, assuming you had LJWin32ABI, you could imagine making that available on Linux.  Then, as long as the higher level libraries programmed against the Lua version, they would be available on Linux as well.  Therefore, you could get Windows apps running on Linux, or any other environment that supported LuaJIT sufficiently.

That’s a very interesting prospect I think.

At any rate, this is a fairly speculative topic, and I’m reaching deep into the crystal ball to make some predictions.  I’d love to see a world in which programming at a system level becomes easier in order to enable a lot more interesting scenarios.  I believe LJSyscall is one tool that points the way towards a future in which this might be possible.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s