LJIT2libc – LuaJIT IS “batteries included”

I would say one of the most common complaints about any platform, framework, product is a paucity of available stuff to fiddle about with.  I’ve heard this criticism a few times leveled at the lua world in general.  Fatter frameworks are usually “batteries included”.  That’s great, when you get a whole ton of stuff that makes writing all sorts of apps relatively easy to do right off the bat.  The challenge of “batteries included” is that you get a whole ton of stuff, most of which you will never use, and some of which doesn’t have the best implementation.

Recently, I’ve been doing quite a lot off luajit bindings on the Linux platform:

If you were into anything from ASCII art graphics drivers to raw frame buffer management in a Linux system, these might start to look like batteries.  They’re not included with the luajit compiler, but they’re relatively easy to add.

But, there’s another one that I’ve been playing with recently which I think is even better:

Luajit is typically built and compiled against the libc and libm libraries.  As such, being able to access routines within those libraries comes for ‘free’ from within luajit, courtesy of the ffi capabilities.  This is very interesting because it means these routines, which are already on your system, are in fact just a stones throw away from being available within your app.
Let’s imagine you wanted to write some script like this, using the libc provided ‘puts()’ function:
puts("Hello, World!");

Well, the lua language has its own print routine, so this is a bit contrived, but let’s say you wanted to do it anyway. Well, to make this work, you need access to the function signature for the ‘puts’ routine and you need that to be accessible in the global namespace. So, it would actually look like this:

local ffi = require("ffi")

ffi.cdef("int puts(const char *);");
puts = ffi.C.puts;

puts("Hello, World!")

Great. A bit of work, and now I can program as wrecklessly as I could in C with all the batteries included in the libc and libm libraries. But, it gets tedious having to write that ffi stuff all over the place, so this is where the LJIT2libc thing comes in. It basically goes and implements a ton of the required ffi.cdef statements to make all those functions readily available to your script. An easy way to pull it into any script is to do the following:

local init = require("test_setup")()

puts("Hello, World!");

That first line ‘init = …’ in this case pulls in all of the definitions, and puts them into the global namespace, so that you can simply write your program without having to know anything about the ffi, or where functions come from or anything like that. Just start writing your code knowing that the batteries are already included.

Now, this example might seem too trivial for words, but just think about what’s in a typical libc library. It’s all the file system, sockets, system calls, math, random numbers, everything you’re likely to need to create higher level application stuff. It’s a lot of stuff that other systems end up creating from scratch, as part of their ‘batteries’.

Of course this is trivializing what’s happening in those batteries, because what you’re getting here is the raw C implementations of things, with all the headaches and dangers that are typically associated with writing code at that level. But, for those who want to have access to that level of code, and apply the safety net of lua where they see fit, this is quite a useful tool I think.

In short, batteries ARE included with every luajit. Those batteries might be just anode, cathode, and electrolyte, without the shell, but there are the raw ingredients. Having these wrappers available just makes it that much easier to think about and deal with a lot of low level stuff where I might previously had to resort to some sort of ‘package’ to achieve something as simple as traverse the file system.

So there you have it. luajit is a ‘batteries included’ system.


Gems on the interwebs – luakernel, stb, and me

Since the day of their inception, the interwebs have been a tremendous ‘resource’ to mankind.  Sharing academic papers… cat videos!!  Endless amounts of shopping, and who can ever do without pop-up ads!

Well, I have actually found very interesting things on the interwebs of late, particularly related to coding.  There’s almost nothing under the sun which has not been coded to date, or can’t be used as a building block to code up the next big thing.

Interesting people and tidbits I’ve looked at in the past year:

Karl Malbrain – Tons of interesting stuff related to data structures in particular

Sean Barrett – Tons of stuff from truetype font rendering to image reading/writing

Bob Jenkins – Hashes, random, math, physics, just all around good stuff

There are plenty more interesting libraries out there in the world.  These few stand out because they’re not libraries.  They’re not a part of some big grand platform.  They just plain standalone pieces of code which tend to be useful as learning tools.  The other thing that stands out is their liberal licensing.  There’s nothing worse than that monkey hand in the gourd moment when you’ve found some great piece of code to borrow from, and then you realize it’s a GPL’d thing, which is enough to curdle the strongest of milks.

I favor public domain, MIT, Apache, WTF, licensing myself because if you’re going to share, then share…  Otherwise, keep it to yourself.

I the Lua world, I’ve run across some gems lately that have been very interesting.

Reuben Thomaslua-stdlib : Perhaps the most useful thing out of this library is the std.optparse routines.  They allow you to easily specify, and then parse the arguments you pass to your program.  That takes care of an amazingly annoying set of routines (getopt, optind, optarg, argc, argv…) which are extremely buggy for me, and error prone.  There are other gems and techniques in this library which make it a real treasure trove to the Lua developer.

Eric R. Schulzluakernel Wait, what?  A kernel for x64 which boots into a Lua runtime?  Well, ok, embedded Lua did that a long time ago, but this seems like something I can easily approach.  I’ve got to add my graphic library to this thing so that I can boot into a Lua kernel, and party on with my killer graphics.  So, Lua kernels from tiny devices such as the ESP8266, to higher end x64 machines.  Not bad for the little language that could.

I’m just constantly amazed at how, after slaving away at something for a while, I’ll stumble across a gem on the internet which solves the exact problem I was after, or comes close enough to make me feel like I need to leverage that work, rather than start from scratch.

The best thing to do as a developer, is to get in the habit of looking out, leveraging, building your own treasure chest of tools, so that you can create that next big thing even quicker and more robustly than before.

Now I’m going to have to make a project of getting luakernel on one of my tired old boxes, so that I can host my graphics library, and develop the UI code (including full trutype support) even further.