LuaJIT 2.0 Final Released

If you cruise on over to the LuaJIT website:  You can see the announcement of the LuaJIT compiler finally going to version 2.0.

I must say, watching the evolution of this ‘compiler’ has been quite an eye opener for me.  The Lua language itself has been under continuous development for quite a few years.  LuaJIT is a different beast though.  LuaJIT constrains itself to being ABI compatible with the Lua 5.1 language, while providing a JIT compiler, which is on par with many a standard C compiler.  More amazing to me is this is an effort by a singular individual, Mike Pall.  Of course there is a community that supports his efforts by trying out his compiler, reporting bugs, discussing best practices, and the like, but the primary coding is being done by one guy.  This guy is very smart indeed, and has mad programming skills far beyond anything I’ll ever understand in my career.

In contrast to my own recent prognostications on the evolution of C++l, I’m actually looking forward to the evolution of dynamic scripting languages.  JavaScript is in vogue these days, and more recently there’s TypeScript on top of that, to make programming easier when teams of programmers are involved (yes, type hints are useful).

So, why is LuaJIT any special thing?  Well, it has two tricks that I find useful.  The first is the fact that it’s a JIT compiler.  The basic Lua language has bytecode which is usually interpreted at runtime.  This gives maximal flexibility and distribution across multiple environments.  It’s not the fastest thing in the world as there’s not much chance for hardware specific optimizations.  LuaJIT on the other hand, will compile down to native instructions.  So, much like a C compiler, you’ll get native instructions rather than just the interpreted byte codes. This makes for a much faster runtime environment.  We’re not just talking 10-20%, but orders of magnitude in certain situations.  The second trick of LuaJIT is the mechanism for interop to standard libraries.  These days, there’s a ton of code to be leveraged in the world.  When writing a web service, you probably need a cache, so memcached is sitting right there.  You need to interop with it.  With most other scripting languages, that means writing a ‘shim’ in ‘C’, which handles the communication between the two worlds.  With LuaJIT, you can fairly include the standard ‘.h’ file for memcached, and just start calling functions, as if the shim were already created.  That’s pretty amazing in and of itself.  I suspect that if the same feature makes it into javascript, the programming world will take a leap forward.

There’s one more thing that I find fascinating about this language.  Lua supports coroutines natively.  Co-routines are a poor man’s version of multi-threading.  Basically cooperative multi-tasking, same as what existed in Windows 1.0 back in the day.  The difference here though is that low level frameworks can use the feature, so the upper level consumer doesn’t really deal with it that much.  Some fairly serious systems have been created using this little feature.  One that I’m most impressed with at the moment is OpenResty.  This is an app-server based on Nginx, which incorporates Lua and co-routines deep into its bowels.  Haven’t heard of it?  Well, OpenResty was originally created to handle the webload of TaoBao, which recently set an online sales record.  We’re talking online services that are handling hundreds of millions of transactions, and nary a hiccup.  I don’t know if Taobao is still using an openresty framework, but it seems to bode well for anyone who wants to go down this path.

Programming systems is really hard.  Getting multi-threading is really challenging.  As a software engineer, I need a toolset that allows me to quickly prototype variations on themes, try new approaches, and just fiddle about, trying different APIs that are provided by my underlying platform. Rather than getting tied down by a particular language and environment, I have found that LuaJIT gives me a tool which allows me to interact with all environments, frameworks, and hardware devices.  When an approach gets baked, I can then look around and see if the platform supports a better/faster, more efficient way of implementing what I’m doing.

Discovering LuaJIT as a multi-tool has been a great discovery for me as a software engineer.  Now that it has reached a 2.0 release, I’ll happily put it in the proper place in my tool belt along with my C#, JavaScript/TypeScript, and C/C++ tools.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s