Step By Step… Inch by Inch…

…Slowly I turned…

I guess I’m old enough to make references to slapstick comedy movie scenes that noone is likely to understand…

At any rate, continuing on the quest to be able to do IO Completion ports on Windows, I find that I need to be able to launch a thread that has an independent lua_State running in it. Although I can easily create an independent lua_State from within Lua, using the Luaffi that I created earlier, there is kind of a chicken and egg thing when it comes to creating a thread.

The fundamental problem is, the CreateThread signature looks like this:

HANDLE CreateThread(
	LPSECURITY_ATTRIBUTES lpThreadAttributes,
	size_t dwStackSize,
	LPTHREAD_START_ROUTINE lpStartAddress,
	LPVOID lpParameter,
	DWORD dwCreationFlags,
	LPDWORD lpThreadId);

And I want to call it roughly like this:

CreateThread(nil, 0,ThreadRoutine,codechunk,flags,nil);

I can ignore the security_attributes, for now, and I’ll take the default stack size, but I need the address of a start routine. Won’t a “callback” work here?  Well, a callback, like with a WindowProc, is the OS calling back to code that runs within the lua context that called the OS in the first place.  There must be a lua_state already running, so that the OS has something to call back to.  In this case, the code doesn’t already exist.  It’s just in a string, and I want to setup a LuaState to run that string as the guts of the routine.  So, I need an actual routine somewhere to get things started.  I need that routine to do just two things.  First, create a new independent lua_State.  And second, run the codechunk that I hand it, as the guts of its routine.  When that chunk of code finishes, the thread is finished, and will just exit.  It’s just like the ‘main()’ of LuaJit.exe itself.  Just create a state, load the code specified, and run it, returning when it is done.

So, I did have to write a bit of ‘C’ code to get past this little hurdle. First, I added a couple of new files to my LuaJIT distribution (beta 9). lwin32lib.h and lib_win32.c. All I need is a function that will take a void *, assume it is a chunk of lua code, create a lua state, and execute that chunk of code in that state. The header looks like this:

#ifndef lwin32lib_h
#define lwin32lib_h
#include
#include
#include "lua.h"

LUALIB_API int (RunLuaThread) (void *s);

#endif

Simple enough. And the actual implementation is equally simple:

#include
#include
#include 

#define lib_win32_c
#define LUA_LIB

#include "lua.h"
#include "lauxlib.h"
#include "lualib.h"
#include "lwin32lib.h"

LUALIB_API int RunLuaThread(void *s)
{
	const char *codecunk = (const char*)s;
	int status;
	lua_State *L = lua_open();  /* create state */

	if (L == NULL) {
		puts("RunLuaThread, cannot create state: not enough memory");
		return -1;
	}

	lua_gc(L, LUA_GCSTOP, 0);  /* stop collector during initialization */
	luaL_openlibs(L);  		/* open libraries */
	lua_gc(L, LUA_GCRESTART, -1);

	status = luaL_dostring(L, s);

	lua_close(L);

	return status;
}

That little bit of code is lifted almost verbatim from the equivalent LuaState code that I had written earlier. The only exception here is that I stop the GC before opening libs, and start it again. I borrowed that bit from the luajit.c file itself. If it’s good enough there, I figure why not here.

So, with that in place, my call to create a thread that can run a bit of Lua code, might look like this:

ffi.cdef[[
int RunLuaThread(void *s);
]]

local lua = ffi.load("lua51")
lua.RunLuaThread(ffi.cast("void *",[[print("Hello, Lua!!")]]))

I use the ffi.cdef to declare the signature of the little routine that I created in the Lua library. Then I just call it like I would call any other routine in the Lua library, passing in my code chunk. The result will be, my little code chunk will execute, in a separate thread, with its own lua state, and that’s exactly what I want.

Having that fundamental there, I can now make it easier by wrapping it up in a little object (because I’m just drawn that way).

require "BanateCore"
require "win_kernel32"

local kernel32 = ffi.load("kernel32")

class.OSThread()

function OSThread:_init(threadRoutine, param, flags)
	flags = flags or 0
	param = param or nil

	self.ThreadRoutine = threadRoutine
	self.Parameter = param
	self.Flags = flags

	local threadId = ffi.new("DWORD[1]")
	self.Handle = kernel32.CreateThread(nil,
		0,
		self.ThreadRoutine,
		self.Parameter,
		flags,
		threadId)
	threadId = threadId[0]
	self.ThreadId = threadId
end

This gives me a couple of things. First, an easy encapsulation of a thread in the OS. This could easily be changed to get the Linux, MacOS, or other OS specific version of the same thing.

In the case of Windows, it gives me a place to hold onto the thread handle, and ID, which will be useful later when I want to communicate with the thread. This also gives me a way to easily control the lifetime of the thread as I can add a __gc method to the metatable and decide if I want to take the opportunity to kill off, or otherwise terminate the thread.

With this in hand, my actual code looks like this:

local code1 = [[print("thread code 1")]]
local code2 = [[print("thread code 2")]]

thread1 = OSThread(lua.RunLuaThread, ffi.cast("void *", code1))
thread2 = OSThread(lua.RunLuaThread, ffi.cast("void *", code2))

That will create, and execute, two threads, each with their own independent LuaState.

Now I’m equipped to take on those completion ports. Why? Because I want to code that handles what happens with each IO completion to be written in Lua. Armed with this little threading trick, I can now just create an iocompletion thread, giving it the Lua code, and move on with life. It’s that simple. I can fully utilize the nice async queue I wrote that leverages co-routines, and do any manner of other lua stuff.

Just think of it. Co-routines are already a nice way to do stuff in Lua, but they’re limited to the single thread that’s executing your lua state. With true OS threads, you can actually spread your wings a bit, and leverage the multiple cores that are within your machine. Of course you need a way to communicate between your threads, but given global memory handles, and on Windows at least, PostThreadMessage, there are plenty of ways to do that. So, next step…

---