Is that a template in your LuaJIT pocket?

One of the nicest things about programming, and the explosion of languages, frameworks, and whatnot, is there are plenty of opportunities to learn new stuff, and morph old things.  For quite a long time I’ve been stalking the solution to a particular problem, and I finally had the Homer Simpson moment (doh!!).

Here’s the basic problem.  Because I use OpenGL (insert favorite C API here), I need to pass in pointers to arrays of certain types.  Like this call: glWindowPos3fv(const GFloat *v);

In order to do this in lua, I could easily do:

local vec3 = ffi.typeof("float[3]");
local v1 = vec3(10,20,30);

Yah, that would work. But, with graphics, there’s some math involved. I also want the following:

local v2 = v1*5;
local xaxis = vec3(1,0,0);
local yaxis = vec3(0,1,0);
local zaxis = xaxis:cross(yaxis);

So, clearly I need to attach a metatable to the vec3 type that I created. Well, I can’t. That’s a “float *” essentially, and it can’t be distinguished from other ‘float *’ declarations, so it’s kind of ambiguous. In addition, sometimes I want to access these structures using a numeric index ‘v1[2]’, and sometimes I want to get at elements using some symbolic name with dot notation: ‘v1.y’.

If that weren’t bad enough, I also want to be able to change the base type used in the array. Sometimes I want to use float as the core type, and sometimes I want to use double, and other times I’d like to use integer.

Oh what to do, what to do? wringing of hands, gnashing of teeth. I finally came up with the following:

local function make_vec3_kind(ct)
	local vec3_kind = ffi.typeof("struct {$ x; $ y; $ z;}", ct, ct, ct);
	local ptrType = ffi.typeof("$ *", ct);
	local constptrType = ffi.typeof("const $ *", ct);
	local vec3_t = {
		cross = function(self, v)
			return vec3_kind(
				self.y*v.z - v.y*self.z,
				-self.x*v.z + v.x*self.z,
				self.x*v.y - v.x*self.y);

		PlaneNormal = function(point1, point2, point3)
			local v1 = point1 - point2
			local v2 = point2 - point3

			return v1:Cross(v2);
	setmetatable(vec3_t, vec_t);

	local vec3_mt = {
		__len = function(self) return 3; end;
		__tostring = function(self) return self.x..','..self.y..','..self.z; end;

		-- Math Operators
		__eq = function(self, rhs) return vec_t.equal(self, rhs); end;
		__unm = function(self) return vec_t.unm(self); end;
		__add = function(self, rhs) return vec_t.add(self, rhs); end;
		__sub = function(self, rhs) return vec_t.mul(self, rhs); end;
		__mul = function(self, scalar) return vec_t.mul(self, scalar); end;
		__div = function(self, scalar) return vec_t.div(self, scalar); end;
		__index = function(self, key)
			if type(key) == "number" then
				return ffi.cast(ptrType,self)[key];
			elseif key == "asPointer" then
				return ffi.cast(ptrType,self);
			elseif key == "asConstPointer" then
				return ffi.cast(constptrType, self);

			return vec3_t[key];
		__newindex = function(self, key, value)
			if type(key) == "number" then
				ffi.cast(ptrType,self)[key] = value;

	return ffi.metatype(vec3_kind, vec3_mt)

This is basically the closest equivalent of a C++ template that I’ve used so far. With it you can do the following:

-- create the vec3 type based on a 'float' base type
vec3 = make_vec3_kind(ffi.typeof("float"));
v1 = vec3(1,7,30);

print("v1[2]: ", v1[2]);
>v1[2]:  30

v2 = vec3(10,20,30);
v3 = v1 + v2;
print("V2: ", v2);
>V2:  10,20,30

print("V3: ", v3);
>V3:  11,27,60

v4 = vec3();
v4[0] = 11;
v4[1] = 12;
v4[2] = 13;

print("V4: ", v4);
>V4:  11,12,13

local xaxis = vec3(1,0,0);
local yaxis = vec3(0,1,0);
local zaxis = xaxis:cross(yaxis);

print("xaxis:cross(yaxis): ", zaxis);
>xasis:cross(yaxis): 0,0,1

How does it work? First, there’s the definition of the vec3_kind. This is a parameterized type, whereby a data structure is created with the specified type substituted in wherever the ‘$’ exists. Now that we have a real data structure, a metatable can be attached to it.

The vec3_t table is meant to contain functions that are specific to the vec3_kind. All the other functions that are generic to vector types are stored in a vec_t table.

The ffi.metatype() function is different from ‘setmetatable’. While ‘setmetatable’ is used to add metatables to standard Lua tables, ffi.metatype() takes an ordinary ‘C’ structure, and associates a metatable with it. Wha? Basically assigning functions to a structure. Both of these operations need to occur to cover all cases here. Basically, if a function is not found in the vec3_t table, it will then be looked up in the vec_t table. This is like creating a sub-class, or specialization of the vec_t type.

In this case, I created a structure that explicitly has ‘x,y,z’ as the structure fields. That’s convenient because for all cases that I intend to use this structure for, I will either access it with ‘x,y,z’, or through the array indexing approach ‘[]’. How do you do array indexing on this thing though? This is the part that had me perplexed for a while. They key is, I want to be able to treat the structure as a ‘float *’ as well as a regular structure.

In the beginning, I declare ‘ptrType = ffi.typeof(“$ *”, ct)’

This basically gives me a pointer type based on the base element type. Having this in hand, I can then go look at the ‘__index’ metamethod. ‘__index’ is called whenever you try to ‘lookup’ something in a table and it doesn’t exist. In the case of doing ‘v1[2]’, I am basically doing a lookup on the v1 structure, and I’m looking for the entry that has index ‘2’. It doesn’t actually exist, so my __index function is invoked. The __index function in turn does a cast of the structure to a “float *”, and indexes that structure, returning the value it finds as the offset ‘2’. The ‘__newindex’ method is called whenever you’re trying to do an assignment in the same way.

And there you have it.

The reason I call this a template is because this is pretty much exactly what I would do in C++. I’d create a bunch of code that is fairly generic for a type, then I’d use a bunch of ‘<>’ annotations to say it was a template. In this case, I achieve the same thing by encapsulating everything into a ‘type factory’, which is the ‘make_vec3_kind()’ function.

The ‘doh!’ moment was simply realizing that by encapsulating things in a function in this way, all the enclosed functions have access to the ‘upvalues’ available in the function. That just means the inner functions and tables have access to the outer variables, such as ptrType, vec3_t, and the like.

This particular template could be made one step better by using something like this:

local function make_vec_kind(ct, nelems)
vec_kind = ffi.typeof("struct {$ data[$];}", ct, nelems);

But, that’s an exercise for a rainy day.

I now have what I need to create various types of vector types, and I can easily reuse the code, and gain the access patterns that I need. If you’re used to doing glsl programming, you can imagine how to add the swizzling feature of that language.

local rgb = v1.rgb;

And that will become even more fun.

And so it goes. Realizing something new every day, and the world becomes an easier place to program.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s