Pedantic Semantics – Parameterized Types

There are certain things I’ve done since time immemorial, that get repeated at least once per new language/environment, if not more than once. One of those things is the definition of some core types related to graphics systems. There’s everything from basic color and pixel representation to the higher level vector algebra stuff.

With such things, there’s always a desire to ‘templatize’ or otherwise create abstractions because you realize that finding the distance between two colors looks exactly the same as finding the distance bectween to ‘vectors’. In C++, and C#, I would do this with various forms of classes, templates, and the like. With LuaJIT, a new opportunity has arisen, and after a year of fumbling about, I think I’m finally happy with doing things a certain way.

I’ll start with some basic criteria:
I want to be able to easily pass structures between Lua, and the underlying platform native UI library (DirectX, OpenGL, etc).
I want some convenience in constructing objects so it should feel natural
I want some convenience in applying certain operators.
I do not need to overide ‘operator *’ to mean “cross product”, or any other hidden semantics

And with these criteria, I have the following:

For convenience, I create some aliases for some types, just so that my code looks easier and cleaner. Strictly speaking, the following isn’t the most performant thing to do, but since part of “performance” is ease of reading my own code later, I feel it is well worth it.

-- c99_types.lua

local ffi = require "ffi"

local array_tv = function(ct)
	return ffi.typeof("$[?]", ct)

local pointer_t = function(ct)
	return ffi.typeof("$ * ", ct)

local int8_t = ffi.typeof("int8_t");
local int8_tv = array_tv(int8_t);

local uint8_t = ffi.typeof("uint8_t");
local uint8_tv = array_tv(uint8_t);

local int16_t = ffi.typeof("int16_t");
local int16_tv = array_tv(int16_t);

local uint16_t = ffi.typeof("uint16_t");
local uint16_tv = array_tv(uint16_t);

local int32_t = ffi.typeof("int32_t");
local int32_tv = array_tv(int32_t);

local uint32_t = ffi.typeof("uint32_t");
local uint32_tv = array_tv(uint32_t);

local int64_t = ffi.typeof("int64_t");
local int64_tv = array_tv(int64_t);

local uint64_t = ffi.typeof("uint64_t");
local uint64_tv = array_tv(uint64_t);

local float = ffi.typeof("float");
local floatv = array_tv(float);

local double = ffi.typeof("double");
local doublev = array_tv(double);

local wchar_t = ffi.typeof("uint16_t");
local wchar_tv = array_tv(wchar_t);

local c99_types = {
	array_tv = array_tv,
	pointer_t = pointer_t,

	int8_t = int8_t,
	int8_tv = int8_tv,

	uint8_t = uint8_t,
	uint8_tv = uint8_tv,

	int16_t = int16_t,
	int16_tv = int16_tv,
	uint16_t = uint16_t,
	uint16_tv = uint16_tv,

	int32_t = int32_t,
	int32_tv = int32_tv,
	uint32_t = uint32_t,
	uint32_tv = uint32_tv,

	int64_t = int64_t,
	int64_tv = int64_tv,
	uint64_t = uint64_t,
	uint64_tv = uint64_tv,

	float = float,
	floatv = floatv,
	double = double,
	doublev= doublev,

	wchar_t = wchar_t,
	wchart_tv = wchar_tv,

return c99_types;

The LuaJIT FFI mechanism supports the various types located in the C99 standard. Making them readily available in this form allows you to do the following:

local ffi = require ("ffi")

c99 = require ("c99_types")

-- Allocate an array of 20 doubles
d1 = c99.doublev(20)

Of course, this is no big deal because you could have simply done the following:

d1 ="double[20]")
>> ctype

But, here’s another case:

f1 = c99.floatv(20)
>> 80
>> ctype

f2 = ffi.typeof(f1)(40)
>> 160

In this case, I’m allocating an array of floats in f1, and then I’m using the type of f1 (ctype) to allocate a new instance of something. Since the type is a variable length array, I can specify a new length for the array, and get a completely new thing, with a new size. If I had hard coded the length of the array, I would have gotten the same sized array (20 elements) no matter what I did.

This is the first form of parameterization. Just being able to allocate variable length arrays easily. This makes reallocation of arrays fairly straight forward in my mind. If I need to create a standard C++ style vector that grows, I can use this method to first allocate a certain sized array, then when it comes time to reallocate, I can just allocate a new array, and copy content from the old one.

Now for a bit of convenience that helps with my semantics. I want to be able to write the following:

f1 = float4(10, 20 30, 1)

A ‘float4’ is short hand for an array of single precision values that contains 4 elements. This is a common type for DirectX and OpenGL. I then want to be able to access this using simple ‘0’ based array indexing:

>> 30

In addition, I want to be able to pass this to a ‘C’ function, that takes a ‘float *’ parameter:

ffi.cdef[[void glVertex4fv (const GLfloat *v);]]


This all works, and my code starts to look very similar to what it would look like if I were writing standard C. With proper helper functions and such, it even starts looking like GLSL code if I choose.

One more little trick though. One thing I don’t like about typical template systems, is things get out of hand. In C++, templates are nothing more than a trick done at the lexical level. They are super advanced macros, as far as my usage has been over the years. They can add a lot of bloat to your code as they get expanded. So too with LuaJIT parameterized types. You have to be very careful when and how you create them. You don’t want to create a new type every time you allocate an instance. That’s why I used the type constructors that I’m using here.

But there’s one more thing. Mixing behavior with core types can be a bit tricky. Best to separate things out and stay as minimalist as possible. This last trick doesn’t have so much to do with parameterized types as it does with moving between type representations.

typedef struct {
	float x,y,z,w;
} Point3H, *PPoint3H;
Point3H = ffi.typeof("Point3H");
PPoint3H = ffi.typeof("PPoint3H")

local Point3D_mt = {
	__len = function(self) return 3 end,

	__tostring = function(self) return string.format("%3.4f, %3.4f, %3.4f", self.x, self.y, self.z) end,

	__index = {
		AsArray = function(self)
			return ffi.cast("float *", self)
Point3H = ffi.metatype(Point3H, Point3D_mt)

-- Allocate storage for the homogenous point
v4 = float4(10,20,30,1)

-- Cast the array as a pointer to a Point3H
local p1 = ffi.cast(PPoint3H, v4)

-- Prove that the metamethod is working
print("P1 Len: ", #p1)
>> 3

print("P1 Value: ", p1)
>> 10.0000  20.0000  30.0000

printVector(p1:AsArray(), 4)
>>10, 20, 30, 1

What’s the setup here? Well, there is the concept of a homogenized point. These are a convenience for representation in graphics systems where you’re going to multiply against a 4×4 matrix. So, each 3D point has a 4th parameter, which is always ‘1’. In this case, I’ve allocated the storage for a point in the form of the ‘v4’ array.

You can not assign a metatype to such an array directly, but you can cast it as something else. In this case, I’ve created a convenient representation for my Point, called “Point3H”, and a pointer to that type as well, called “PPoint3H”. I’ve created a metatable called ‘Point3D_mt’. Finally, I associate the metatable to the type with: Point3H = ffi.metatype(Point3H, Point3D_mt)

The bit of magic that happens here is that when you associate a metatable with a type, it is also associated to pointers of that type.

So, when I finally cast my v4 to be of type “PPoint3H”, I’ve essentially associated a metatable to it. At this point, I can now call the metatable functions, like __tostring, and __len, or whatever else I choose to throw in there.

I can also access the elements of my array using standard symbolic names, such as ‘x, y, z’. And, lastly, I can flip back tot he array representation by simply calling “AsArray()”.

When all is said and done, what I have here are the makings of a system that provides me with a great deal of clarity and convenience. These techniques may or may not be the most performant way to go, depending on what you’re trying to achieve. For my time and money though, it certainly makes my code a lot more compact and clean.

Next, I’m going to apply this same technique to refresh some other areas of the graphics system I had developed over the last year.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s