Accessing Open vSwitch using LuaJIT – The LJIT2ovs project

I’ve recently had the need to learn Open vSwitch to do some virtualized networking stuff. As part of my learning experience, I’ve created the LJIT2ovs project on GitHub.

Open vSwitch (OVS) is a pretty decent bit of kit.  It’s use is to act as that bit of bridge code on a machine that supports some software defined networking.  Essentially, a network hub/switch/bridge on your machine.  My particular use case is to support interesting networking topologies with VMs, virtual lans and the like in the context of a data center.  Having OVS allows you to do some fairly fancy things in terms of managing VM movements, and virtual lans.

I have certainly done tons of integration with libraries of various types on platforms from the Raspberry pi, to Windows, and Linux in general.  Some code is easier to interop with and some code is harder.  I wrote a series a couple years back on the benefits and pitfalls of different styles of API development.

The OVS code is fairly easy to interop with using LuaJIT.  The publicly exposed stuff is all C99, and not particularly fancy.  There are a few cases where the usage of macros had me scratching my head, but a quick pass with a C compiler typically helps flesh those out.

One example will help illustrate the ease and pitfalls of doing some forms of interop.  I’ll use the UUID features in the library as an example.

First there’s data structures:

ffi.cdef[[
struct uuid {
uint32_t parts[4];
};
]]

The LuaJIT ffi makes pretty short work of that. Now the various routines that need a struct uuid will have access to what they need. The various functions are thus:

ffi.cdef[[
void uuid_init(void);
void uuid_generate(struct uuid *);
void uuid_zero(struct uuid *);
bool uuid_is_zero(const struct uuid *);
int uuid_compare_3way(const struct uuid *, const struct uuid *);
bool uuid_from_string(struct uuid *, const char *);
bool uuid_from_string_prefix(struct uuid *, const char *);
]]

Again, fairly simple. The use of ‘bool’ instead of ‘int’ as a return type from functions makes life a lot easier as the ffi will translate that to lua’s native boolean type. If it were an int, then instead of simpling doing

if not uuid_from_string(someuuidstring) then
  print("FAIL")
end

You’d have to write

if uuid_from_string(someuuidstring) ~= 0 then
  print("FAIL")
end

It seems like such a minor difference, but it’s actually a pretty huge pain in the backside. When the return value is an int, there are myriad things it can mean. One of them is ‘0 == success’, another is ‘0 == failure’, and there can by various others of course, including checking errno to find out what really went wrong or right. the bool is much more straight forward, success is usually indicated with ‘true’ and failure ‘false’.

Where things get a little more tricky and you have to pay special attention, is when you’re doing interop between different parts of the system and you flow through lua along the way.

Part of OVS has this dynamic string thing, which is one of those stretchy buffers, which will grow as necessary as you add stuff to it. One of the things it has is a ‘printf’ style formating thing.

ffi.cdef[[
void ds_put_format(struct ds *, const char *, ...) ;
]]

Yes, the LuaJIT ffi can deal with variadic functions, but you have to take extra care when feeding it the variable arguments. In particular, you’re responsible for dealing with the types of those arguments. This is where the care comes in. With normal functions (non-variadic), the ffi marshaling will just take care of things, and you don’t even realize a “number” is being turned into an ‘int32_t’, or whatever it needs to be. The ffi just figures it out based on the signature, and does the right thing. Not so with variadics. It can’t know what you need to pass.

So, here’s more from the UUID routines, some macros.

#define UUID_LEN 36
#define UUID_FMT "%08x-%04x-%04x-%04x-%04x%08x"
#define UUID_ARGS(UUID)                             \
    ((unsigned int) ((UUID)->parts[0])),            \
    ((unsigned int) ((UUID)->parts[1] >> 16)),      \
    ((unsigned int) ((UUID)->parts[1] & 0xffff)),   \
    ((unsigned int) ((UUID)->parts[2] >> 16)),      \
    ((unsigned int) ((UUID)->parts[2] & 0xffff)),   \
    ((unsigned int) ((UUID)->parts[3]))

In general, macros are of two types. First are the simple #defines which can turn into local variables. In this particular case, the UUID_LEN and UUID_FMT fit the bill. The UUID_LEN is just a numeric constant, so it can just become a regular number. The UUID_FMT doesn’t use any particularly fancy format routines (luckly lua string.format is similar enough in most cases), so that can become a simple string.

local UUID_LEN = 36
local UUID_FMT = "%08x-%04x-%04x-%04x-%04x%08x"

But what about that macro? Can we just turn it into a function, strip out the naughty bits, and call it a day?

local function UUID_ARGS(UUID)                             
    return ( ((UUID).parts[0])),            
    ( rshift((UUID).parts[1], 16)),      
    ( band((UUID).parts[1], 0xffff)),   
    ( rshift((UUID).parts[2], 16)),      
    ( band((UUID).parts[2], 0xffff)),   
    ( ((UUID).parts[3]))
end

Well, I want to write the following:

local success = uuid.uuid_from_string_prefix(id1, "12345678-1234-5678-1234-5678123456781234")
local output = ds.dynamic_string();
ds.ds_put_format(output, uuid.UUID_FMT, uuid.UUID_ARGS(id1))

print("OUTPUT: ", ds.ds_cstr(output));

Basically just round trip a string into a UUID and back out into a stretchy buffer, and print it as a lua string. Does it work? Nope, not quite. In this case, the output would be:

00000000-0000-0000-0000-45e4de9000000000

Not quite what I was expecting. Head scratch. OK, well, obviously I was a bit simplistic in my conversion of that UUID_ARGS macro. Maybe I should have preserved that casting stuff.

local uint32_t = ffi.typeof("uint32_t")

local function UUID_ARGS(UUID)                             
    local p1 = uint32_t(UUID.parts[0])
    local p2 = uint32_t(rshift(UUID.parts[1], 16))
    local p3 = uint32_t(band(UUID.parts[1], 0xffff))
    local p4 = uint32_t(rshift(UUID.parts[2], 16))
    local p5 = uint32_t(band(UUID.parts[2], 0xffff))
    local p6 = uint32_t(UUID.parts[3])

    return p1, p2, p3, p4, p5, p6
end

In addition to the temporary variables (used for debugging), I stripped out a number of parenthesis, because lua lives better with fewer parens. Basically, I used the ‘uint32_t’ type in a way that is similar to a typecast. Of course, it’s actually worse the way I’ve used it here because it creates a temporary number, which needs to be garbage collected, but the result is now:

OUTPUT: 12345678-1234-5678-1234-567812345678

Which is what I was expecting. So, maybe one last pass to clean it up…

local function UUID_ARGS(UUID)                             
    return ffi.cast("unsigned int", (UUID.parts[0])),
        ffi.cast("unsigned int", rshift(UUID.parts[1], 16)),
        ffi.cast("unsigned int", band(UUID.parts[1], 0xffff)),
        ffi.cast("unsigned int", rshift(UUID.parts[2], 16)),
        ffi.cast("unsigned int", band(UUID.parts[2], 0xffff)),
        ffi.cast("unsigned int", UUID.parts[3])
end

So, essentially, a more faithful representation of the original. Note though, although this will work with the UUID_FMT, when using a ‘C’ based ‘printf’ statements, it will NOT work with the lua string.format routine. It will complain that it was expecting a number, but received a ‘cdata’ instead. So, you have to think fairly clearly about how those APIs and macros are going to be used, and therefore how they should be translated.

Here’s that uuid.lua file in its entirety:

local ffi = require("ffi")
local bit = require("bit")

local rshift, band = bit.rshift, bit.band

local function BUILD_ASSERT_DECL(...)
    assert(...);
end

local UUID_BIT = 128;            -- Number of bits in a UUID. */
local UUID_OCTET = (UUID_BIT / 8); -- Number of bytes in a UUID. */

ffi.cdef[[
/* A Universally Unique IDentifier (UUID) compliant with RFC 4122.
 *
 * Each of the parts is stored in host byte order, but the parts themselves are
 * ordered from left to right.  That is, (parts[0] >> 24) is the first 8 bits
 * of the UUID when output in the standard form, and (parts[3] & 0xff) is the
 * final 8 bits. */
struct uuid {
    uint32_t parts[4];
};
]]

BUILD_ASSERT_DECL(ffi.sizeof("struct uuid") == UUID_OCTET);


local UUID_LEN = 36;
local UUID_FMT = "%08x-%04x-%04x-%04x-%04x%08x";

local function UUID_ARGS(UUID)                             
    return ffi.cast("unsigned int", (UUID.parts[0])),
        ffi.cast("unsigned int", rshift(UUID.parts[1], 16)),
        ffi.cast("unsigned int", band(UUID.parts[1], 0xffff)),
        ffi.cast("unsigned int", rshift(UUID.parts[2], 16)),
        ffi.cast("unsigned int", band(UUID.parts[2], 0xffff)),
        ffi.cast("unsigned int", UUID.parts[3])
end


--[[
/* Returns a hash value for 'uuid'.  This hash value is the same regardless of
 * whether we are running on a 32-bit or 64-bit or big-endian or little-endian
 * architecture. */
--]]
local function uuid_hash(uuid)
    return uuid.parts[0];
end

-- Returns true if 'a == b', false otherwise. */
local function uuid_equals(a, b)

    return (a.parts[0] == b.parts[0]
            and a.parts[1] == b.parts[1]
            and a.parts[2] == b.parts[2]
            and a.parts[3] == b.parts[3]);
end


  
ffi.cdef[[
void uuid_init(void);
void uuid_generate(struct uuid *);
void uuid_zero(struct uuid *);
bool uuid_is_zero(const struct uuid *);
int uuid_compare_3way(const struct uuid *, const struct uuid *);
bool uuid_from_string(struct uuid *, const char *);
bool uuid_from_string_prefix(struct uuid *, const char *);
]]

local Lib_uuid = ffi.load("openvswitch")

-- initialize uuid routines
Lib_uuid.uuid_init();

local exports = {
    Lib_uuid = Lib_uuid;

    UUID_BIT = UUID_BIT;
    UUID_OCTET = UUID_OCTET;
    UUID_FMT = UUID_FMT;
    UUID_ARGS = UUID_ARGS;

    -- types
    uuid = ffi.typeof("struct uuid");

    -- inline routines
    uuid_hash = uuid_hash;
    uuid_equals = uuid_equals;

    -- library functions
    uuid_init = Lib_uuid.uuid_init;
    uuid_generate = Lib_uuid.uuid_generate;
    uuid_zero = Lib_uuid.uuid_zero;
    uuid_is_zero = Lib_uuid.uuid_is_zero;
    uuid_compare_3way = Lib_uuid.uuid_compare_3way;
    uuid_from_string = Lib_uuid.uuid_from_string;
    uuid_from_string_prefix = Lib_uuid.uuid_from_string_prefix;
}

return exports

Just a couple more notes on style. When I’m using such a binding, I’ll typically do the following:

local uuid = require("uuid")

From there, I will want to utilize a simple notation to get at things.

local id = uuid.uuid(); 
uuid.uuid_generate(id);

local output = ds.dynamic_string();
ds.ds_put_format(output, uuid.UUID_FMT, uuid.UUID_ARGS(id))

I want to minimize the amount of times I have to use the ffi routines directly because they’re fairly low level, and can lead to some very subtle errors if you’re not careful. So, a properly constructed module will wrap up as much as possible.

In some cases I’ll create a ‘class’, which is a table to encapsulate the various bits and related bytes. In this case I did not, but I might still do it later if I find it useful.

I will typically do the ffi.load to get a handle on the library from which the routines are called, and stick that reference into a table, so the library reference won’t be garbage collected, and therefore possibly unloaded from the running system. And lastly, the function names have this construct:

    uuid_generate = Lib_uuid.uuid_generate;

It’s a simple aliasing so that I don’t have to remember which library the function is located in, I can just call uuid.uuid_generate, and the right thing will happen. This isn’t the most performant way to do it as the compiler can’t optimize the function call as much, so it will be slower. These routines are not typically called in a performant loop, so it’s an ok thing. If you were being really hard core, and needed the extra cycles, you’d make the library call directly instead of going through the table indirections.

The last trick you can do is to import that exports table into the global namespace:

for k,v in pairs(uuid) do
    _G[k] = v;
end

If you do this, then your code can drop the table indirection and become simply:

local id = uuid(); 
uuid_generate(id);

local output = dynamic_string();
ds_put_format(output, UUID_FMT, UUID_ARGS(id))

This is looking pretty close to the original ‘C’ code. Fewer pointers to deal with, no memory allocation problems, no funky difference between ‘.’ and ‘->’ indirection. In this way, Lua is just a nicer version of ‘C’ 🙂 My coworkers will get a kick out of that sentiment.

And so, there you have it. A fairly small example of beginning to interop with OVS. Open vSwitch is a fairly large codebase. It’s split between simple base library routines, reporting, core packet handling, and higher level apps, daemons and utilities. Starting from the top, the utilities are fairly easy to replace, once the interop at the lowest levels is dealt with. In LJIT2ovs, the lower levels are fairly well covered, and coverage increases as needs dictate. At the higher level, I am beginning with replacing simple routines, such as querying the database. At the end of the day, these higher level utilities are much easier to write, maintain, expand, and incorporate when they are written in script, rather than C.

Next time around I’ll examine some of these routines, and how they can be rewritten, and what benefits might acrue from that exercise.