Simplified API Development – Part 4, Wrapping Up

The Series Thus far:
Simplified API Development – Part 3, Second Steps
Simplified API Development – Part 2, First Steps
Simplified API Development – Part I, Ground Rules
So far, we’ve laid some ground rules:

1) All functions will have a return value

2) The first value is either true/false, or the return value of the function

Taken some first steps in creating structures and constants, then mated some functions to structures to make life a little bit easier.  Now it’s time to wrap things up, satisfy the ground rules, and launch into a programming Nirvana.

In some ways, the further you go “up the stack” the easier things becomes.  With the very basics in place, it becomes a relatively easy stitching job to pull things together and shape them in ways that are truly useful.  So, let’s take a look at some function wrappers.

local SocketLib = ffi.load("ws2_32")

local socket = function(af, socktype, protocol)
  af = af or AF_INET
  socktype = socktype or SOCK_STREAM
  protocol = protocol or IPPROTO_TCP

  local sock = SocketLib.socket(af, socktype, protocol);
  if sock == INVALID_SOCKET then
    return false, SocketLib.WSAGetLastError();
  end

  return sock;
end

This is a nice function to dissect because it has a lot going on, but it’s fairly simple. First of all, it’s the function you call in order to create a socket. Creating a socket is different from using a socket. When you create a socket, you’re just announcing an intention to communicate over the network. You still need to determine whether you’re going to be passively listening on the socket (server), or actively connecting (client). But first, you must create the socket, and this is what this call does.

First are the options. With Lua, parameters can be optional. One easy way to catch and deal with this is the construct seen here: af = af or AF_INET

If you were programming in ‘C’, this is roughly equivalent to af ? af : AF_INET
Really it’s not the same thing at all, but you can think of it that way in this particular case. So, the user can call this function, just like they would in ‘C’:

local sock = socket(AF_INET, SOCK_STREAM, IPPROTO_IP);

That’s pretty interesting I think. Of course, there are a couple of other benefits though. Given our ground rules, you can also call it like this:

local sock, err = socket(AF_INET, SOCK_STREAM, IPPROTO_IP);

That way, if there’s an error, the ‘sock’ will be nil, and the ‘err’ will tell you what went wrong. So, in practical terms, you’d normally do something like this:

local sock, err = socket(AF_INET, SOCK_STREAM, IPPROTO_IP);
assert(sock, err)

That’s quite different from having to check the return value of the function against ‘INVALID_SOCKET’ and then calling “WSAGetLastError()” every single place you use the raw function in your code. What people typically do is ignore the error handling, and the breakage begins to propagate. Here, error handling always occurs, and there’s no way for you to get an invalid socket. That doesn’t mean your communications will be flawless, but at least your error won’t be coming from here. Another benefit to doing this wrapper is isolation. In this particular case, the wrapper deals with the Win32 WinSock way of doing things. This would not be true for a Linux system. Finding and reporting errors is different there, so you’d have to adjust. But, you only have to adjust the wrapper. The user’s code, since it’s calling the wrapper function, would not have to change at all.

OK, so this seems to be a useful concept. Here’s some more wrappers:

local accept = function(s, addr, addrlen)
	local socket = SocketLib.accept(s,addr,addrlen);
	if socket == INVALID_SOCKET then
		return false, SocketLib.WSAGetLastError();
	end

	return socket;
end

local bind = function(s, name, namelen)
	if 0 == SocketLib.bind(s, ffi.cast("const struct sockaddr *",name), namelen) then
		return true;
	end

	return false, WinSock.WSAGetLastError();
end

local connect = function(s, name, namelen)
	if 0 == SocketLib.connect(s, ffi.cast("const struct sockaddr *", name), namelen) then
		return true
	end

	return false, SocketLib.WSAGetLastError();
end

local closesocket = function(s)
	if 0 == SocketLib.closesocket(s) then
		return true
	end

	return false, SocketLib.WSAGetLastError();
end

Of particular note are these two:

local recv = function(s, buf, len, flags)
	len = len or #buf;
	flags = flags or 0;

	local bytesreceived = SocketLib.recv(s, ffi.cast("char*", buf), len, flags);

	if bytesreceived == SOCKET_ERROR then
		return false, SocketLib.WSAGetLastError();
	end

	return bytesreceived;
end

local send = function(s, buf, len, flags)
	len = len or #buf;
	flags = flags or 0;

	local bytessent = SocketLib.send(s, ffi.cast("const char*", buf), len, flags);

	if bytessent == SOCKET_ERROR then
		return false, SocketLib.WSAGetLastError();
	end

	return bytessent;
end

Under most circumstances, when you’re sending bytes, you’ll have a construct like the following:

local bufflen = 1024;
local buff = ffi.new('char [?]', bufflen);

send(sock, buff, bufflen, 0)

And if you wanted to send the contents of a ‘string’

local str = "Hello, World!";
send(sock, str, #str, 0)

But, since the wrapper will check the size (‘#’) of the buffer if you don’t give it explicitly, and because the flags will default to ‘0’ if you don’t give them explicitly, you can simply do this:

local bytessent, err = send(sock, "Hello, World!");

and be done with it…

I think this is very valuable, and very easy. No allocation of buffers (the string is already allocated anyway). No fussing with parameters that the compiler can easily figure out. Just write your code, have the errors automatically checked, and move on with life.

OK, great, now there are a bunch of wrappers for all of those Berkeley functions. One thing of note is that you could have written all these wrappers and put them into a ‘C’ library. If you’re writing in a language like standard Lua, or JavaScript, that’s exactly what you’ve got to do. It’s a hairy tedious, error prone, and very boring task. And, once you do it, you’ve now complicated your build/deployment story. You’ve got to build that code separately, and make sure the right C library is included with your project, etc…

Doing it this way, where everything is in the ‘native’ dynamic language is just that much easier.

Moving right along, now that we have all these wrappers, what does the rest of this helper file look like?

Recap: There is one file, winsock_ffi.lua, which contains the basic FFI wrappings of the winsock library. The file that all these helper wrappers go into is WinSock_Utils.lua. It contains all the wrappers, classes, metamethods for structures and the like. Again, the principle here is, if all you want is the lowest level ffi interaction with whatever core library you’re dealing with, you should not have to incur the cost of the wrappers.

The tail end of the WinSock_Utils.lua file looks like this:

return {
	WSAData = wsadata,

	Lib = SocketLib,
	FFI = wsock,

	-- Data Structures
	IN_ADDR = IN_ADDR,
	sockaddr = sockaddr,
	sockaddr_in = sockaddr_in,
	sockaddr_in6 = sockaddr_in6,
	addrinfo = addrinfo,

	-- Library Functions
	accept = accept,
	bind = bind,
	connect = connect,
	closesocket = closesocket,
	ioctlsocket = ioctlsocket,
	listen = listen,
	recv = recv,
	send = send,
	setsockopt = setsockopt,
	shutdown = shutdown,
	socket = socket,

	WSAPoll = WSAPoll,
	WSASocket = WSASocket,

	-- Helper functions
	GetLocalHostName = GetLocalHostName,
	GetSocketErrorString = GetSocketErrorString,
}

So, usage is like this:

local WinSock = require "WinSock_Utils.lua"

local sock = WinSock.socket();
-- do your socket thing from here

Basically, everything in the file is defined as ‘local’ so that it does not pollute the global namespace. This is an important point. Make things local if at all possible. If you do this, consumers will be happy, and they can always turn them into global functions if they want. But, if you start out global, it’s hard to make them local.

I include a reference to the library as the “Lib” member of the list, and a reference to the FFI as the “FFI” member. The rest are all the functions and structures that have been defined.

And that’s about it. These are the basics of a simple API. You get multiple levels of access, error handling wrapped up in a common way, a common calling convention and return value semantics, and relative portability of API across different host environments. In my mind, this is a very good thing, and the general approach I take in API development.

From here, it’s a relatively easy thing to go another step and start introducing ‘classes’, that is, further grouping of functions into logical objects to make life even easier. But again, I’d keep it simple and layered. Don’t force someone who’s not really into the whole class thing to deal with it, make it a nice add-on, in another file, or set of files.

And there you have it. Simplified API Development.

I have been slowly but surely encapsulating a lot of Win32 APIs, and they can be found in the LJIT2Win32 project.

---