Knowledge Surfing

Surviving in the age of AI

As is fitting of the times, I spent a few minutes with ChatGPT/Dall*E to prompt my way to that image. In the distant past (last year), I would have used a search engine, scouring the internet for such an image. If I could even find anything close to what I wanted, I’d then be worried about copyright, and the fact the image is probably also being used by someone else. Now, I know this image is “my own”, at least I think so, and it represents exactly what I wanted.

I went through several iterations before I got to this image, and I could go through several more rounds, tweaking along the way, but it was less than 5 minute’s worth of work. Such is “the power of AI”.

This image represents a couple of things that are going on right now. Between advances in hardware tech, and the rise of new things such as LLMs, and quantum computing, and humanoid robotics, there’s a wave of things that threaten to swamp our human understanding of the world and the way things work. I recently had a discussion with a friend who’s working in the core of all this AI hullabaloo, and they are clearly thinking “what does this all mean? It’s moving so fast we haven’t truly thought about the implications to humanity…”. Just let that sink in for a moment. If you thought you were being overwhelmed as an outsider, the people on the inside are having the same thoughts.

OK, so… What to do?

The other thing the image represents is a coping mechanism. When faced with a wave, the best thing to do is learn how to surf. I’ve had many a discussion with people where the topic is the tech of today, and how overwhelming it can be. My advice is always “We are being handed magic wands, learn how to be a magician”. Whether it be magic wands, or surf boards, the advice is about finding a right way to consume the knowledge, and use the tools, rather than being in awe and swamped by them.

Absorbing knowledge quickly is hard. What’s particularly hard is trying to have complete understanding of a bunch of new things all at once. Our brains just aren’t made for that. We’re much slower, taking years to absorb the simplest concepts such as using linear algebra to transform from 3D space to 2D space. I mean, understanding how consuming the internet can turn into humanoid robotics? That’s a bridge too far I think. And just as you finally grok what you think an LLM is, the models are becoming tiny instead of large, and Apple is telling you inference is best done on your mobile device, rather than in the cloud.

How to cope?

Well, way back in 1986, I actually wrote a story in a self published magazine, that deals with this exact topic. I’m going to have to scan those old stories in and re-publish them, because they seem relevant to today. The gist of it was, you have to lower your comprehension quotient in order to understand the bigger picture. Essentially relax your mind, and your body follows (to misquote a movie line).

Another reference for the Berkeley hippies such as myself, Abhyanga yoga. It’s a technique that uses oil, and many touches at the same time. The general idea is, you can’t possibly focus on all the stimuli at once, so, at some point, you just kind of give up, and take all the individual touches as one, and relax into the overall experience.

Alright hippy, what does that have to do with the tech of today? Well, everything. But let’s be practical and structured about it.

How to become a knowledge surfer

  • Find a short list of information outlets that you can refer to constantly. These outlets need to be low on drama and gas lighting, and present you with a steady stream of factual information
  • Pick a short list of topics that you’re going to track. It’s hard to follow everything, so just choose two or three things, such as “green energy”, “humanoid robotics”, “quantum communications”
  • Read your info outlets only once a day
  • Write a summary of what you’ve learned at the end of each week
  • Once a month, write a “note to self” style of thing in which you summarize the learnings for the month, and some conclusions about what you think about what you’ve learned
  • Adjust your list of topics, or information outlets at the beginning of the month, but keep the list short
  • Interact with other humans, and share your monthly summaries, to get their reactions, and possibly new perspectives

It’s all about structure and balance. You need to know which waves you’re riding, and you need to have a solid foundation (the surf board) upon which to do your information surfing. “Knowledge” comes from gathering data, testing a hypothesis, learning, and moving ahead. If you don’t have such structure, then you get swamped by the data, always grasping and gasping, and definitely not riding the crest of the wave.

This is my practice on a daily basis. I gather information, I summarize in the form of “Hot Takes”, I disseminate to small audiences and get feedback, I move on to the next week’s worth of news. There’s a lot going on in tech, and the world in general. I find that having a structured approach to information gathering makes knowledge surfing easier.


Manufactory – CNC Router Table

Well, there it is. An actual functioning CNC router table.

Being one to invent random words, I came up with “manufatory”, to mean “manufacturing at the speed of thought. What am I trying to get at? In general, I’m on a crusade for tequity, having an equity share in technology, for intergenerational wealth creation. Whether it be owning stocks in a company, or patents, or other artifacts, being able to ride the rising tide of tech wealth requires owning a piece of it, and not just being a consumer of tech.

So, what’s this manufactory business about? Software production is one kind of intellectual property, but not all things are software. Everything we interact with in the world was created by someone, or some machine, somewhere in the world. Being able to think up a design, produce the goods economically, and sell them into open markets, is tequity.

A couple blogs back (Embodied AI – Software seeking hardware) I mentioned three machines,

  • 3D printer
  • CNC router
  • Robot arm

There are myriad machines that are used to manufacture goods of all kinds. I am choosing to focus on these three forms because they are immediately approachable, relatively inexpensive, easy to build, and can be used to both create immediately useful and sellable goods, as well as create the parts necessary to build more and different machines.

The machine I built is the Lowrider CNC 3, by V1 Engineering. This machine is very simple, primarily able to cut sheet goods, with an emphasis on full sheets (8’x4′) of plywood, MDF, and the like. This is NOT the machine you’re going to use to cut an engine block out of a billet of aluminum. There are many ways to get started on this one. There are a number of parts that are 3D printed. It also uses ‘rails’, which are nothing more than tubing you can buy at any hardware store (electrical conduit tubing). There are bits and bobs of hardware (screws, nuts, bolts, timing belts, motors, electrical board, linear rails) which you can source yourself, or you can just buy the hardware kit, for $306 USD. For this first one, I purchased the hardware kit, and printed all the necessary parts myself. Save a little money on printing, at the expense of spending a fair amount of time doing it. After gaining experience with the first one, I’ve embarked on building a second one.

Same machine, sample printed parts, this time in PETG instead of PLA for some of them. I’d say the printed parts cost roughly $50 in plastic, and a couple days of printing, depending on what kind of machine you have. Mostly I print on my prusa mini+, because it’s relatively fast. The larger part I had to print on the larger Prusa MK3, because it is too big for the mini.

Once you’ve got all your printed parts, and hardware kit, you’re ready to assemble. The instructions are very easy, and step by step. You don’t need really fancy tools. A screwdriver, couple of socket wrenches, and that’s about it. Following the instructions, I was able to assemble in about 3 days of casual time. This could easily go together within a few hours in a single day, assuming you’re well organized with your tools and a nice workspace.

Alright then, you’ve got a basic machine. Next up are the electronics.

This is the jackpot board. It was designed specifically to be a low cost brain for various kinds of CNC machines. In this case, we’re doing a large format CNC router table, but it can also run a laser cutter, or any kind of machine with up to six motors. It’s all you need when you’ve got small motors (up to nema 17 realistically). It has the motor drivers onboard, and a ESP 32 compute module is the heart of its intelligence.

The board is running a firmware called fluidnc. Fluidnc does two things on this board. The first is, it takes a machine description file, and uses that to understand how to move the motors based on various commands. The second is, it interprets the CNC commands that are your design, and generates the appropriate movements based on those commands “cut this arc, move this much in that direction, lift the cutter…”.

Another thing this board does is support a web interface.

This web interface is enough to get you going. You can upload your design to be cut, over the network, press the button to start, and away it goes.

The ESP32 compute module is doing a pretty heavy lift here. It’s running the brains of the machine, sending movement commands to the motors, plus putting up a web interface, plus responding to commands from that interface. All of that from a compute module that costs about $10!! That’s what I call a true computicle!

OK, this is all great. I can build a machine rather quickly, and inexpensively. It has a brain, and it can cut stuff. Now what?

Well, I’m a software guy, so all of this is ultimately in service to a software solution right? I want to put “AI” to use. Ideally, I’d be able to articulate some design, working with something like ChatGPT, and send that design to the machine to be manufactured. To that end, I’ve started creating my own GPT (powered by ChatGPT).

I’ve been training CNC Buddy for a while now. I’ve told it about fluidnc, how you create a configuration file, what gcodes it supports, etc. I’ve gone back and forth, telling it the challenges I had while building my machine, and how I overcame them. CNC Buddy, being a “GPT” knows language stuff. You could actually ask it to help you generate a gcode file, if you’re really good at prompt engineering. More than likely though it will point you at a CAD/CAM program for that.

I find CNC Buddy to be most useful in answering questions I might typically post in a build forum, or asking some experienced person. This is great for classroom environments, as the collective can enhance the knowledge base of CNC Buddy with new experiences, making it better at answering questions in the future.

So, that’s where we are today. The basic machine is done, and capable of doing some cuts.

Where do we go from here? Well, after the excitement of “just get it running”, I will now go back and clean up this machine. The wiring can be tidied up mostly, and I need to make some improvements to the table this sits on. In the meanwhile, I will build another one, pretty much the same as this one, but for a larger full sized table.

The beauty and benefit of this large format gantry CNC is that other things can be done. I’m not going to try and cut metal with this, although it can cut aluminum. I’m going to do things like mount a laser diode, and possibly a 3D printing head. Then the machine becomes more than it was intended for. This is a base platform for enabling all sorts of automated tasks, where having a tool of some sort mounted to a low gantry is useful. I will also be looking at fluidnc, with an eye towards making it smaller, and simpler. It’s great that it hosts a web site as part of its duties, but I don’t actually want that UI running there, so alternatives.

At any rate, this is the first of the 3 machines we’re currently building. If we want to manufacture at the speed of thought, we need to start with being able to affordably build machines that give us manufacturing capabilities, so here we are.


About that robot…

  • A robot may not injure humanity or, through inaction, allow humanity to come to harm.
  • A robot may not harm a human being.
  • A robot must obey the orders given it by human beings except where such orders would conflict with the First Law
  • A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
  • A robot must know it is a robot.

Isaac Asimov popularized the “Three Laws of Robotics” in his Robot series, starting in 1940. Asimov is a pillar of modern day science fiction. Not “just a writer”, he was a full on scientist, philosopher, and all around renaissance man. Asimov gave us the word “robotics”, and here we are!

What I find most interesting about his earliest explorations in this area is that he put a lot of time (50 years) into thinking about how these automaton would interact with humanity, and how humans would interact with them. Written into his “Three laws of robotics” were an obvious attempt to keep humans relevant and unharmed by the advancement of robotics. You might think he had a deference to humans and only viewed robots as slaves, but that is not the case. He ends up with a master universe scale robot that guides and protects “humanity’s” evolution over the span of millenia.

If you’re wondering what all the hubub is about, I strongly suggest you read the “Foundation Trilogy”, or even start with “iRobot”. Then you’ll say “Hay, wasn’t that the Will Smith movie?”, or “Hay, isn’t “Foundation” on Apple TV right now? Yup, this is where it comes from alright.

Right now, in 2024, we’re marking through AI and Robots at a very rapid clip. The ink has barely dried on our understanding of how to program computers, and we’ve launched full tilt into the creation of autonomous driving, Large Language Models, and humanoid robotics. Of course, “automation” writ large is nothing new. Humanity has been automating since we first picked up tools to smash insects open on rocks. Now is something new though. Now, particularly with LLMs, we’re getting closer to the automation of intellect, and not just labor. All the tools to date have helped up smash bugs better. Food production with automated farm equipment generates yields our grandparents could only dream of. We can produce goods in factories, turning out cars in hours, rather than days. We can even produce rocket engines by the hundreds a year at this point. All simple automation improvements.

But now, we’re automating intellect, meaning, you don’t need that human in the loop to generate the quarterly report. You don’t need the human in the loop to come up with the letter to send to employees wishing them happy holidays. In some cases, you don’t need humans in the loop to come up with the design of a new product, or an original score for a movie, or the entirety of a movie. With emerging tools, such as Sora, you can prompt your way to creating video scenes that are unique, and completely derived from the dream state of an AI.

I do ask myself this though; Where are the three+ laws of robotics?

Are we rushing headlong down this path to automation without any concern for whether humanity is served, or to be served up?

Humanoid robots are particularly interesting, because they embody all the physical automation stuff, while simultaneously being able to embody the intellectual automation as well.

At this moment, there are several efforts under way to create humanoid robots. In some cases, they’re just toys for the kids. Something to kick a little socker ball, or play nursery rhymes. In other cases, they are serious affairs, meant to replace humans on assembly lines in factories. In some cases, they don’t have humanoid features, just a head with some lights. In others, they are striving for human realism, with full on facial expressions and “skin”.

I don’t think there are any universal “three laws of robotics” though, and there probably should be. Having worked at Microsoft, I know there was some effort to put “ethics” into the AI development space, but those efforts were largely around where and how to source the raw data to feed the emerging AI systems, it wasn’t close to talking about how humanoid robots should behave in society. It’s probably similar in other places.

I have gathered a watch list of companies that are engaged in the development of humanoid robotics. I want to watch how they do, not just on the hardware evolution, but on this question of ‘ethics’, or generally how we’re going to evolve these capable beings. Do they become our servants, an evolved form of humans, standing next to us, or our masters and commanders?

A few companies to watch for humanoid robot development

  • Tesla – Optimus
  • Figure AI – Figure 01
  • Boston Dynamics – Atlas
  • Unitree – H1

Why now? There is a confluence of factors making humanoid robotics more of a thing now than ever in the past. One are the myriad breakthroughs in machine learning. The LLM market has certainly shown the way, but essentially, we’ve reached a point where ‘training’ is the way to evolve a system, rather than “programming”. It’s easier to show a robot videos of a task, and say “do this”, than to write a ton of code with error conditions, branches, corner cases, and the like. So, that’s a breakthrough.

On the mechanical side, there are advances in batteries (smaller, more power density, cheaper), driven by the revolution in the Electric Vehicle industry. At the same time, compute density continues to increase every year, and become cheaper. And lastly, new kinds of electric motors, actuators, and sensors are being created, partly because of EVs again, and partly because MEMS technologies continue to miniaturize and become more ubiquitous.

The last bit of this is the training models. Being able to simulate an environment as accurately as possible is critical to the rapid training evolution of these new systems. nVidia has been providing the compute density and leading the way for simulation of all manner of environments for AI training.

Couple all this with the likes of Microsoft, Facebook, Apple, Google, nVidia, and you have the world’s most valued companies investing in a space that is the stuff of science fiction. There is no doubt. We will have practical humanoid robots capable of accomplishing typical human tasks, within 2024, or 2025 at the lastest. There will be tons of side benefits along the way from continue evolution of electronics, battery, and materials tech. The real question is, will we evolve humanity to match?

This is the essential question. Asimov thought about, articulated, refined, and put into words the laws of robotics, because he was concerned with how humanity would evolve with these new tools. Now that we’re on the precipice of actually delivering on the humanoid robot promise, are we putting in the same consideration, or are we in a headlong rush to push product out the door, no matter the cost or consequence to humanity?

This is one area I want to give some focus to. I think there’s tremendous benefit to be had from continuing down this evolutionary path. I want to do it in such a way that humanity doesn’t end up on the evolutionary scrap heap.


Home Automation – 5 years of experience

It was 5 years ago when I wrote about my home automation efforts: Home Automation – Choosing Bulbs

Back then, I was enthusiastic about replacing the myriad halogen bulbs in my house with LED based bulbs, and while I was at it, ‘automating’ them as well. I installed Philips Hue, some Lutron dimmers, and hooked it all up to Alexa so I could say “downstairs lights on!”. So, 5 years on, how has it gone?

Well, aside from doing the occasional demo of turning the lights on and off by talking to the ever present speaker, we really don’t use the automation at all. Why not? It’s far easier to just flick the light switch on or off when you enter/exit a room, and we never really got into ‘automation’, like setting up on/off patterns when on vacation.

These past few days, I decided to have another go at it. Throughout the house, I’ve been retiring various consumer electronics related to TVs, lights, computers, and the like. I won’t say that I’m an Apple fan boi, but they have a lot of kit in their ecosystem that “just works”. So, what have I done?

Lighting

Well, I’ve replaced the Philips Hue lights with those from Nanoleaf. Beyond just basic light bulbs, they have lots of panels, strips, shapes, to play with. I just bought a 3 pack of Matter A19 bulbs from the Apple store.

Simple, wifi enabled, you add them to the Apple Home app, and then you can control them in all the usual ways. I’ve re-purposed an older iPad to play the role of “home control” tablet, so I can change light values and whatnot.

We’re not walking through the house saying “Siri, lights on”, but I can set their color/temperature, so that when they do come on, they’ll be a certain value.

This is easy enough, and allows me to eliminate one bit of kit from the house, the Phillips controller, which was located in my office, which is not actually ideal because lights that are far from that spot might have trouble connecting.

Video Streaming

Over the years, we’ve used all manner of video streaming device. Way back in the day, it was ripping DVDs to the Synology NAS, and streaming through plex on the laptop. Then streaming from the same on the advanced DVD player. Commercial stuff was first streamed on various incarnations of roku, then along came Amazon fire sticks, then along came google tv, then finally Apple tv.

Given current prices of cheap LED tvs in the 55″ category, these apple tv boxes are not cheap ($130-$150), but, they are pretty darned good. Currently capable of 4K video. With the right HDMI cable, and a 4K screen, such a super sharp picture. I know that technology continues an invevitable mark of progress, but I’m ok with 4K output on the streaming box for a while. The only 4K screens we have in the house right now are in my office, and not even our main viewing screens. Upgrading our main screen to 4K will be a big upgrade, so these things will have life for a few years now that they are installed. The other benefit is they tie into the same Apple Home app that the lights do, so I can control them from one place also.

The last part is ‘smart speaker’. We’ve had the Echo Dot from Amazon from as early as they were available. We bought the dot, the cylinder, and even the one with a display (short lived experiment). What we’ve found through the years is the only real use case for us has been playing music at bed time. Other than that one hour of the day, and the occasional kids session on the weekends, we leave this thing unplugged, not wanting to allow Amazon to snoop in our conversations, and make purchasing suggestions based on what it heard.

Nope. Instead, I’m going to plug in the Apple HomePod mini I bought a couple years back.

I’m already paying for Apple music anyway (for listening on my bike rides), so why not use it more. It will sit in my office, not out in the house. I trust Apple more than Amazon, when it comes to not selling me to advertisers.

This all seems like a hard core Apple commercial, and until now I hadn’t realized how much Apple kit has replaced a bunch of other stuff. But, I guess they’re the largest consumer electronics company in the world for a reason (by stock value). I’ve gotta admit, when you’re not wanting to get into the guts of how it works, and tweak to the nth degree, it just works, and provides a coherent ecosystem.

So, 5 years on, what’s happened? We’ve replaced all the random experimental stuff from companies that have varying degrees of support, to be a full on apple home, and I have no regrets. We’re still not turning lights on with our voices, but it’s a relief to just de-commission random bits of tech that served half purposes. We’ll see what things look like in the next 5 years.

I am thinking I want a whole home “virtual assistant”, I mean, given the march of AI and all. So, I’m curious who will be the purveyor of that. As far as the home is concerned, Apple might just take that crown. Until then though, I’m sure I’ll be experimenting with a lot of stuff to shake out the winner that works for us.


Tinkerer’s Closet – Selecting hardware

Well, there’s no better reason to clean out your closet or workspace, than to fill it with new stuff!! I’ve been spending a fair amount of time cleaning out the old stuff, and I’ve gotten far enough along that I feel it’s safe to poke my head up and see what’s new and exciting in the world of hardware components today.

What to look at though, so I don’t just fill up with a bunch more stuff that doesn’t get used for the next 10 years? Well, this time around I’m going project based. That means, I will limit my searching to stuff that can help a project at hand. Yes, it’s useful to get some items just for the learning of it, but for a hoarder, it’s better to have an actual project in mind before making purchases.

On the compute front, I’ve been standardizing the low end around ESP 32 modules. I’ve mentioned this in the past, but it’s worth a bit more detail. The company, Espressif, came along within the past decade, and just kind of took the maker community by storm. Low cost, communications built in (wifi, bluetooth), capable processors (32-bit). They are a decent replacement at the low end of things, taking the place of the venerable Arduino, which itself was a watershed in its day.

The keen thing about the Espressif modules is how programmable they are. You can use the Aruino IDE, or PlatformIO (tied to Visual Studio), or their standalone IDE. You can program it like a single CPU with full control of everything, or you can run a Real-time OS (FreeRTOS) on it. This makes it super easy to integrate into anything from simple servo motor control, to full on robotics.

As for form factor, I’m currently favoring the Adafruit ‘feather’ forms. The ‘feather’ form factor is a board specification, which puts pins in certain locations, regardless of which actual processor is on the board. This makes it a module that can be easily integrated into designs, because you have known patterns to build around. I have been using the ESP32 Feather V2 primarily.

It’s just enough. USB-C connector for power, and programming. Battery connector for easy deployment (battery charges when USB-C is plugged in). STEMMA QT connector (tiny 8 pin connector) for easy I2C connection of things like joysticks, sensors, anything on the I2C bus. Antenna built in (wifi/bluetooth radio on the right, with black PCB antenna next to it).

It’s just a handy little package, and my current best “computicle”. You can go even smaller, and get ESP 32 modules in different packages, but this is the best for prototyping in my lab.

As an aside, I want to mention Adafruit, the company, as a good source for electronic components. You can checkout their about page to get their history. Basically, they were created in 2005, and have been cranking out the hits in the maker space ever since. What attracted me to them initially was their tutorials on the components they sell. They have kits and tutorials on how to solder, as well as how to integrate motors into an ESP 32 design. Step by step, detailed specs, they’re just good people. They also pursue quality components. I mean, every USB cable is the same right? Nope, and they go through the myriad options, and only sell the best ones. So, if you’re in the market, check them out, at least for their tutorials.

Going up the scale from here, you have “Single Board Computers”. The mindshare leader in this space is definitely the Raspberry Pi. When they sprung onto the scene, there really wasn’t any option in the sub-$50 range. Since then (2006ish), there has been an entire renaissance and explosion of single board computers. They are typically characterized by: Arm based processor, 4-8Gb RAM, USB powered, HDMI output, a couple of rows of IO pins, running Linux (Ubuntu typically).

I’ve certainly purchased my share of Raspberry Pi boards, and variants. I tend to favor those coming from Hard Kernel. I find their board innovations over the years to be better than what the Pi Foundation is typically doing. Also, they are more readily available. Hard Kernel has commercial customers that use their boards in embedded applications, so they tend to have Long Term Support for them. They have boards based on ARM typically, meant to run Linux, but they also have Windows based boards as well.

Here’s a typical offering,

The Odroid M1S.

The one thing that’s critical to have in a single board computer is software support. There are as many single board computers available in the world as there are grains of sand on a beach. What differentiates them is typically the software support, and the community around it. This is why the Raspberry Pi has been so popular. They have core OS support, and a super active community that’s always making contributions.

I find the Odroid boards to be similar, albeit a much smaller community. They do have core OS support, and typically get whatever changes they make integrated into the mainline Linux development tree.

This M1S I am considering as a brain for machines that need more than what the ESP32 can handle. A typical situation might be a CNC machine, where I want to have a camera to watch how things are going, and make adjustments if things are out of wack. For example, the camera sees that the cutting bit has broken, and will automatically stop the machine. Or, it can see how the material is burring or burning, and make adjustments to feeds and speeds automatically.

For such usage, it’s nice to have the IO pins available, but communicating over I2C, CANBus, or other means, should be readily available.

This is reason enough for me to purchase one of these boards. I will look specifically for pieces I can run on it, like OpenCV or some other visual module for the vision stuff. I have another CNC router that I am about to retrofit with new brains, and this could be the main brain, while the ESP32 can be used for the motor control side of things.

Last is the dreamy stuff.

The BeagleV-Fire

This is the latest creation of BeagleBoard.org. This organization is awesome because they are dedicated to creating “open source” hardware designs. That’s useful to the community because it means various people will create variants of the board for different uses, making the whole ecosystem more robust.

There are two special things about this board. One is that it uses a RISC-V chip, instead of ARM. RISC-V is an instruction set, which itself is open source, and license free. It is a counter to the ARM chips, which have license fees and various restrictions. RISC-V in general, will likely take up the low end of the market for CPU processors in all sorts of applications that typically had ARM based chips.

The other feature of these boards is onboard integrated FPGA (Field Programmable Gate Array). FPGA is a technology which makes the IO pins programmable. If you did not have a USB port, or you wanted another one, you could program some pins on the chip to be that kind of port. You can even program a FPGA to emulate a CPU, or other kinds of logic chips. Very flexible stuff, although challenging to program.

I’ve had various FPGA boards in the past, and even ones that are integrated with a CPU. This particular board is another iteration of the theme, done by a company that has been a strong contributor in the maker community for quite some time.

Why I won’t buy this board, as much as I want to; I don’t have an immediate need for it. I want to explore FPGA programming, and this may or may not be the best way to learn that. But, I don’t have an immediate need. Getting an Odroid for creating a smarter CNC makes sense right now, so one of those boards is a more likely purchase in the near term. It might be that in my explorations of CNC, I find myself saying “I need the programmability the BeagleBone has to offer”, but it will be a discovery based on usage, rather than raw “I want one!”, which is a departure from my past tinkerings.

At this point, I don’t need to explore above Single Board computers. They are more than powerful enough for the kinds of things I am exploring, so nothing about rack mountable servers and kubernetes clusters.

At the low end, ESP32 as my computicles. At the high end, Hard Kernel based Single Board Computers for brains.


Tinkerer’s Closet – Hardware Refresh

I am a tinkerer by birth. I’ve been fiddling about with stuff since I was old enough to take a screwdriver off my dad’s workbench. I’ve done mechanical things, home repair, wood working, gardening, 3d printing, lasering, just a little bit of everything over the years. While my main profession for about 40 years has been software development, I make the rounds through my various hobbies on a fairly regular basis.

Back around 2010, it was the age of 3D printers, and iOT devices. I should say, it was the genesis of those movements, so things were a little rough. 3D printers, for example, are almost formulaic at this point. Kits are easily obtained, and finished products can be had for $300-$400 for something that would totally blow away what we had in 2010.

At that time, I was playing around with tiny devices as well. How to make a light turn on from the internet. How to turn anything on, from a simple radio controller. As such, I was into Arduino microcontrollers, which we making the rounds of popularity, and much later, the Raspberry Pi, and other “Single Board Computers”. There were also tons of sensor modules (temperature, accelerometers, light, moisture, motion, etc), and little radio transmitters and receivers. The protocols were things like Xigbee, and just raw radio waves that could be decoded to ASCII streams.

As such, I accumulated quite a lot of kit to cover all the bases. My general moto was; “Buy two of everything, because if one breaks…”

The purchasing playground for all this kit was limited to a few choice vendors. In the past it would have been Radio Shack and HeathKit, but in 2010, it was:

AdaFruit

SeeedStudio

SparkFun

There were myriad other creators coming up with various dev boards, like the low power JeeLabs, or Dangerous Prototypes and their BusPirate product (still going today). But, mostly their stuff would end up at one of these reliable vendors, along with their own creations.

Lately, and why I’m writing this missive, I’ve been looking at the landscape of my workshop, wanting to regain some space, and make space for new projects. As such, I started looking through those hidey holes, where electronics components tend to hide, and hang out for generations. I’ve started going through the plastic bins, looking for things that are truly out of date, no longer needed, never going to find their way into a project, no longer supported by anyone, and generally, just taking up space.

To Wit, I’ve got a growing list of things that are headed for the scrap heaps;

433Mhz RF link kit, 915Mhz RF Link Kit, Various versions of Arduinos, Various versions of Raspberry Pi, TV B Gone KIt (built one, tossing the other, maybe save for soldering practice for the kids), Various Xigbee modules, Parallax Propellar (real neat stuff), SIM Card Reader, Gadget Factory FPGA boards and wings, trinkets, wearables, and myriad other things either as kits, boards, and what have you.

I’m sad to see it go, knowing how lovingly I put it all together over the years. But, most of that stuff from from 13 years ago. Things have advanced since then.

It used to be the “Arduino” was the dominant microcontroller and form factor for very small projects. Those boards could run $30, and weren’t much compared to what we have today. Nowadays, the new kids in town are the ESP 32 line of compute modules, along with form factors such as the Adafruit supported “Feather”. A lot of the modules you used to buy separate, like Wifi, are just a part of the chip package, along with BlueTooth. Even the battery charging circuitry, which used to be a whole separate board, is just a part of the module now. I can buy a feather board for $15, and it will have LiPo charging circuitry, USB-C connectivity for power and programming, Wifi (abgn), and BlueTooth LE. The same board will have 8 or 16Mb or RAM, and possibly even dual cores! That’s about $100 worth of components from 2010, all shrunken down to a board about the size of my big thumb. Yes, it’s definitely time to hit refresh.

So, I’m getting rid of all this old stuff, with a tear in my eye, but a smile on my face, because there’s new stuff to be purchased!! The hobby will continue.

I’m happily building new machines, so my purchases are more targeted than the general education I was pursuing back then. New CPUs, new instructions sets, new data sheets, new capabilities, dreams, and possibilities. It’s both a sad and joyous day, because some of the new stuff at the low end even has the words “AI Enabled” on it, so let’s see.


Embodied AI – Software seeking hardware

The “AI” space writ large, covers an array of different topics. At this moment in time, the Large Language Models (LLMs) have captured everyone’s imagination, due to their uncanny ability to give seemingly good answers to a number of run of the mill questions. I have been using ChatGPT specifically for the past year or so, and have found it to be a useful companion for certain tasks. The combination I use is GitHub Copilot, in my Visual Studio development environment, and ChatGPT on the side. Copilot is great for doing very sophisticated copy and paste based on comments I type in my code. ChatGPT is good for exploring new areas I’m not familiar with, and making suggestions as to things I can try.

That’s great stuff, and Microsoft isn’t the only game in town now. Google with their Bard/Gemini is coming right along the same path, and Facebook isn’t far behind with their various llama based offerings. I am currently exploring beyond what the LLM models provide.

One of the great benefits I see of AI is the ability to help automate various tasks. Earlier in the 20th century, we electrified, and motorized a lot of tasks, which resulted in the industrial revolution, giving us everything from cars to tractors, trains, airplanes, and rockets. Now we sit at a similar nexus. We have the means to not just motorize everything, but to give everything a little bit of intelligence as well. What I’m really after in this is the ability to create more complex machines, without having to spend the months and years to develop the software to run them. I want them to ‘learn’. I believe this can make the means of production of goods accessible to a much broader base of the population than ever before.

What I’m talking about is manufacturing at the speed of thought. A facility where this is done is a manufactory.

In my idealized manufactory, I have various semi-intelligent machines that are capable of learning how to perform various tasks. At a high level, I want to simply think about, and perhaps visualize a piece of furniture, turn to my manufactory and say “I need a queen sized bed, with four posts, that I can assemble using a screwdriver”. What ensues is what you might expect from a session with ChatGPT, a suggestion of options, some visualization with some sort of Dall-E piece, and ultimately an actual plan that shows the various pieces that need to be cut, and how to assemble them. I would then turn these plans over to the manufactory and simply say “make it so”, and the machinery would spring into life, cutting, shaping, printing, all the necessary pieces, and delivering them to me. Bonus if there is an assembly robot that I can hire to actually put it together in my bedroom.

Well, this is pure fantasy at this moment in time, but I have no doubt it is achievable. To that end, I’ve been exploring various kinds of machines from first principles to determine where the intelligence needs to be placed in order to speed up the process.

I am interested in three kinds of machines

CNC Router – Essentially a router, or spindle, which has a spinning cutting bit. Typically rides on a gantry across a flat surface, and is capable of carving pieces.

3D Printer – Automated hot glue gun. The workhorse of plastic part generation. Basically a hot glue gun mounted to a tool head that can be moved in a 3D space to additively created a workpiece.

Robotic Arm – Typically with 5 or 6 joints, can have various tools mounted to the end. Good for many different kinds of tasks from welding, to picking stuff up, to packing items into a box.

There are plenty of other base machines, including laser cutters, milling machines, lathes, and presses, but I’ve chosen these three because they represent different enough capabilities, but they’re all relatively easy to build using standard tools that I have on hand. So, what’s interesting, and what does AI have to do with it?

Let’s look at the 3D Printer.

the100 – This is a relatively small 3D printer where most of the parts are 3D printed. The other distinction is holds is that it’s super fast when it prints, rivaling anything in the consumer commercial realm. The printability is what drew me to this one because that means all I need to start is another relatively inexpensive ($300) 3D printer to start. And of course once the100 is built, it can 3D print the next version, even faster, and so on and so forth.

The thing about this, and all tools, is they have a kinematic model. That is, they have some motors, belts, pulleys, etc. Combined, these guts determine that this is a machine capable of moving a toolhead in a 3D space in a certain way. I can raise and lower the print bed in the Z direction. I can move the tool head in the XY direction. The model also has some constraints, such as speed limits based on the motors and other components I’m using. There’s also constraints as to the size of the area within which it can move.

The way this is all handled today is clever people come up with the programs that tie all this stuff together. We hard code the kinematic model into the software, and run something like Klipper, or Marlin, or various others, which take all that information, are fed a stream of commands (gcode), and know how to make the motors move in the right way to execute the commands.

There is typically a motherboard in these machines that has a combination of motor control and motion control, all wrapped up in a tight package.

I want to separate these things. I want motor control to be explicit, and here I want to inject a bit of AI. In order to ’embody’ AI, I need to teach a model about it’s kinematics. From there, I want to train it on how to move based on those kinematics. I don’t want to write the code telling it every step how to move from point A to B, which is what we do now. I want to let it flop around, giving it positive re-enforcement when it does the right thing, and negatives when it doesn’t. Just like we do with cars, just like we do with characters in video games. This is the first step of embodiment. Let the machine know its senses and actuators, and encourage it to learn how to use itself to perform a task.

Basic motor control is something the model needs to be told, as part of the kinematic model. Motion control is the next level up. Given a task, such as ‘draw a curved line from here to there’, which motors to engage, for how long, in which sequence, when to accelerate, how fast, how long, best deceleration curve, that’s all part of the motion control, and something a second level of intelligence needs to learn.

On top of all that, you want to layer an ability to translate from one domain to another. As a human, or perhaps another entity in the manufacturing process, I’m going to hand you a ‘.stl’ or ‘.step’ or various other kinds of design files. You will then need to translate that into the series of commands you understand you can give to your embodied self to carry out the task of creating the item.

But, it all starts down in motor control, and kinematic modeling.

Next up is the CNC Router

This is the lowrider 3 by V1 Engineering. What’s special here again is the ease of creating the machine. It has mostly 3D printed parts, and used standard components that can be found at a local hardware store. At it’s core is a motor controller, which is very similar to the ones used in the 3D printer case. Here again, the machine is running in a pretty constrained 3D space, and the motor control is very similar to that of the 3D printer. These two devices run off different motherboards, but I will be changing that so they essentially run with the same brain when it comes to their basic motor control and kinematic understanding.

Whereas the 3D printer is good for small parts (like the ones used to construct this larger machine), the CNC router, in this case, is good for cutting and shaping of sheet goods, like large 4ftx8ft sheets of playwood for cabinet and furniture making. Giving this platform intelligence gives us the ability to send it a cut list for a piece of furniture and have it figure that out and just do it.

Of course, these capabilities exist in larger industrial machines, that have typically been programmed, and are tied to CAD/CAM software. Here though, I’m after something different. I don’t want to “program” it, I want to teach it, starting from the base principles of its own kinematics.

Last is the venerable Robot Arm

Here, I am building a version of the AR4 MK2 robot arm from Annin Robotics

This machine represents a departure from the other two, with 6 degrees of freedom (shoulder, elbow, wrist, etc). The motors are larger than those found in the 3D printer or CNC router, but how to control and sense them is relatively the same. So, again, ultimately I want to separate sense and motor control from motion control. I will describe a kinematic model, and have the bot learn how to move itself based on reinforcement learning on that model.

All of this is possible now because of the start of the technology. Microcontrollers, or very small computers, are more than capable of handling the complex instructions to control a set of motors. This is a departure from just 10 years ago when I needed a complete real-time Linux PC with a parallel port to control the motors alone. Now I can do it with an esp32 based device that costs less than $20, and can run off a hobby battery. Similarly, the cost of ‘intelligence’ keeps dropping. There are LLMs such as llama.cpp which can run on a Raspberry pi class machine, which can be easily incorporated into these robot frames.

So, my general approach to creating the manufactory is to create these robot frames from first principles, and embody them with AI as low as we can go, then build up intelligence from there.

At this time, I have completed the AR4 arm, and the Lowrider CNC. the100 printer is in progress, and should complete in a couple of weeks. Then begins the task of creating the software to animate them all, run simulations, train models, and see where we get to.


It’s all about the Artificial Intelligence?

I can remember when in the 1980s and 90s, we programmers were talking about things such as SmallTalk, Lisp machines, the Prolog programming language, and this upstart C++. Lots of discussions around modularity, simulation, patterns of programming and whatnot. Back then, we even had the discussions around neural networks, back propagation, and even nano scale computers, driven by push rods…

The machines of the time were not anywhere near as capable as even my now ‘older’ iPhone 12. I mean, a machine with a few megabytes of RAM, let alone a hard disk with a 100Megabytes, would have been extraordinary. And yet, we envisaged the rise of the machines (Terminator – first DVD ever!), and if you were a programmer, “Snowcrash” fueled fantasies of a world of connected intelligence, that drove a generation to create the distributed multi-player gaming environments we have today.

If we had then, what we have now, in terms of hardware, would we already be served by our robotic assistants? Probably not, but we’ve had decades to stew on the science of the tech, to refine the ‘artificial’ in the intelligence, and finally, the neural networks of yore have enough horsepower and training data today to fulfill a lot of the fantasy operations we envisaged back then.

I have mentioned a few key technologies in newsletters over the past year, so here I want to launch into a bit more detail about what I’ve been using, and how I see it impacting the future of programming. I have been working with Microsoft recently on the topic of “The Future of Work”. In particular, I have been exploring how roles change, how engineering itself changes, and how and when we might expect those changes.

One particular thing of note is that it’s very hard for anyone, even those creating the tech, to predict which specific features will be emerging in any given timeframe. Just like early on, it was hard to know when a machine might be able to beat a chess grand master (Feb 1996). Instead of trying to predict the feature futures, I’ve instead started to predict a chronology. For example, I don’t know when copilot or ChatGPT will be able to write code for any given situation better than I can, but I know this will happen eventually. Knowing this, I also know that my job as a ‘software engineer’ will change over time, to the point where I am no longer doing the mundane code writing, but have switched to the more abstract systems design, so I can prepare for that.

Artificial Intelligence is a very broad term of course. In common speech, I take it to mean “that which we don’t quite understand, but we know it’s related to computers, and it’s going to be taking my job some day”. This can be a fearful interpretation. It’s the unknown. Much like the emergence of the ‘horseless carriage’, it might cause some fear at first, but over time, with gained experience, it becomes less a thing to fear, and more a tool to fuel a whole new industrial age.

What are some of the sign posts in my chronology? I’ll use “computers” to represent the thing that is evolving.

  1. Computers will be able to assist in the development of software. That assistance will be rudimentary, more of a ‘copy/paste’ on overdrive, but it will accelerate software writing by 30%
  2. Computers will be able to create a small scale system based on domain knowledge and coding abilities.
  3. Computers will be able to interpret large scale system designs and generate and verify code for such systems, according to a human derived specification
  4. Computers will be able to create systems on the fly via human conversational interactions.
  5. Computers will be able to create new systems without human input, to satisfy the needs of a group of people, or other entities.

Those are sign posts along a journey of development, only considering software development. Over the next few months, I will explore further what each of these sign posts might really look like, and what we can do to prepare for their arrival and maximize our benefit from them.

I’ve been doing tech for 40+ years at this point. It’s been a year since I left Microsoft, and for that entire time I’ve been using the likes of ChatGPT and GitHub copilot to enhance my coding capabilities. I have been able to create a level and quantity of software in that short amount of time that I would not have been able to do in the past without such tools. We are at an inflection point, where the tools are good enough that we need to make very conscious choices about how to engage and use them to ensure we reap the benefits to the betterment of humanity, rather than cowering in fear of what might be done to us.


Impressions – One week in a Tesla Model Y

We recently took a family vacation to Southern California. Some fun in the sun at the beaches, with a side trip to Disneyland to boot. We had first considered driving down from Seattle to Newport Beach, a drive we’ve actually done before, but we ultimately decided to fly down instead, and rent a car.

When I went to the Hertz site, I saw they had Tesla’s for rent, at a lower price than a comparable sedan, so I thought, “why not, I’ve never actually driven one before, let’s see what all the fuss is about”. So, a Tesla Model Y would await.

After a brief scare at the rental facility (you’re going to wait 3 hours to get the car), turns out, it was actually sitting right there waiting for us (thank you super credit card).

Now, I’m clearly a tech head, and have known about and followed the Tesla story from day one. Surprisingly, I’ve never actually driven one in all these years. I’ve read tons of reviews, and even own some shares of the stock, but actually experiencing the hype, not so much. My perspectives are a mix of technical analysis, and pragmatics. I’ll start with the technical perspective.

You get in, it’s a car, solid enough, comfortable enough, slow roll out of the rental facility at 10mph. When can I hit that accelerator and be thrown back into my seat like I’m launching a jet fighter? OK, we finally hit the freeway and… POW!! Shazowaa!! What a rush. OK, that lasted about 3 seconds, now settle down.

The drive

First is the ‘accelerator’ otherwise known as the ‘gas pedal’. Of course there’s no gas, but push the pedal on the right, and you go faster. Electric cars typically have regenerative braking. What that means is that the motor itself is used to help slow the car down, and while it’s doing that it generates electricity to recharge the battery a bit. You notice it most with slow city driving. Take your foot off the accelerator, and you instantly start to slow down. It took me several tries before I got good at stopping at the right place at a stop light. I was usually far short, because in our gas cars, taking your foot off the accelerator just puts the car into ‘roll’, it just keeps going til you put your foot on the brake. Of course, with the Tesla, you can actually tune this. There is a ‘roll’ mode, but I left the regen braking on because I wanted that new experience.

As far as handling, it’s a car, turn the wheel, and it turns. One thing I did notice was that when I put on the turn signal to change lanes, it would automatically turn off once we were in the lane. That’s different than my much older Toyota Venza, which requires a certain amount of physically turning the wheel before it “clicks” and turns itself off.

In general, it drives like a car. Nothing super spectacular, and no glaring omissions. Although I did not figure out how to use the full self driving, there was this “go the speed of traffic” feature we did use. I used it in stop and go traffic. Just turn it on, and you can remove your foot from the accelerator and brake. When the car in front of you moves, the Tesla will move. When it slows down or stops, the Tesla will follow suit. This is great for saving you feet from pedal fatigue.

Modernity

There are a class of conveniences that I think can be chalked up to simply being a modern car. My Venza is circa 2010, so a bit long in the tooth (and it lacks bluetooth playback). Many of the features of the Tesla can probably be found in any modern car. Of course bluetooth pairing, although it did not like my wife’s One+, but paired with my iPhone 12. Very curious that. Having Netflix and Youtube built in was a plus for when we visited the charging station (more on that later). There is a convenient ‘click to talk’ feature as well. Just click the button no the stearing wheel and say what you want “turn the AC up”. This is great, because since all the commands are hidden in the touch screen, and there are very few physical buttons in the cabin, having viable voice command is an absolute must.

Fit and Finish

In general, it seemed to be a fairly solid car. In particular the doors slam solid. The frunk though, that seems to be another story. That was the flimsiest hood I’ve every experienced. I thought I’d bend/break it if I slammed it down, rather than cradling it back into its closed position. I hope that improves over time, it seems like an oversight on an otherwise solid feeling car.

The ‘sun roof’ extends the whole length of the car. That was a funny one in the beginning because we were first looking for the ‘sun roof’ before we realized it was the whole roof.

Range/Charge Anxiety

Never having driven an electric, I was not familiar with how far you can go on a charge, when you should recharge, etc. Since we were staying at a hotel that did not have an on-premise charging station, I was probably more conscious about our state of charge each time we went out. We started at about 70%, and went 30 miles to the hotel, down to 50ish%. Then a trip to Long Beach, and we were looking for a super charger for the return trip. 20 minutes on the super charger and we were back to 80%. A few more local small trips, and by friday, we were back in the 40% range, and looking for another top up. Another super charger, another 20 minutes, another 80%. We did not bother charging before returning the car, but there was a charging station near the airport.

I can say I felt range anxiety. We were going to take a side trip to Arizona, and the internal mapping app showed we’d make it with a couple of charges along the way, but I thought better of it, and we did not take that trip.

I’m sure if I were a regular driver, and charging it at home, I would not feel the anxiety, and I’d love never having to go to a gas station again, ever.

Final Impressions

The Tesla Model Y is a great rental car. A dual motor, maxed out unit is quite nice. There’s a bit of getting used to, but if you already have a modern car, the differences are so minor, you get over them within minutes.

Would I buy one for my family? Well, no, not really. This didn’t come down to any technical features, it came down to pragmatics. We have a family of 4, with two young children. Our lives require a minivan, or equivalent. We need to hault stuff, bikes, equipment, multiple kids (beyond our own), and a sedan just doesn’t do it for us. I’m not quite sure how a model Y gets an “SUV” designation, but it’s not even a ‘station wagon’ like the Venza is.

For our family, we need utility, either that of a minivan (Chrysler Pacifica is our daily driver), or a king cab truck.

I’m desperate to get into electrical vehicles, and we’re just pumping out more pollution every time we drive today, but I want to make a practical choice if I can. I would love to get an all electric minivan, although I haven’t seen one in production that would foot the bill yet. Barring that, I would not mind a Ford F-150 lightning. I had money down on a Rivian originally, but cancelled at the last moment because the price was just too outrageous for what it was (a mid-sized truck).

In short, the Tesla Model Y was a great rental car experience, and I will likely rent one again next time. As a practical matter for our family though, it’s not the car for us to make as our next purchase.


Build vs Buy in software development – Part 1, the selection criteria

There is a common theme throughout life; “Should I build, or buy?” It doesn’t seem to matter which thing is under consideration, the process we used to make the decision should remain the same. This is a multi-part series on making the build vs buy decision for software projects.

This is WAAV Studio, a rapidly evolving application used for “Smart City” visualization and management. There are several components that go into it, and plenty of ‘build vs buy’ decisions to be made.

Here are the considerations I’ve had in deciding which way to go on several components

  • What is the scope of the component – How broad, and fundamental is it
  • Do I have the expertise to build it
  • How long will it take to integrate a purchased component
  • How long will it take to build it myself
  • What is the cost associated with purchasing it
  • How will maintenance look over time
  • The solution must be easy
  • The solution must be portable to multiple platforms
  • The solution must be small, fast, and intuitive

There are perhaps a couple more things to consider, but these are the highlights. There are some market considerations as well, but they essentially boil down to:

How important is time to market?;

and

What is your budget?

I’ll start with the WAAV Studio time to market and budget first. Time to market in this case is measured in months. A polished product needs to be available within a 12 month time period from when I started (January). The product must meet various milestones of usability along the way, but overall, the 1.0 version must be user friendly within 12 months.

Second, is the budget. This is the product of a very small team, using tools such as Copilot, and ChatGPT, as well as their core skills and programming tools. There is effectively no real budget to speak of, in terms of purchasing other bits of software.

With those hard constraints, I consider the list of ‘modules’ that need to be in this application.

  • Geo spacial mapping
  • Visualize various geo-spacial data sets
    • KMZ
    • GEOJSon
    • Shapefile
    • .csv data sets
  • Visualize various graphics assets
    • .png images
    • .gif images
    • .jpeg images
    • .svg images

That’s the rough list, with lots of detail on each item. The biggest and first build/buy decision was around mapping visualization. The easy and typical answer would be “just use Google Earth”, or something like that. Even before that, it might be “Use ArgGIS”, or any number of GIS software packages. Going this route might be expedient, but will lead you down a path of being constrained by whatever that core package is capable of.

A few of the criteria are around ease of use, and size. This application is something a typical city administrator will use on occasion. Their key job function might not be related to this software, so they need to be able to come to it after 3 months of absence, and still be able to pick it up and use it effectively to achieve some immediate goal. This is a hard one to achieve, and has to do with the UI elements, their layout, and natural usage. Again, when you select a core package, you may not have enough control over these factors to have a satisfying outcome. Using ArcGIS, for example, it has all the features a GIS professional could possibly want. The package size is measured in the 10s of megabytes, and the user’s manual would make an encyclopedia blush if it were printed in paper. This is not an app that can be picked up by your typical city clerk in a 10 minute session, let alone mastered six months later, without constant usage.

First decision: Create a core mapping component from scratch, without reliance, or dependence on any existing mapping components.

This is such a core, fundamental decision, it drives decision making across the other components, so, it better be good.

I have never built a mapping platform before WAAV Studio, so I started with the naive notion that I could actually do it. I mean, how hard could it be, right? All software engineers have the urge to build from scratch, and just jump onto any coding challenge. My years of wisdom told me, I better have a way to evaluate my progress on the task, and determine if it was time to abandon my naive first choice for a better choice later down the line.

In the next part, I’ll look into what goes into the core mapping platform, and which other components were chosen for the build vs buy machine.