lbrandy.com

1. The reason we invented civilization is so you can do one thing really well, and give money to other people to do everything else for you. I discovered this during my move. Movers charge a lot of money. They are worth it.
2. There is no such thing as one-coat paint. Don’t believe the hype.
3. People you know in real life will mock you if you write a blog. As they should.
4. Don’t write something you don’t believe, just to be provocative. People who don’t know you that well might take it seriously.
5. Never trust an Austrian talking about power.
6. The more people who go to a meeting, the more efficient it is. I have no idea how. I can’t explain this.
7. Hard consonants are funny. If you need an arbitrary noun for a joke, choose one with good hard consonants. Tuesday is the best day. Seventeen is a good number. James K Polk. Buicks. This also explains why a well-placed ‘fuck’ makes jokes better.
8. The egg came first. I’ve seen through the bullshit.
9. I have never been more wrong about anything in my life than twitter. Sorry twitter. Good luck Iran.
10. I am still correct about the vast majority of twitterers.

If you spend all day doing it, when you get home, you really don’t want to do anything else.

Like, for example, write a blog post.

I know that joke was lame but look on the bright side, at least you didn’t get a lame analogy where I seriously tried to connect moving and programming.

So, we’ve bought a new house. It was a good time. We got an interest rate below 5%, a great price, and Pittsburgh has been rated the best place to buy instead of rent by CNN/Money. We are paying about as much in mortgage as we’ve been paying in rent. So off we go, painting, and cleaning, and packing, and moving. And that means anything I write will be full of exhaustion and anger. Like normal, you might say.  At any rate, I’ll be back in two weeks, at the latest.

Also, go Pens.

This is a post I’ve wanted to write for awhile. This wasn’t the post I had hoped it to be. The original working title was “Writing a library interface is hard. Really hard.” The way the post was supposed to go was with me explaining a whole bunch of general problems with writing good programmatic interfaces for libraries and then giving some helpful advice.

Over time I realized that each library has such unique problems that general advice is almost entirely worthless. It involves balancing so many factor and in some cases the answers are obvious (a jpeg library really only has a handful of functions) or prior-art exists (GUI toolkits) that will help you. Other times, you are on your own and it becomes quite a struggle. So instead of my general advice that will save all of mankind, let me explain some of our struggles, and I’ll let you figure out the lessons learned for your situation.

Oh, and as if the engineering challenges weren’t enough, you needn’t forget to add in these supplementary exercises that are among the most tedious in all of programming.

Crushed Souls

Documentation. It is not fun. We both know it. And yet, you absolutely must have it and if you’ve done it right, it’ll be incredibly useful. The sad truth is that most people won’t read it.

Example Programs. Virtually everyone who uses your library will skip straight by the docs and straight to the example programs. These cannot be bad. Try to make at least one short, simple, and hello-worldish. It should be easy to find.

Error reporting. Figure out a way to handle errors in a useful manner and report that information back to the developer. This is probably the second most soul crushing endeavor in all of programming.

Meetings. Did someone mentioning soul crushing? The only way to hammer out a good API is to get a bunch of people in a room and look at interfaces, functionality, paradigms, and conventions, and decide what is good and bad. As a word of warning, these meetings will not be short.

Iterate. You simply cannot get it right the first time. This shouldn’t come as a surprise. Iterate early, and iterate often. Work out a prototype H file first and work from there. Get that into a room with other devs and walk through the use cases. That H file is your spec.

Usability vs Flexibility

This is where it gets difficult. Our SDK typically has been geared towards ease of use, not power. This meant that certain sophisticated things we do with our internal codebase aren’t possible to replicate using our SDK. For example, we have a live camera-fed demo that devoted some threads to video processing (face detection & tracking) and real-time display, and some other threads for psuedo-background recognition that updates the onscreen results as they come in. The complicated mix of threads and asynchronous nature of the recognition made this a non-trivial program to write. While possible, our old SDK was fundamentally ill-equipped to replicate it.

If you want to build an easy-to-use library, it’s fairly straightforward. You pick a few use cases, do those right, and everyone else be damned. We decided, recently, to expose more functionality to make it capable of all the complicated and nuanced things you might want to do. This resulted in marathon meetings, lots of teeth gnashing, and at least one heated argument.

And you can take it too far. More than once, while at the whiteboard,  we’d finally convinced ourselves that this particular set of functionality lets everyone do everything. At that point it became clear, however, that it’s all so complicated that you’ll need five times the documentation and no particular user has any hope of actually getting it right on their own. So now you’ve traded feature request emails for “how do I get this to work?” emails. We went back to the literal drawing board.

High-level vs Low-level

Our original face recognition SDK basically had one function. Given two faces, you got a score. That was it — pretty much the epitome of low level. From there, everything else follows. It turns out, of course, that everything else isn’t so simple. Some people want to compare a probe face to a gallery of faces and find the best match. Other people want to take all the faces in some unorganized collections and organize it. In any use case, you’d need to build up a fair bit of infrastructure. You’d have to handle saving, caching, and re-using previous results. If you want to use a huge dataset, its up to you to figure out how to partition the data, and how best to thread it. There was quite a bit for you to get right. And more often than not, we found people getting it very, very wrong.

Eventually we realized that in any one use case, a developer using our SDK would have to create so many things and get so many of them right, that it simply wouldn’t be feasible for him to get the system he wanted. At least not as well as we could.  We’d thought long and hard about some of these problems and could build solutions that were simply better.

Adding all of these functions while maintaining full flexibility resulted in a dozen functions (instead of our original one). Every function you add adds to the pain. It create complexity, documentation, and work. But we trimmed down to some minimal set that best balanced all these factors and pushed forward.

Elegance

When you find a non-trivial library that works spectacularly out of the box, you’ve found a piece of programming gold. It’s a library that handles the easy and common cases trivially. It’s a library whose abstractions snap together so well that you can almost do what you want without looking at the documentation. It’s documentation that is easy to use and results in quick and accurate answers. And when one day you realize you need something a bit more sophisticated, the feature is there waiting for you. You shouldn’t take it for granted. It’s so incredibly hard to get right.

So in that spirit, I’d like to nominate Qt as the most elegant and well-documented library I’ve ever used. Our SDK is not there, yet. All we need is about 100 iterations of progressive refinement.

…but the notion that your code will be reviewed is most certainly not. I argue that the vast majority of the benefit of code reviews comes from the belief that many other programmers will look at this code and judge you for it. There is no doubt that this belief will produce better code. The ensuing review itself, though, adds quite a bit less value to the situation, especially given its cost. This distinction is important.

There are several types of code reviews ranging from desk checks (which taken to its natural extreme becomes pair programming) to full conference room thesis-defense-style interrogations. I am certain that these types of activities catch bugs, improve interoperability, and help programmers stay on the same page… but at what cost? Code reviews are terrifyingly expensive. Compulsory and frequent meetings (even impromptu ones at a desk) destroy the flow of a work day. It crushes the ability to get things done. It will slow down the best programmers for the possibility of improving the worst. I’m not a big fan.

Managers managing

A friend of mine works for a software development house who believes in code reviews. Their managers became convinced (undoubtedly at some management training seminar) that code reviews produce better code. Given that belief, it necessarily follows that more code reviews produce even better code. Managers constantly searching for ways to quantify and measure their employees developed a fun optimization: let’s try to increase the number of eyeballs that looked at every piece of code. This rapidly became a classic case of non-technical management trying to invent ways to improve a process they do not understand. They found something they could measure that they believed correlated with code quality. Optimize it!

At first, there was formal code reviews. My suspicion says this produced a sizable increase in the quality of the code. This initial success and a touch of confirmation bias did the rest. Once a week meetings for code reviews probably isn’t too disruptive. Next, they instituted desk checks before all commits. I suspect this added virtually no benefit. Now, though, throughout the day, someone might drop in in the middle of something you were working on and request your presence at their desk for a commit. If you want to finish what you are working on, they have to wait, if you go, you lose your train of thought. It’s a lose-lose. And then, best of all, they decided that for each commit, each developer must send out an email containing the changes that were made, the files they are made to, and the diffs of the changes. All technical readers will note that this should be trivially automatic. It wasn’t.

I’m certain that the programmers quickly automated the process anyway, but think, for a moment, about what these managers must be thinking. For every commit, you must have someone else come check your code, and then manually compose an email containing a whole bunch of information. The sheer amount of time used up in code reviews, reading emails, sending emails, dealing with distraction, and “re-immersion” for programmers must have been staggering. I’d doubt that anyone could get more than an hour honest work in per day. If they followed the concept in earnest, they would spend most of the time doing meta-work (work that lets you do actual work).

The real goal

If you are a manager and do not understand software, let me give you a piece of advice: interruptions are the devil. Instead of min/maxing whatever other variable you had in mind, try minimizing disruptions. If your developers know that their code will be extensively used, maintained, and extended, you already have most of the benefits of code review. The difference is it will be done organically and without the need for constant interruptions.

Our story has a happy ending. The programmers at said development shop, of course, adapted quickly. The emails were automated and the desk checks were ignored. The formal review was impetus enough for people to write better code. The only negative consequence turned out to be the revelation from manager to employee of the former’s inadequacy.

Vacation means no writing for me this weekend. But just for content’s sake, here’s some movie reviews.

Angels & Demon: A conspiracy thriller centered on religion vs. science. It’s a pretty good conspiracy thriller. The science/religion part is really weak. The science part, especially, is eye-rolling.

Star Trek: Awesome. Even my wife liked it. Highly entertaining. Characters are awesome. I hope they make more.