By	Casey Muratori

Building upon last week’s introduction to compression-oriented programming, I’d like to introduce an important concept that is best described by reference to a fairytale I’m sure you will recognize from your childhood. It’s the story of a young girl who comes upon a house in the woods inhabited by three bears, all of whom happen to be out for the day. I am referring, of course, to Codielocks and the Three Warez. Specifically, I’d like to call your attention to the famous scene where Codielocks discovers the server rack at the center of the three bears’ house, and she roots a blade using herPwnie Express Pwn Plug R2. She then looks at the three headers, usr/include/mamabear.h, usr/include/papabear.h, and usr/include/babybear.h. She tries writing a program using /usr/include/mamabear.h, but finds that her code comes out bloated and contorted. “This is too verbose and ill-conceived,” says Codielocks. Next she tries writing the same program using /usr/include/papabear.h. At first it seems easy to use, and she gets her program working very quickly, but then when she goes to polish it and optimize its performance, she finds that there is no way to do most of the things she needs to do. “This is too restrictive,” said Codielocks. Finally, Codielocks tries using /usr/include/babybear.h. “OMG!” she exclaims. “This is so easy to use, and yet it never gets in my way when I need to do something tricky!” Why was Baby Bear able to make his code so that it was “just right” for Codielocks, as the story goes? Well, it’s because unlike Mama Bear and Papa Bear, Baby Bear practiced good compression-oriented programming, and understood the importance of continuous granularity.

To illustrate the concept of granularity, I’d like to continue with the example from last week where I was compressing the code for the Witness Movement Panel. If you remember last week’s starting point, you will recall that there was some code that I never got around to compressing in that article:

This code can’t actually call the push_button() that I made for the simpler code to use. The reason is because push_button() didn’t pass the optional parameters to draw_big_text_button() that specify the colors to use to draw the button, and this turns out to be important for the buttons in this snippet because they are “checkbox” style buttons: they turn on and off, rather than just doing an operation when clicked like the ones that used push_button(). So I needed a second version of push_button(), one that makes a pushable button that can be highlighted:

Now, ordinarily, I might consider trying to compress the two push_button() calls further: what I would do is have the non-highlight one call the highlight one with the highlight parameter being false. However, in this particular case, that wasn’t a good idea, because the underlying UI code that I was wrapping didn’t actually draw regular buttons in the same colors as it used for highlightable buttons. So, in this case, they stayed separate. Also, as an aside, it is doubtful that I should have called them both push_button(). Perhaps push_button() and highlight_button() might have been better. But, I prefer to stay true to what actually happened, rather than what should have happened, and what I actually did on The Witness was make two functions both called push_button(). OK, so with a second push_button(), I could now compress the original snippet like this:

But this didn’t exactly reproduce the original code. The original code put all the buttons on one row, and this puts each button on a separate row. So I needed to make the layout code a little more flexible in order to finish the compression properly. If I was going to allow multiple things to pack themselves into a single row, I needed a way of knowing how the space of the row would be divided up. The original code just took the number of buttons it knew it was going to use, and divided the width of the row by that so it knew in advance where each button was by just doing the math. Replicating this behavior was trivial if I forced the user of the code to pass the number of buttons. With the number of buttons, I could make row() more intelligent so that it used both a width and a column_width, which I added to the Panel_Layout as well as a left_x that stored the starting x so that at_x could be reset on each row:

And then the push_button() routines could be modified to advance to the next column after drawing a button:

Now the original snippet can be modified to exactly reproduce the old behavior:

If you remember the compressions I did last week, you’ll be tempted to look at the integer 4 in that last snippet and try to eliminate it the same way I eliminated the up-front specification of the total number of rows in the original code. While I applaud that sentiment, I didn’t do that. The reason I didn’t try to eliminate the pre-specification of the number of columns is because the control flow is actually dependent on that number, which was not true for the number of rows. The number of rows was used solely in a computation that didn’t affect any of the subsequent layout operations, so it was trivial to move it to the end where it could be computed automatically. The number of columns, on the other hand, determines the width of the buttons, and until the width of the buttons is known, hit testing can’t be performed. If hit testing can’t be performed, then the push_button() call couldn’t return “true” in the case when the button was hit, which is necessary for the control flow to work the way it did originally. So, in order to automatically calculate the number of columns from the number of push_button() calls, I would have had to introduce something relatively complicated. There are many options for this sort of thing  —  one would be a deferred boolean, something that you get back from push_button() and inspect later. This is error prone, and prevents the clean-looking if(push_button()) style, which was a very nice way to write the code. Another option would be to move to a unique name or partially retained system, which would introduce a frame of latency to the processing so that hit testing actually reported the results from the previous frame. Etc., etc. Since specifying the number of columns is very simple, and is impossible to mess up in any catastrophic way (the worst possible outcome is that the buttons are sized wrong until someone notices), I made the judgment call that the more complex options weren’t worth it. These are the kind of tradeoffs that are extremely annoying, and which exist only because the people who write the C++ spec have no idea what production programming is or how to do it properly. But that’s a separate article. For now, suffice it to say that this is a tradeoff that has very little to do with the nature of programming, and a lot more to do with the fact that C++ hasn’t had a qualified language designer on it since… well, since Stroustrup stuck “with Classes” on the end of C and starting crapping it up with stuff that mostly made things worse. Sadly, this kind of tradeoff is still involved in programming of all kinds, whether it’s compression-oriented programming or object-oriented programming or anything else. There may be a very clear way in which you could have expressed what you wanted to express, but the language simply doesn’t supply the primitives to do it. When faced with a problem like this, there is often a strong temptation among programmers (myself included) to try to circumvent the shortcomings of the language and achieve the desired result through a great deal of extra code. Unfortunately, this is almost always a lose, and so while I can say from the bottom of my heart that I empathize greatly with the folks out there who would try to make seventy layers of templates to solve a problem like pre-specifying the columns, because believe me I’ve done stuff like that in that past, I’m afraid I have to say from (quite a bit of) experience that it almost never ends up being a win in terms of total time spent, which is really the only thing that matters. So although this point has little to do with compression-oriented programming per se, and is more about programming in general, I thought it was an important thing to point out. It’s OK to accept a solution that’s a little ugly if the alternatives are uglier. The important thing to remember is that you must always focus on the end result. It’s easy to fall into the habit of thinking of abstract things like whether code is “clean” or whether it is “elegant” or god forbid “exception safe” or whether it “obeys RAII” and all these other things you may have heard about or believe in. But in the end you have to remember that all of them are just concepts and guidelines that someone made up on the way to achieving a lower total cost for code (as I described in the first article). So you must always make sure that the primary thing you’re evaluating in your head, when you make decisions in places where there are tradeoffs, is the total cost and only the total cost. Because if the total cost is going to be worse, it really doesn’t matter what else you might say about the code philosophically. You still made the wrong decision.

Getting back to the code at hand, I liked the look of the snippet at this point:

But I saw a lot of code like it in the Witness UI system, where there were booleans that were getting toggled, and you constantly saw this pattern of using the boolean for the highlight and also using the result of the push_button() to toggle that same boolean. So really, the second push_button() call that I made for doing these toggle buttons could be changed to make the code much more compact:

Now the code snippet could be more compressed:

But that’s not what I actually did. I did something slightly different. And this is where the concept of granularity comes into play. See, something subtle but important would have happened had I changed the highlightable push_button() to work this way: I would have created a hole in the API. With one push_button() call not having any highlight capability, and another push_button() call that supports highlighting but requires a modifiable boolean, I’d have no way to conveniently use the API for something that wanted to be highlighted but that wasn’t stored as a toggleable boolean. For example, suppose there had been some code that looked like this before I modified the second push_button() (and in fact, there was code that looked like this in other places in The Witness editor):

With a modified highlightable push_button(), it would have had to change to look like this:

Modifying push_button() would have made the boolean toggle case better, but it would have made this case worse. That might be a good API tradeoff to make if I’d had to make it (I’d just pick whichever was used more and make that way be the convenient way), but the key here is that I didn’t have to make it. Instead, the right solution was to leave both push_button() calls alone, and instead introduce a third call that does the modifiable boolean version as a layer on top of the highlightable push_button():

Now, the code snippet could be written compactly, just as before:

But since I didn’t touch the other calls, code that wanted to use the highlightable push_button() directly to do things that are more complicated than a simple modifiable boolean could still be compact, too. This is an extremely simple example of how important it is to maintain continuous granularity when compressing code. Before I started making Jon’s UI system support more compression, it was extremely granular. Everything was done by hand, so the user of the library had full control, but had to do a lot of work. When I made Panel_Layout and started adding things to it, I was making a less granular way to use the UI system. The code that uses Panel_Layout makes less calls to it, and does less work, preferring instead to let Panel_Layout do some things automatically. But obviously anyone who needed full control could always just not use Panel_Layout and call the UI system directly. This created two levels of granularity, which was nice and flexible. Adding bool_button(), instead of modifying push_button() was crucial in that it created a third level of granularity. If you happen to want a toggle boolean, you can call bool_button(). If you want a highlightable button, but don’t want a toggle boolean, you can call push_button(). And if you want something else altogether, you can still call the UI system directly. If instead I had changed push_button(), I would have created a hole in the second level of granularity, a discontinuity. If you wanted a toggle boolean, you’d still be OK, because you call the modified push_button(). But if you want a highlightable button that doesn’t work with a boolean at all, you either have to go all the way down to the lowest level of granularity and call the UI system directly, or you have to introduce a temporary boolean everywhere in a very inconvenient way so that you can try to use the higher-level call. Hopefully you can see how this extends to more complex situations. As subsystems and APIs become more complex, continuous granularity can become essential to avoiding very bad situations. In this simple example, the discontinuous case isn’t horrible. It’s not great, but it’s not that bad. In cases where there were more far-reaching effects of a granularity change, things can be much worse. For example, suppose that I had been careless and ended up with a system where the granularity switch had cascading effects. Suppose that I’d made Panel_Layout’s data entirely private and didn’t support any way of querying anything in it. In that case, you would be forced to either use or not use Panel_Layout for everything in a panel. So if you’d done all your buttons with push_button() and the modified toggle-boolean push_button(), then got to a button that couldn’t use one of those, you’d literally be forced to rewrite the entire panel using the low-level UI calls! As horrible as that sounds, it is actually a very common thing to have happen in APIs these days, many of which are “designed” by people who do not actually know anything about API design. It is always important to avoid granularity discontinuities, but fortunately, it is really easy to do if you just remember a simple rule of thumb: never supply a higher-level function that can’t be trivially replaced by a few lower-level functions that do the same thing. This is often a hard rule of thumb to follow if you do object-oriented programming, and is yet another reason that object-oriented programming is a bad idea. It’s something you have to explicitly design into the system of objects a priori, and it is often very difficult to ensure that you have done it correctly, since there are lots of details to get right and lots of circumstances you have to correctly predict (and there’s another problem here that deals with opaqueness, but I’m going to cover that in a later article). If instead you are following a compression-oriented approach to programming, the rule of thumb is trivial to follow because it is how you made the code in the first place! It becomes an implicit result of the programming process, rather than an explicit issue you have to deal with. Since compression-oriented programming means building the code by progressively bundling pieces of code into functions that operate at higher and higher levels, so as long as you just don’t delete the smaller pieces as you build bigger ones, you automatically end up avoiding granularity discontinuities.

That brings me to the end of this week’s Witness Wednesday. Although I added a bunch more stuff to Panel_Layout (collapsible panels, tooltips, default controls for various things, scrolling, etc.), I can’t think of anything particularly special about any of them that warrants detailed coverage. So next week I will be moving on to “the main event” of this series, which is the construction of the new “Lister Panel”. But before I sign off, I wanted to briefly address two types of comments that I received about last week’s Witness Wednesday. First, there seemed to be some confusion about what I meant by “object-oriented programming”. Just to clarify, I don’t consider object-oriented programming to have anything to do with whether or not there may or may not be things that look like objects existing in the code base. To me, that would be a very silly way to use that word, because that would imply that, for example, you were doing “functional programming” simply because your program happens to have some functions in it that don’t have side effects. Rather, when I say “object-oriented programming”, I mean literally that: you have oriented your programming around the objects. The objects are the important thing to you, and you are thinking in terms of the objects. It is the practice of thinking that objects are important (or that they’re even worth thinking about at all) that is wrong, and is best avoided. It is not objects themselves that are bad, because objects are really nothing more than some functions that happen to be strongly associated with some data. If you happen to end up with that in your programs, there’s nothing wrong with that. Sometimes some functions are very closely related to the data with which they operate, and that’s fine. And to go one further, just to underscore that I’m talking about a way of programming rather than the result of that programming, if for some reason you really absolutely positively need to have a bunch of objects in your code in order for you to sleep soundly at night, as much as I might disagree with that opinion, I would still advocate that you build those objects using a compression-oriented approach! You would still be much better served to avoid pre-designing your objects, or your hierarchies, or whatever else it is that you want, and instead write the low-level code first and progressively compress it into a suitable set of objects. Second, after last week’s article, multiple people strangely suggested that, for some reason, compression-oriented programming would only work with small numbers of people (perhaps one or two), but is somehow not possible to do or not effective when working on teams (or large teams), and so object-oriented programming must be used. I don’t really understand this suggestion. I’d love to discuss it at some point, but honestly, I really just don’t even know what the underlying assumptions would be that would cause someone to arrive at this conclusion. I’m also not sure why they think that it isn’t something I’ve used effectively on projects that involved teams of people (which I actually have). So until I get a better idea of why someone would think that the number of people involved has anything to do with the efficacy of compression-oriented programming, I’ll just say this: Writing code for yourself to use is no different than writing code for someone else to use. Unless you are some sort of idiot savant who remembers every piece of code they’ve ever written, it is always in your best interest to write reusable code that presents a nice, usable, hard-to-screw-up interface to the code that uses it, because even if it’s only you who’s using it, you’ll probably have to use it at some point when you’ve forgotten one or more of its internal details. Compression-oriented programming isn’t some kind of shortcut thing you do when it’s just you  —  it’s the right way to build code that presents a clean interface to a complex set of operations, which is basically what any non-trivial piece of code needs to do. I’ve been on projects where I was the only programmer (RAD’s Granny 3D), I’ve been on projects where there were so many programmers that I didn’t even know who they were or how many there were (Intel’s Larrabee project), and I’ve been on projects where there were just enough programmers to make things interesting, like The Witness, where there were five (well, seven, technically, but I don’t think all seven were ever on the project simultaneously). Throughout all of this, I have never once found myself thinking that the style of programming should change based on the number of people. I always adopt a compression-oriented approach, and if I am working on a team, as I get things to a point where I think they will be useful to others, I start putting things in a shared header file for them to use. That’s really all there is to it. But, certainly, if there are a lot of specific issues that people are thinking of in relation to compression-oriented programming and object-oriented programming that vary based on the number of people involved, please write in and let me know what they are. If I get enough specifics I’ll do a whole article talking about them.