This is the beginning of the book on programming I will never write. I'll never have enough free time to do that, so let's mimic Clausewitz's "On War" and publish some random thoughts instead of a coherent narrative.
Problem solvers
First of all, programmers are and also think about themselves as problem solvers.
Everyone knows the annoyance of the client asking for specific implementation rather than just providing the requirements. It's kind of a humiliating experience, degrading the programmer from the problem solver to the code monkey and programmers are, quite naturally, upset about it.
After all they are problem solvers, right?
Let's consider the following story to get some insight into the problem.
It's based on my personal experience in the messaging industry, so it's not totally made up.
The problem: We want fully reliable messaging. One application sends a message and wants it to be delivered to another application no matter what.
The programmer starts with TCP. TCP is fully reliable. As long as the two communication applications are alive and there's a connecion between them the data will pass through.
Well, yes, but what if the target application is not alive at the moment? Well, in such case the sending application can wait till it comes online and send the message then.
Ok, but what if the two applications are never alive at the same time. What if, say, one application runs only during business hours Central European Time and the other one during business hours Pacific Time? In such case we need some kind of allways-available application in the middle that will store the message in the meantime. What we are asking for is, bascally, an email-like system with intermediary server(s) et c.
Fine, but what if the server fails while it holds an undelivered message? We obviously need some kind of clustering strategy. Store the message in muliple servers and reliability will go up exponentially.
But what about split-brain condition, when the cluster is split in two with no communication between the two parts?
And so on.
In the end the programmer has to acknowledge that there's no 100% solution and it's better to compensate the customer for the rare missing messages rather than try to improve reliability of the system any further.
So, for example, if message goes missing,the software vendor will pay the customer $10. We know what amounts of messages are our customers passing, we know the overall failure rate and so we can compute the cost. We are going to act in part like an insurance company. We'll increase the price of the software license a bit to account for the "insurance fee" and we'll use that money to compensate the customers when messages go missing.
Nice, isn't it?
Except that one of the customers is a bank that uses the messages for large money transfers. If $1,000,000 transfer is lost, $10 compensation may be considered a bad joke rather than a viable solution. And if the software vendor is to pay the full $1,000,000 compensation, it's going to get bankrupt pretty soon.
Ok, here's a better solution: Write the transaction down on paper and use FedEx to move the paper around. The nice property of the system is that it continues working even if the whole IT infrastructure is down.
But there's a different problem here: We are a software vendor. If we ask the customer to just use FedEx, we are not going to sell any software!
And the programmer (being a problem solver rather then code monkey) asnwers: Do we care? We don't have to be a sofware vendor forever? What about making incursion into the shipping industry?
And that's a pretty serious change for the company as a whole. A very different set of questions is not to be contemplated by the programmer: Do I have enough political influence in the company to steer it from being software vendor to being a shipping company?
Maybe it's not viable? But maybe it would be viable to create a shipping spin-off company? Or, if the business model looks sound enough, maybe I should quit and start a shipping comany myself?
And so on.
So that's a description of how an actual problem solver would progress.
And now, let's be frank: Would you do the same? Probably not. And how many programmers that you know would think like this? Chances are that none. If you are lucky, you may know couple of people who think like this.
Let me give you my experience: I've actually made suggestion that ensuring 100% reliability is actually an administrative/processual task rather than software task couple of times. The reaction was quite telling. I was not like anybody affectionately agreed with me. Nor it was like prople were confused, offended, or passinately challenged my point. Rather it was like if nobody have noticed. Absolute silence.
If it happened once only, it would have been just a weird anecdote. However, I've encountered this phenomenon many times. Once you break out of the technological box, it's almost as if people suddenly stop being able to pay attention to what you are saying.
At this point I am almost sorry I am not a psychologist so that I can run some tests to find out what's going on. However, my layman's feeling is that while programmers may be problem solvers, thay are trained to be domain-specific problem solvers (information technology) and stepping out of that box is very hard for them.
The process of the training is also interesting. So, at school, the assignments have to do only with programming, it's not relevant to the problem at hand. Later on, in work, programmers routinely encounter problems that have no true technical solutions, but can be solved only on economic or political level. And yet they are asked to solve them using technology. While you are a junior programmer you may not even notice. Then you step out of the box few times and learn that it's almost impossible to achieve anything at that level.
The understanding pushes you back to the coding. However, the problem cannot be solved an technological level…
And here one of the interesting observation: Programmers faced with a problem and given no levers to solve it turn back to coding, but they tend to invent new (irrelevant) problems to solve to easy the pain created by the cognitive dissonance. What ensues is a project where there are different interesting problems being solved, but nobody really works on the original problem.
I wanted to discuss one concrete scenario to give you feeling what is this book going to be about…
Simple test: When solving a technical problem do you consider your work contract to be part of the problem space? You may not, but if so, you are not doing proper problem solving job, because it is.
APIs
APIs are a good place to start.
First, it turns out people have different ideas about what API means. In the context of this book API will mean public interface of a software product. The product may be composed of many components, each having its own interface. However, as long as the interface is not exposed to the users of the product, we won't consider it to be an API.
Of course, the definition depends on how we define the product. Just split the codebase into many small "products" and you can treat any interface as an API, right? Not really. Here are some points that distinguish API from an internal interface:
- Is there a detailed documentation for the interface?
- Does the "product" have a business name? ("Tequilla Enterprise Server" vs. "CachedConfigurationTransmutator")
- Is the inerface used in more than two distinct places?
- Et c.
Now, most people tend to think about APIs as technical boundaries and API design as a technical task.
In reality, however, the task is political in nature. Let me give you a metaphor:
Some may say that APIs are actually "contracts" between developers. Fair enough, but the metaphor of border between countries reflects the reality much better.
Borders can match natural boundaries, such as mountain ranges or rivers. Slovakia, the country I come from is bounded by High Tatras mountain range to the north and river Danube to the south. However, beyond the natural border of Carpathians to the west, there's a flat sandy lump of land fittingly called "Záhorie" ("The land beyond the mountains") which is still part of Slovakia. To the east, very much in the opposite way, there's a region aptly called "Zakarpattia" ("The lang beyond Carpathians") which lays on the Slovak side of the mountains and was part of Czechoslovakia once yet it was annexed by Soviet Union after WWII. And that's not to speak of the south boundary — the Danube — which is perfectly natural, however, it doesn't match the ethnical boundaries. So called "Great Rye Island" that become part of Czechoslovakia by the Trianon treaty was and very much still is inhabited by ethnic hungarians.
Same way, APIs may match technical boundaries, but if they do, it's a fortunate outcome of the past political struggles raher than a sign of technical superiority.
The example above shows how there's a vague desire to craft the political boundaries to match geographical ones, but in reality the two rarely match. Same applies to APIs: There's a vague desire to craft APIs so that they make sense from engineering point of view, however, in reality, the political concerns take precedence.
On Bears
Alexander Luria: "In the north, where there is always snow, the bears are white. Novaya Zemlya is in the far north and there is always snow there. Can you gather what kind of bears there are in Novaya Zemlya?"
Uzbek man: "If a man was sixty of eighty and had seen a white bear and had told about it, he could be believed, but I've never seen one and hence I can't say."
—-
And as logic-munchers and corner-case-sensitive folk: Do you see the logical failure in the answer Luria have expected ("The bears in Novaya Zemlya are white.")? Think about it. If it looks OK to you, scroll down to the end of this section.
—-
And here's the logical failure: If there were no bears at all in Novaya Zemlya, there would be no white bears either.
Figuring out why such a simple logical rule as modus ponens doesn't work in this case is left as an exercise for the reader.
—-
And, just for curiosity's sake I've done a quick web search about fauna of Novaya Zemlya. It turns out that along with other species there lives Ursus Maritimus (white polar bear) as well as Ursus Arctos (brown bear). That much for the reliability of logic.
RoI
RoI estimation is not a well-defined skill you can learn. It's mostly a function of experience. If you've dealt with a similar problem in the past, you estimate is going to be more relevant.
Nontheless, RoI estimation is the single technique with the most impact on the health and cost of a project.
So, the goal here is not to teach you how to do RoI estimation (get a lot of experience, hire a person with a lot of experience), but emphasise the significance of RoI estimation as such: Don't do anything without thinking of its RoI.
Domain blindness
If all you know is programming everyhing looks like a nail.
Hands-on
If you don't do the coding, your architecture design is much worse. But it's a two-edged sword: If you do coding you are tempted to prefer developer's perspective to user's perspective.