It is, of course, very difficult to know anything for certain. Authorities as widely separated as Werner Heisenberg and Laurence Sterne have written at length about the impossibility of measuring everything (in the first case) and the impossibility of fully understanding anything (in the second). This poses problems for engineers who are in the business of making decisions about precisely what to do based on a set of supposed “facts.” It poses even more problems for managers like me, whose faculties are so withered that telling fact from fiction is usually an exercise in blindfolded dart-throwing.

A good manager, faced with her inability to distinguish engineering fact from engineering myth, builds a network of people she can trust to do that for her. Naturally this presents all kinds of problems with reinforced bias, institutional inertia, and so on, but it’s honestly the only recourse for someone trying to manage a technical team without having the time to get down into the details and see for herself. In my case, I rely on my own sense of the self-awareness and even self-assuredness of the engineers I talk to to try to get a picture of the truth. The most reliable information generally comes from the engineers who are more self-aware than self-assured; put another way, never trust anyone who is absolutely certain they are correct.

You might ask, why, as a manager, do I even need to understand the truth at all? Can’t I just delegate that to some senior people, let them direct everything, and sit back and collect my paycheck? To some extent the answer is yes, I can and I should. Unfortunately, as someone trying to prioritize which research projects Red Hat focuses on, I do actually need to know enough about the facts and what’s coming to make a reasonable judgement. This is especially true because I am, more or less out of necessity, the only one with the full picture of everything we’re doing. I think what I’m saying is that it is impossible to understand the big picture well, while also being aware of all the details. It’s not just a matter of mental capacity. Creating a big picture requires eliding some details, requires approximating things, and that activity is incompatible with knowing for certain all the details of the case.

Here’s a real example of what I’m talking about. A very senior Red Hat engineer, whom I trust implicitly partly out of awe and partly because I know a whole bunch of other people who also trust him, has taken a strong position that we are approaching the end of the era of the general-purpose CPU. Without getting too technical, if his position is correct, then we are in for some massive upheavals in the business of computing in the next few years. Everything from the design of the hardware we buy to the way we build and distribute software will need to change in significant ways to accommodate a new world of purpose-built processors.

Another very senior Red Hat engineer, whom I also trust because he has forgotten more about actual processor engineering than I will ever know, believes this is bunk. His position is that we are still ten years out from the theoretical limit of processor scaling and that we have been through cycles of people wrongly predicting the end of the general-purpose CPU multiple times. Every one of those times, the processor manufacturers have tricked their way through to the next generation and the major change my first source is anticipating has not appeared.

The facts in this case are particularly important because I am about to schedule a half-day, very public discussion at the Red Hat Summit around the end of the general-purpose CPU, and if it turns out to obviously be bunk between now and then, I’m going to look foolish along with Red Hat.

I’m not sure there is a real solution to this. Sometimes I am going to back the wrong horse, and sometimes I’ll pick the right one and look like a genius. I guess the important thing is not to go too long on any single position unless you’re prepared to lose big, no matter how much you trust the person you’re relying on. The only alternative is to dive into the weeds on every decision and try to understand it all yourself, which is not only exhausting but also ineffective as described above.

I have one other point on this. If you read the above carefully you will notice that all the work I have to do in reaching a decision is social. I have to decide whom I trust, whether they have a hidden agenda (whether they are aware of it or not), their degree of bias and in which direction, and so on. If there is a technical component of this decision at all, it is well outweighed by the social component. I believe this is typical of the decisions managers and even engineers make. So, the next time someone tells you that the right way to come to an agreed solution to a problem is to have a rational debate on the merits, tell them there is no such thing. The best you can do is listen carefully, take a deep breath, and throw for the bullseye.