It's Not How Well the Dog Dances
a blog by hewbrocca
I have become a regular reader of Seth Godin’s blog, partly just because he actually posts something every day and partly because some of those things are truly awesome.
Yesterday’s entry falls in the latter category. You can read it there but the gist is:
Points for successful prioritization. Points for efficiency and productivity. Points for doing work that matters.
No points for busy.
I need to do a better job of remembering this…
I spent the weekend at a Hackathon. I mean fortunately I didn’t spend the entire weekend there, unlike the participants who actually in fact did spend the weekend trying to produce some decent code, taking catnaps on the floor in between work sessions. No, I am much too old to do any productive work this way, but I did spend a lot of time there talking to some very smart young people who were trying to solve interesting problems and competing to win prizes. My employer Red Hat was one of the sponsors of the event, called Tech Together, and from our standpoint it was a great success — we have signed up I believe five interns for this summer, possibly more, and we learned a lot about how to make the event more successful for us in the future.
The interesting thing about this particular Hackathon is that it was restricted to people who prefer to be referred to as “she/her.” There are lots of good reasons to have a Hackathon so devised, an important one being that young people who prefer the same pronoun I do (“he/him”) are so goddamned obnoxious and horrible in large groups. (I say “are” without speculation as to the reason why, although being a staunch relativist I think it’s mostly just the way us he/hims are socialized.) At any rate, Hackathons restricted to she/hers provide participants a way to work in groups, learn, and compete without some he/him dominating the conversation, keyboard-barging, and otherwise impeding the group dynamic. The teams involved actually do write useful code and solve interesting problems, and my guess is they become more confident in their own abilities and meet a bunch of other interesting people taking a similar path in life.
There was one aspect of Tech Together that struck me as problematic, though, and to be honest I’m not quite sure what to make of it. The point of a Hackathon, for the participants at least, is to compete to win prizes. Some of the prizes are quite substantial — Microsoft gave away like 5 X-Boxes, for example — and worth competing for. Yet I found Tech Together’s overall vibe, from the name on down, was mostly about inclusiveness and “Isn’t this great that we’re all here together you are all so amazing.” It very much did not feel like it was about “Our team is going to kick all your asses and win this thing.” I think we can take it as a given that she/hers are every bit as competitive and even cutthroat as he/hims, so why sublimate the competition part? I suppose it is a natural consequence of trying to make the event a protected space, which is absolutely a good thing… but could we have a protected space, that nonetheless still has a little ass-kicking going on inside it?
I think what I mean to say here is that finding the right approach to rectify an imbalance — like the ridculous “she/her” deficit in tech — is subtle. You have to provide a space where people can achieve, without feeling like their achievement was only made possible by having the space. To that end, I’d like to see a little more explicit acknowledgement or even encouragement of the competitive aspect of Tech Together. Let the winners come away feeling like they have triumphed; it will make the losers leave determined to return the next year and crush everything in their path.
Kim and I were lucky enough to be having dinner at the bar at Les Zygomates last night when a unique (to me, at least) trio came on called The Gatsby Trio. They had a guitar player with a super chill hollow-body, a trumpet/flugelhorn player, and most interestingly a singer in a kind of 20s getup who was keeping time with brushes on a music stand. Turns out she is called Gabriela Martina and she also does a bunch of other stuff than the 20s shtik.
As a drummer I was both intrigued and mildly annoyed when I realized this singer was really going to keep time with nothing but a pair of brushes on a music stand. On the one hand, cool idea — brushes are idiomatic for 20s swing, after all, and good drummers know well that you can play anything that makes an interesting noise. On the other hand, given she’s mainly singing, can she possibly be doing a great job of keeping time at the same time? Doubtful.
Well I was very pleasantly surprised. She did in fact keep good time once she got warmed up and truth be told she’s better with brushes than I ever was. Plus she was a very accomplished singer who did a credible job scatting (hard to pull off with a straight face, much less well) and also took a couple solos whistling. Whistling, no less. All this while also keeping decent time with brushes on a flat music stand. We thoroughly enjoyed the music, which was not at all confined to 20s swing thank heavens but ranged through a whole bunch of interesting styles.
When Kim and I were revisiting the experience later, we realized that the great thing about this trio was that they had pulled together a very non-standard configuration — no bass, no drummer, no keyboard — and made us forget about it. That in turn put me in mind of one of the things I like most about jazz, which is that there is no wrong way to play it. In most cases, you have a tune — a melody — with a suggested harmonization. You’re not bound to play the tune as written and you’re not bound to the suggested harmonization, or to any particular combination of instruments. You play the tune at the beginning of the number and then you repeat (usually) the form while various people improvise over it, and then you play the tune again. It is just enough structure to let you play and bring your audience along with you, without limiting you very much at all. All you have to do to succeed is assemble good players, listen to one another, and not let your mind wander.
(Truly great players, of course, can discard even this meager framework. Coltrane’s Live In Seattle for example is one of the all time great jazz recordings ever, but it pays almost no homage to conventional form. But do not deceive yourself into thinking there is no form or structure — there absolutely is, you just have to be really familiar with jazz to know where to look for it.)
The question is, is it possible to live and work according to these same principles? Can a team at work function like a jazz combo — assemble good people, provide the absolute minimum structure, listen to one another, don’t let your mind wander, and success will follow? In some cases I think yes, and it is absolutely the best way to work when it is appropriate. But be careful: because there is no wrong way to do it, there is also no formula for how it’s done. So as a manager, if you want to assemble the best group you can, you’re forced to improvise every time.
This must be why I like being a manager…
Hiring, it is said, is the most difficult thing one can do as a manager. Managers who are good at hiring can’t tell you how they do it, and the literature is filled with tales of the horror that befalls managers who do it badly.
I don’t know exactly why it should be so hard to determine whether a candidate will fit well in your team and become a net addition, but it clearly is, and the more senior the person is the harder it gets. There is every possibility that the person who seems so easygoing and willing to learn in an interview, will turn out to be difficult and set in their ways once they get on the job. It is also nigh on impossible to evaluate someone’s skill as an engineer from a test or a transcript or a few code samples. This is especially true in an open-source context, where the definition of “skill” includes everything from “Writes the right code cleanly and quickly” to “Is able to convince unknown and probably cantankerous peers in the community that this patch should get in.” These two abilities, in particular, rarely appear in the same person, at least not when they walk in my door.
(If you are among those who believe that in open source the best ideas always win, I have another conversation to have with you, but for now you can read this post).
You would be right to ask me at this point why I think I’m so smart — what legendary track record do I have in hiring that allows me to pontificate on it? It’s a worthwhile question. I feel like I’ve built a few solid teams over the years, and I’ve taken them through some godawful wrenching changes — “We’re discontinuing your product because we acquired a startup with a competing product, so you all have to learn a new language and work on something completely different in a new community” — multiple times with a very low attrition rate. But I can’t claim any special ability to evaluate people in an interview and I doubt my ability to select the right person is any better than anyone else’s. Certainly, I have hired some great people, but… well, this is public, so I shouldn’t be any more specific than to say I have made a few mistakes too.
I do have one secret, however, and it is summed up in some very bad advice from a song from the ’50s musical Guys And Dolls: “Marry The Man Today.” The song holds that women are better off marrying an imperfect man and then reforming him into what they want him to be, than holding out for the perfect mate to come along. My mother, who had some experience with this theory (sorry Dad), told me early on “This is very very bad advice! Do not follow it!” She is right, of course — in the context of a marriage, it’s a terrible idea, a recipe for unhappiness or worse. In the context of a well functioning team, though, with a strong and healthy culture, it’s not necessarily wrong, especially if you are in the position we all find ourselves in now of having to recruit inexperienced folks straight out of college.
What does this mean in practice? It means I tend to interview people, especially young people, on non-technical criteria. Do they speak clearly and thoughtfully? Do they appear to be self aware, perhaps with a refreshing trace of irony? Did they take the time to find out who I am, what I do, what Red Hat does? If I’m lucky enough to find someone who meets these criteria, I will usually hire them, whether they have any experience in the area I’m hiring for or not. I can teach them to code, and I have a whole team with a whole culture that will teach them how to work the way we work, but I probably can’t teach them to give a crap.
So, instead of spending your time interviewing hundreds of candidates, spend your time building an organization with a culture that will make them into what you need. Hire the developer today, I say, and train them subsequently.
It is, of course, very difficult to know anything for certain. Authorities as widely separated as Werner Heisenberg and Laurence Sterne have written at length about the impossibility of measuring everything (in the first case) and the impossibility of fully understanding anything (in the second). This poses problems for engineers who are in the business of making decisions about precisely what to do based on a set of supposed “facts.” It poses even more problems for managers like me, whose faculties are so withered that telling fact from fiction is usually an exercise in blindfolded dart-throwing.
A good manager, faced with her inability to distinguish engineering fact from engineering myth, builds a network of people she can trust to do that for her. Naturally this presents all kinds of problems with reinforced bias, institutional inertia, and so on, but it’s honestly the only recourse for someone trying to manage a technical team without having the time to get down into the details and see for herself. In my case, I rely on my own sense of the self-awareness and even self-assuredness of the engineers I talk to to try to get a picture of the truth. The most reliable information generally comes from the engineers who are more self-aware than self-assured; put another way, never trust anyone who is absolutely certain they are correct.
You might ask, why, as a manager, do I even need to understand the truth at all? Can’t I just delegate that to some senior people, let them direct everything, and sit back and collect my paycheck? To some extent the answer is yes, I can and I should. Unfortunately, as someone trying to prioritize which research projects Red Hat focuses on, I do actually need to know enough about the facts and what’s coming to make a reasonable judgement. This is especially true because I am, more or less out of necessity, the only one with the full picture of everything we’re doing. I think what I’m saying is that it is impossible to understand the big picture well, while also being aware of all the details. It’s not just a matter of mental capacity. Creating a big picture requires eliding some details, requires approximating things, and that activity is incompatible with knowing for certain all the details of the case.
Here’s a real example of what I’m talking about. A very senior Red Hat engineer, whom I trust implicitly partly out of awe and partly because I know a whole bunch of other people who also trust him, has taken a strong position that we are approaching the end of the era of the general-purpose CPU. Without getting too technical, if his position is correct, then we are in for some massive upheavals in the business of computing in the next few years. Everything from the design of the hardware we buy to the way we build and distribute software will need to change in significant ways to accommodate a new world of purpose-built processors.
Another very senior Red Hat engineer, whom I also trust because he has forgotten more about actual processor engineering than I will ever know, believes this is bunk. His position is that we are still ten years out from the theoretical limit of processor scaling and that we have been through cycles of people wrongly predicting the end of the general-purpose CPU multiple times. Every one of those times, the processor manufacturers have tricked their way through to the next generation and the major change my first source is anticipating has not appeared.
The facts in this case are particularly important because I am about to schedule a half-day, very public discussion at the Red Hat Summit around the end of the general-purpose CPU, and if it turns out to obviously be bunk between now and then, I’m going to look foolish along with Red Hat.
I’m not sure there is a real solution to this. Sometimes I am going to back the wrong horse, and sometimes I’ll pick the right one and look like a genius. I guess the important thing is not to go too long on any single position unless you’re prepared to lose big, no matter how much you trust the person you’re relying on. The only alternative is to dive into the weeds on every decision and try to understand it all yourself, which is not only exhausting but also ineffective as described above.
I have one other point on this. If you read the above carefully you will notice that all the work I have to do in reaching a decision is social. I have to decide whom I trust, whether they have a hidden agenda (whether they are aware of it or not), their degree of bias and in which direction, and so on. If there is a technical component of this decision at all, it is well outweighed by the social component. I believe this is typical of the decisions managers and even engineers make. So, the next time someone tells you that the right way to come to an agreed solution to a problem is to have a rational debate on the merits, tell them there is no such thing. The best you can do is listen carefully, take a deep breath, and throw for the bullseye.
