Much buzz today on HN about developer hiring with Evan Carmi sharing his Google interview experience, which mostly consisted of language trivia and algorithmic tasks, and a devininterviews follow up arguing that this interviewing style leads to bad hires.
I don’t think there is one right developer interviewing technique. It is worth mastering a variety of interviewing techniques and then adjusting based on the job requirements, team composition, manager style and company culture.
Here is an initial (incomplete) list. I’ll update with new content from comments & conversations.
Programming language questions
Examples: what’s the order of C++ class data member initialization, how is the value of self determined in a Ruby block and what’s the difference between class and function decorators in Python.
Why: I want to know whether you know the tools of your trade.
Good for: quickly checking the box on language proficiency.
Useless for: determining whether you can code well in the language. I know developers who write kick-ass, elegant code using 50% of a modern language’s features. That’s awesome.
On-the-spot coding of algorithmic problems
Examples: write a binary search, write a Sudoku solution checker and write a substring search.
Why: I want to know which developer group you belong to: the folks who can solve algorithmic problems or the ones who could churn dozens of MVC screens but couldn’t build a fuzzy string matcher to save their lives, even with resources online.
Good for: vetting analytic thinking and coding precision under pressure as well as basic verbal communication & organization of thought. Bonus: easy way to see how you handle criticism.
Useless for: understanding how you’ll write practical code and handle criticism on a team. In the real world engineers solve problems not in front of a white board but at their desk, with Google, GitHub, their co-workers, hacker buddies, Twitter friends and the Stack Overflow community. If your development velocity is great, I most likely won’t care whether you do it by memorizing the algorithms textbook or submiting a dozen questions per week on Stack Overflow.
Examples: find the bugs in this listing or I have this project loaded in Eclipse and this bug ticket here — go at it.
Why: just as I want to understand your development workflow, I want to understand your debugging workflow.
Good for: getting a sense of how you wrap your head around other people’s code.
Useless for: being sure that’s how you’ll debug normally.
Take home programming tasks
Example: Here are two empty git repos. By 10am PT tomorrow build and deploy two applications on Heroku. The first exposes a REST API for logging session events with arbitrary JSON-based parameters and saves the events in an SQS queue. The second pulls events from SQS and saves them in a MongoDB database hosted on MongoHQ. You can use whatever gems you want.
Why: I want to get a sense of how you work in the real world. I want to look at the code you write but I care even more about what you consider “shippable”. Here are a few things I’d pay close attention to beyond the obvious code style/quality that you may not even know you’ll be evaluated on:
- How did you deal with fuzzy requirements?
- What was your git workflow: commit frequency/atomicity, batch size, branching strategies. How you use source control alone is a great indicator of how you’ll use source control on a team.
- Did you choose Rails or Sinatra? Why?
- Which AWS gems did you pick? Why? How did you evaluate the quality & fitness of a gem?
- Did you use TDD? I don’t mean did you deliver tests in the end. I’m interested in whether the tests hit the repository no later than the features.
- How extensive is your testing? Do you have integration tests? Acceptance tests? Did you set up dev & staging SQS queues?
- Did you think about securing the REST API? How?
- How do you manage sensitive info? Is your Amazon secret in the git repo or does it get set as an environment variable? What about the MongoHQ credentials? What about sanitizing the logs?
- How paranoid are you about failure? Did you pop() from SQS assuming a MongoDB write will succeed or did you peek() and only delete from SQS when you know the message contents made it into Mongo?
- Are you checking for duplicates in Mongo?
- How did you set up Heroku? Do you have staging apps as well as production apps? Did you set up deploy notification hooks? How about monitoring for the apps?
- Did you think about scaling? Did you write any code for it? Did you do any load testing? When does your current deployment break?
- Did you write any docs? Do you know your audience?
Good for: a litmus test of your ability to build, ship, operate and communicate. Getting a “fingerprint” for how you work.
Useless for: deeply understanding how productive you can be on a team. There is no teamwork component here and it’s hard to separate the various elements that contribute to the final product: your prior knowledge and experience, help you get from a buddy or online resources, luck. Plus, to really evaluate your performance will take a lot of work.
Examples: (actual problem I used to give) here is a simple linked list implementation in C++. Find the bugs.
Why: just as I want to understand your development workflow, I want to understand your debugging workflow. Some of the bugs are subtle. I hope you talk to friends about the problem.
Good for: getting a sense of how you wrap your head around other people’s code and how you work with others to solve problems.
Useless for: determining how much of the outcome was “pure you” vs. “you + others”
Examples: from devin interviews:
- What’s the last project you worked on at your former employer?
- Tell me about some of your favorite projects.
- What projects are you working on in your spare time?
- What online hacker communities do you participate in?
- Tell me about some (programming/technical) issues that you feel passionately about.
Why: I want to get a sense of who you are as a developer and how you might work on a team.
Good for: knowing who to reject quickly and who might be a good cultural fit.
Useless for: deeply understanding who to say yes to. Bottom line: great interviewees can prep for and ace this because it’s too general.
Examples: tell me about a situation where you had to deal with a bug that only appeared under load in production + lots of specific follow-on questions
Why: two reasons: (1) other things being equal, past performance in similar situations is the best predictor of future success and (2) behavioral interviewing gets down into the nitty-gritty and breaks through BS effectively.
Good for: drilling into specific areas of interest
Useless for: covering a lot of ground quickly and useless if the interviewer is not experienced in the technique and does not apply it in a disciplined manner.
I know I’ve missed many techniques and the “high-level questions” group should be broken into several. I’d welcome ideas on how to do it.