Saturday, November 17, 2012

General Impressions about InterviewStreet

As a longtime participant of programming competitions (ACM-ICPC, TopCoder, Code Jam, Codeforces, SPOJ, etc...), I initially found the idea of InterviewStreet to be promising. Like TopCoder and other similar competitions, it builds connections between companies and a wide talent pool. However, I feel the attention that InterviewStreet receives is undeserved. While it excels at attracting industry partners to sponsor it and use its platform, the quality of the competitor-facing side of the website is very lacking.

The main problem that I have with InterviewStreet is that the website's problem statements are very rough and unpolished, often lacking crucial details and having contradictory statements or examples. While it seems like the people behind InterviewStreet are familiar with other competitions, they are rather poor producing problem statements of the same quality. The writers' seem to have a poor grasp of English, and are very bad at expressing things in an unambiguous, mathematical manner, as is required by these types of competition sites. For example, consider the following screenshot of a problem statement:

When I see this image, I immediately notice a few things:

  1. The final sentence of the first paragraph asks the reader to "find out the number of zombies at every junction after 'k' timesteps". Many problem-solvers will realise that the problem asks for the expected value of the number of zombies, but to new users or to users for whom English is not a native language, this sentence can seem very confusing.
  2. It's not specified what a zombie will do in the case where there are no neighbouring junctions.
  3. The input format doesn't specify whether the numbers are separated by spaces, newlines, commas, or arbitrary characters. Once again, one can read the example input to figure this out, but this should not be necessary. The input format should completely describe all possible input.
  4. There are lots of stylistic issues. All the input variables are uppercase, except for 'k', which is lowercase. Word choice is not the best, and the grammar and spelling are terrible. All in all, these small details make the statement much harder to parse, unlike statements on other websites like TopCoder.

This is not an isolated case -- in the InterviewStreet Collegiate Challenge last spring, there were contradictions between the problem descriptions and the input, causing participants to lose valuable coding time. (Strangely enough, the results of the collegiate challenge are no longer posted on InterviewStreet's website.) Many of the other challenges on the website suffer from the same problem. The result of having poor problem statements is like having students take a poorly-written exam in a class -- less-talented students may stumble upon correct answers, which smarter students may fail. In this case success on InterviewStreet is as much dependent on being able to decipher poorly-written English as it is on having true programming skills. I do not exaggerate when I say that some of these problems contain the worst writing that I've ever seen in competitive programming. For comparison, look at the ACM-ICPC, TopCoder, Google Code Jam, or Kaggle. Should InterviewStreet wish to remain competitive (or be taken seriously, for that matter), it should put more effort into producing higher-quality problem statements.

InterviewStreet also fails to leverage the power of the programming community. One of Codeforces' first features when the launched in 2010 was to provide a blogging framework for its participants. Likewise, TopCoder has a wiki and forum, and hosts an extensive series of community-written articles. InterviewStreet has a forum, but few people seem to use it (perhaps because of the problems above).

Despite all of this, InterviewStreet does a good job in attracting industry partners, which I would attribute mostly to hype and rhetoric about its appeal as a 'startup' with potential to 'disrupt' the talent search process. If I were a company or startup looking for talent, I would turn instead to Kaggle, TopCoder, or Codeforces -- websites which provide great platforms for similar competitions, with well-established communities and some of the most impressive talent pools in the world. In the past I have heard that InterviewStreet is better for job-searching than TopCoder, but if the current trend continues, that will no longer be the case.

3 comments:

  1. Hi Richard,

    Thanks a bunch for the feedback. We understand these issues and we are working on these. We now check the problem statements of the new codesprints we add.

    Please feel free to drop a mail to founders@interviewstreet.com if find more problems with our website.

    Thanks again for the feedaback

    ReplyDelete
  2. Hi Richard,

    I'm Vivek, one of the founders of Interviewstreet. I'm very sorry about the problem statements and I've to tell you that we're working to correct it. We even advertised for a content writer role (http://jobs.hasgeek.com/view/s7wq6) and taking steps to better it.

    Zombie March problem: All the points you had mentioned have been corrected and the problem statement has been modified.

    Differentiation between CodeForces, Topcoder, ACM, etc.: Every company has a specific agenda and goal towards which it'll move. Plainly copying the features might not work great. For instance, I have no idea why Codeforces or Topcoder doesn't include companies. Maybe their goal is different. And ours too is different.

    We went through a little bit of rough times in the last couple of months but are back strong as ever. I really appreciate your comments, blog posts about us, but I'd like to tell you that there are years of hardwork behind signing up companies and not just hype.

    Thank you again for the comments. It only makes us better and we'll be better. You can directly write to me at my firstname at interviewstreet for any queries.

    ReplyDelete
  3. Hi Richard,

    I am Marcin Kubica, CSO at Codility, Interviewstreet's competitor.
    Maybe you would write a critical review of our service too?
    (I can grant you a free access to our service.)
    It would be interesting for the Readers, and we would also value it a lot.
    It is always good to know one's own weak points.

    You can write me at my-second-name@codility.com
    --
    Best,

    Marcin Kubica

    ReplyDelete