Brainbench

Has anyone had to take tests on www.brainbench.com while trying to get hired? If so, what was your experience? I applied via monster for a company, and was asked to take two tests, one on c, the other on c++. 16 questions each. Some questions were fair. For others, I was left thinking, "really? you're going to weed me out based on my ability to answer this type of question in less than 3 minutes??" It was a mix of annoying and disappointing. I can understand that an employer wants to filter out applicants somehow...I just wasn't encouraged by this particular method. I guess I just wait and see if I get called back...

Do you have an example of such a question?

Tests work both ways - if the prospective employer is asking you to figure out lots of "dumb" tricks in their C test, do you really want to be working for them?

Sometimes the questions may have some second meaning. I mean, they may be more driven towards how they are answered. Do you have any example you could give?

Well brainbench is a third party company that does various kinds of testing. So it really isn't under the control of the paricular employer. First I was given a technical phone interview, where they gave code examples, and I was supposed to find the problem. Easy stuff like deleting the same memory twice, returning a pointer to a local variable, etc. So I breezed through that part, and then the next part was with brainbench.

These were multiple choice questions, and you had only 3 minutes to answer each, and for example many times it had to do with behavior of a particular libc call (in these cases I thought it was annoying because I don't memorize the arguments and return values of libc functions, I just review the man page before I start coding...) and in other cases I felt they were picking unfrequently used language features (I hate giving examples for this sort of thing, because someone is bound to think that whatever I say is must-know information for any good c++ programmer, but I'll give two examples anyway): one question was what's wrong with this code, and it has something like "class D: virtual public C"...sorry I've never even seen the keyword virtual used for inheritance that way, but after some googling it seems that it can be done. And another was something like "char* p[2][2] = { {"xxx","yyy"}, {"zzz","aaa"}}; and then says something like if you wanted to access the element "p[1][1][0]", which of the following would do it, and they give four non-straightforward initialization statements to choose from.

Ok, one more that I thought of, there was void f() throw(std::runtime_error), and then something like try { f(); } catch (some_class) {} and the question is which of the following is valid if substituted for some_class, and they list four of the c++ exeption classes, like range_error, etc. So I know that some_class has to be a runtime_error, but I was forced to run and grab my c++ book, lookup runtime_error, and see exactly what derives from it...

Ok...one more that just occurred to me. It was something like "If you have an ANSI conforming compiler, what will be the result of the following code:"
int j = 0;
for (int i = 0; i < 4; i++) j++;

cout << i + j << endl;

So there are various choices of numeric results, and there is the choice "This won't compile because i isn't in scope." I thought this was annoying because I know that this code compiles on many compilers, but the bit about "ANSI conforming compiler" left me wondering if ansi says i should not be in scope. And then I was thinking, "Who cares?", if it compiles, it compiles, and if it doesn't it takes me all of 30 seconds to fix it up. It says nothing about my c++ skills really...

I will say that some questions seemed both good and fair, for example on the C++ test there was a question related to the correct way to define a class assignment operator.

Quote:

---

So there are various choices of numeric results, and there is the choice "This won't compile because i isn't in scope." I thought this was annoying because I know that this code compiles on many compilers, but the bit about "ANSI conforming compiler" left me wondering if ansi says i should not be in scope.

---

Variable i is not in scope at line 4. I think this falls under the "someone is bound to think that whatever I say is must-know information for any good c++ programmer", but then it was not one of your two examples :P

Yes the scope thing is an addition to C98 that I do believe falls under the category of "You really should know this".

Ok here's my opinion. Take it with a grain of salt. Mainly because I'm not proficient in any way in C++ so I couldn't possible judge what is good or bad question. But I did write a lot of Visual Basic questionaries to potential candidates in the firm I used to work for. At the time I had to study on the subject.

The "class D: virtual public C"

This is what is called (I hope the translation to english stands), Technical Grade Question. A question meant to evaluate in a gross manner the technical skills of the candidate. It's weight varies. But it has two main reasons: 1 - to establish if the candidate is skilled beyond what is considered "normal" and 2 - It, in case the answer (and the way it was obtained) satisfies, can help establish if the candidate can run for higher positions.

The char* p[2][2] = { {"xxx","yyy"}, {"zzz","aaa"}};

I almost bet this one is intended to crudely establish how well you relate to the language. Basically it helps establish if you feel comfortable around it. Your experience in programming, per se. It's weight (like really with every other question in a questionaire) varies.

The f() throw(std::runtime_error)

Probably the same as the previous one. Although there is some level of basic (and yet important) knowledge you are required. I'm tempted to say this is one of those questions they don't like if you fail. I think this question has the highest weight of all the ones you shown.

......

The whole point of these tests is to rarely be eliminatory in any way. Trust me when I say that many variables come into play. It's not a simple matter of counting how many answers you have wrong and how many you have right. I do know for a fact not always the tests are good. The "science" behind them is at best debatable.

However, be as it may, my experience in running these tests on applicants for job positions in the company I used to work for, wasthat for the most part those questions that had a meaning beyond just the basics were exactly the ones that annoyed applicants the most. So... there is a pattern here.

The gross of the tests, all those questions that you considered fair, are the meat of the test. Those are the ones that are basicaly looked at by the reviewer as "X correct - Y wrong". They are also the ones with the highest weights. I do think that the try() question above might fit into these though.

I got the throw question right. The one about the for loop I guess I got wrong...I know that VC allows that...or did allow that at one point. When I was answering the question, I remember thinking, "I bet VC is a terrible measure of ANSI conformance..."

Yes, VC6 is a terrible measure of ANSI compliance.

> and for example many times it had to do with behavior of a particular libc call
Which leads on to why I don't like these sorts of tests. I for example can never seem to remember what order the middle two parameters of fread() and fwrite() are - is it number,size or size,number.

All they do is test your ability to understand program fragments in some context. The real trick in being a competent software developer is how you can put all this stuff together to produce meaningful programs.

They're good for test sellers because they're easy to administer and check automatically.

Quizzing people on the facts of the language is easy to coach for - any trained chimp should be able to pass them after a while. But it won't tell you a damn thing about whether you're a competent programmer or not.

It's like going for a job in journalism, where the only test is how many word definitions you can recite from the dictionary. Tells you nothing about how good a writer you are.

Many companies need to pre-establish a candidate level of knowledge before they hire them, Salem.

The tests are not meant to mimic school. They are not exclusively evaluation tools. They are meant instead to help establish among the candidates, those that are probably more apt. They are also meant to establish some patterns (like if the candidate if comfortable around a particular library, for instance).

The only way for a company to know for sure how good candidate are, is to hire them.

I don't remember any of the DirectX/3D,MFC, Win32, C/C++, and STL library functions when I write code. The good old DirectX/3D,MFC, Win32, C/C++ and STL help files are always open in my IDE. But to me that doesn't test your programming skills b/c memorizing those will get you nothing but a major headache. Imagine trying to memorize the Win32 CreateWindow() or the D3DPRESENT_PARAMETERS structure. That means nothing to me.

Knowing how to use it once you find the thing in the SDK is different.

Standard Test only measure how well you know the standard. In eight years of 40h/week programming, I have never caught an std::exception and I give a rats ass about which class derives from it. If I need to know, there will be plenty of time to look it up. I'm not a real time programmer.

I like tests that present knowledge and ask for an interpretation. I dislike tests that directly ask for knowledge because you cannot know everything. That is what documents and machines are for. Accessing knowledge. Interpreting this knowledge is my job. Not the other way round.