Hehe. Brainbench.
My work had an account with Brainbench, as we were required to take two tests. A rather large number of people at work failed the tests. Some of them even failed the same test three times or more.
I was bored, and as I had about 6 weeks before our contract with Brainbench ran out, I just started taking random tests. Now, I will admit, that I failed one test in taking it the first time (Software Business Analysis, which I passed the second time), but either I test rather well, I'm a whole lot smarter than I think I am, or the tests are way too easy. (eg, I passed the certification for CheckPoint FireWall 1, a program which I have never used, and I've spent at most 6 hrs in my whole life looking over someone else's shoulder who was configuring it.
Other fields, which I've done for years, had some absolutely horrible questions that were irrelevent, or had no good answers. For instance, here was the e-mail that I sent Brainbench after taking the Web Server Administrator test, and only getting a 4.21 on it: (yes, yes, there's a whole lot of typos, but it's accurate as what I sent, and Perl is mentioned twice)
Date: Wed, 7 Apr 2004 12:04:44 -0400 (EDT)
From: Joseph Hourcle <oneiros@...>
To: support@brainbench.com
Subject: The Web Server Administation Test
I was recently required to take a few tests from your company,
due to my job, and I thought I'd take a few other tests that were
recommended, but not required. If you will check my records
(transcript 4935509), you will see that I have taken a number of
tests offered by your company, so I do have a grasp of the
overall high quality of the tests being offered.
I just wanted to let you know that I found the Web Server
Administration Test to be possibly one of the most obnoxious
tests I have ever taken in my life.
I expected the questions to be on general concepts related to
webserver administration, not those specific to particular
programs. The questions given were rambling to the point of
incoherency, and some of the questions were completely irrelevent
to webserver administration.
It doesn't come as a surpise to me that I scored as low as I did,
and was still ranked in the 98th percentile.
A large number of the questions seemed to have some parameters
that were not mentioned, that would have significantly affected
the answers.
Eg, for security, what ports should I bind TCP/IP to? You didn't
mention what the network topology was. How was I to know if
TCP/IP was needed on the backend? I assumed this was a question
for some Microsoft OS, as most UNIX servers would have used
TCP/IP for their backend database communications, but you never
stated if that was the case or not.
There was a question about what was wrong with a perl script that
was moved between systems, and why it didn't execute. Of course,
you didn't mention what the error logs said. It might not have
had permission to execute (it said the directory did, but not the
file...or is this one of those MS specific things?). They might
have different paths to the perl interpreter, and need the
shebang line edited. It might be missing 'require'd or 'use'd
perl modules. It might've been transfered from a DOS to UNIX
system using binary FTP, and see 'perl^M' as the interpreter.
You mentioned that the log files had a high 'percent redirector'
or something like that. My log files are in common, or common+.
I've never heard of 'Redirector', aside from squid, and none of
the questions seemed relevent. I made a stab in the dark that
it's an IIS thing, as I know they redirect if they get too much
load, but I don't know what they use to determine 'too much'.
You asked about the problem with giving users CGI access, but you
didn't state if the processes were wrapped, or if the server were
chrooted, which would have affected the answers.
You asked something to the affect of 'what is the weakest part of
a password' or some such, but you didn't mention to whom. I
consider the users and transmission to both be problems. And I
consider transmission to be weaker, as problems in communication
can give up more than just one user's credentials. However, if
I'm using SSL, that's not the problem, the user is.
Many of the questions did not directly relate to webserver
administration.
There were two questions about SMTP, for god's sake. What's
required before you send mail? What does (x) command mean?
That's great if you're a mail administrator, or programming web
services that might require them to send mail, but they have
absolutely nothing to do with webserver administration. Am I
going to see a question in mail administration asking how a
caching proxy should handle POST, as compared to GET? Or what
are some of the difference between HTTP/0.9 and HTTP/1.0 are? I
know I didn't see any of them in this test, so I'm assuming
they're elsewhere.
If I were a network administrator, I might care what I needed to
correctly bridge NETBUI, TCP/IP and IPX networks, but I'm not,
and I don't claim to be. I have people I can fall back on for
that, and we work well together, as I can keep my systems tuned,
and they give me a place to plug into.
Some of the questions had _no_ technically correct answer.
You asked a question about a line that was a SONET B-ISBN
connection with 23 channels, and was 30% used, and was expecting
to double in a year... so what was the total bandwidth? Ignoring
the fact that this is a networking question, and knowing what the
bandwidth of 23B channel SONET connection is completely unrelated
to web server administration, all of the answers were in KB.
First off, I used to work for an ISP, so I infered that you're
talking about a PRI, so it's 64Kb per channel, per second.
Unfortunately, all of the answers were the wrong order of
magnitude for KB, and they weren't KBps or Kbps. The way the
question was worded, and given the unit for the answers, it would
have to be the total information transfered in the year... but
nothing was even close to the right scale for that.
You asked some question about security thay mentioned 'ensure'ing
something is safe. That goes against security. You can prevent,
but you can never ensure, unless you're protecting against a
fixed vulnerability.
Oh...and 'PERL' is not a language. 'Perl' is.
Many of the questions were overly specific.
One of the questions about problems with disk usage gave a
response that mentioned using RAID 5 because the disk usage was
high. It did say that CPU was low, but why arbitrarily use 5?
Actually, this probably falls to not enough information, as it
didn't say that the problem was reads or writes, but none of the
other answers fixed the disk problems, if I recall. If it's all
read traffic, mirrored data, switching to SCSI, or using
something with larger disk cache would have been useful. You
might have also retuned the OS to better cache disk activity,
better distributed your directories on the disks, seperated
competing data repositories, or any number of things. For all I
know, the issue was with the log files not being on an isolated
disk.
I was okay with the tuning questions. I was okay with the
questions about ISAPI, DOM, and the like, even though I've never
used them. I was okay with the questions about web-proxies. But
asking questions specifically about a particular webserver is
just lame. Asking questions that are networking, operating
systems, or programming related are outside the scope of what I
consider to be 'web server administration'. It's one thing to
ask a question about traceroute or how to debug if the problem
isn't yours, and it's someone elses (eg, if it's a DNS problem,
and the IP works).
When I took the security test, and the general UNIX
administration, I thought there were maybe 2-3 questions in each
one that were ambiguous. This time around, I started keeping
count, and I found over half of the questions to be ambiguous,
not have a correct answer listed, or were off topic.
I was particulary annoyed by the overuse of the word 'ensure' in
questions. Every system is unique. We can't define anythingin
absolutely terms based on a sentence or two of description. We
chould choose the answer that 'might help' or 'should best solve
the problem', but it's just plain ignorant to assume that 5
seconds of diagnosis is going to fix a problem, or that there is
one golden solution that will fix all problems.
Please don't claim this test is all-encompasing for skills needed
for webserver administrators. It barely scratches the surface is
some areas, while completely missing in others. It has poorly
worded and constructed scenarios. Personally, I thought it was
an offense to even consider this to be a test for 'web server
administrators'.
-----
Joe Hourcle
