If a commercial company sells a tool that allows non-programmer end users to write little screen scraping robots, is it unethical or illegal for such a product to not provide a mechanism to allow their end users to respect robots.txt?
First of all, I don't think the answer to the question depends on whether the tool is sold or not, or whether it's produced by a company that may, or may not, be commercial.
As for the legal status, like many other things on the internet, that will remain unclear for a long time. Jurisdiction concerning internet matters grows very slowly - the fact the internet doesn't care about our legal borders is a not unimportant factor in the slow growth of jurisdiction.
As for the ethical thing. If it doesn't allow a mechanism to respect robots.txt, it's certainly a bad product. And I guess selling bad products is unethical. It's certainly unethical for the end user to use such a tool though. (Well, IMO). It's the user of robot created that way that carries the responsibility - just like you carry the responsibility your car has headlights and brakes, the judge won't accept a defense of "but it was sold to me this way".
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|