Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

OT: Spam protection

by t0mas (Priest)
on Jun 10, 2003 at 07:42 UTC ( #264559=perlquestion: print w/ replies, xml ) Need Help??
t0mas has asked for the wisdom of the Perl Monks concerning the following question:

Slashdot have some news today about a survey from Symantec saying that 80% of the children with e-mail addresses receive inappropriate spam mail. This is something I've been thinking about for a few months, since my children want to "get on-line".

I receive about 10-15 spams a day from people who want parts of me to grow bigger, offer me wonderful oppertunities to get rich, and giving me other even more fantastic offers. Mozilla does a great job of filtering away most of theese, but some pass through (mostly those with only a single link in the mail-body). Some of which I don't want my children to watch.

So I've been thinking of some kind of dual inbox thing to weed out more of the mass-mailers (inspired by SPAM eater by boo_radley):

1. Inbox #1 gets processed with SpamAssassin
2. Scan inbox #1 for mails with senders not in addressbook
3. Send mail with confirmation text to unknown senders.
4. Move mails with known senders and confirmed senders to inbox #2
5. Update addressbook with confirmed sender?
6. Child fetches mail from inbox #2 with Mozilla

The confirmation mail would have a text string "encoded" with FIGlets and a polite text explainig that this is the mailbox of a child and asking the sender to reply to the mail with the FIGlet "decoded" string in the mail body.

I think the spammers will have a hard time to reply to such a mail (since the sender@server.com, abuse@server.com and postmaster@server.com seldom exists).

My questions to you, my fellow monks, are theese: Would you be offended by a confirmation mail? Is this a resonable protection (used together with SpamAssassin and Mozilla)? Have someone done something like this before? In Perl?

Thanks for your time.


/brother t0mas

Comment on OT: Spam protection
Re: OT: Spam protection
by Corion (Pope) on Jun 10, 2003 at 07:58 UTC

    One thing to consider is, will Mozilla (or your spam protector) automatically add recipients of your outgoing mail to the whitelist?

    If this is not the case, prepare for a lot of unanswered questions. I will not authenticate myself if you sent me a question and my answer bounces, and it's a matter of courtesy to add people to your whitelist that you send a question to - so this must be automated.

    Other than that, I think you are fine as long as you explain (and your child accepts/understands) why you are monitoring your childs email ...

    perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web
      Thanks for your reply Corion.

      Yes, Mozilla (at least as I've configured it) automatically adds addresses that you send mail to, to your addressbook.

      I really don't want to monitor their mail, I want the process to be automated without me in the middle. The must have their privacy. We have had lots of talks about the internet at our house, and they are fully aware that some pages on the internet are not intended for them. I just want to keep theese pages from hitting them when they don't want them.


      /brother t0mas
Re: OT: Spam protection
by BrowserUk (Pope) on Jun 10, 2003 at 08:12 UTC

    I'd have to caution against ever responding to any UCE.

    Your response is unlikely to ever been seen by human eyes, much less acted upon in th eway you would hope. Any form of response is only ever likely to be read by a computer which will simply transfer the email address from the "possibles" list, to the "Hey, We got a real live, living mug" list.

    I'm afraid I don't have an answer for you, and I seriously do not envy you the task of protecting your kids from this sort of stuff.

    I have a half-cocked notion that the only real way to protect them is to show them the stuff, explain it to them, explain why the parasites that send it do so.

    Perhaps, by getting it into the open, raising the subject to the level of "stupid adult stuff" that they can talk (and laugh about) with you, you can make it become as insignificant as many of the other irritations of daily life are, rather than allowing it to become a "super secret" that they will try to hide from you.

    There I go, moving into an area of life, child rearing, for which I am totally unqualified to express an opinion.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      There I go, moving into an area of life, child rearing, for which I am totally unqualified to express an opinion.

      Aren't we all? As a father of five, I still move into such areas all the time. Like this...

      You have a very good point there. And I'll certanly give this a serious thought before starting my code editor. Getting the public address to a "live" list would not be such a good idea.

      rather than allowing it to become a "super secret" that they will try to hide from you.

      Well, my children have full access to the internet (web) (I don't even snort the traffic) so they can watch pretty much whatever they like without me knowing. If they choose to watch some "inappropriate" stuff, it's a bad choise, but it's their choise. I just want to stop the bad choises from hitting them when they make want to make a good choise.

      /brother t0mas

        I just want to stop the bad choises from hitting them when they make want to make a good choise.
        Well... I get lots of spam just because I published source code on CPAN. I tend to think that participation on the internet precludes any exclusionary mind set absent a human filter. You could pre-vet their e-mail for them if you really needed to, its not as if SpamAssassin is all that great anyway.

      There's no problem with responding to spam if you are using a challenge-response authentication system. If the spammer doesn't respond back properly (which he won't, since the pammer is a program routing mail through China and using a fake MSN address), you will never see any of his mail. He can send you offers all day and they will never reach your inbox.

        Whilst it is true that you will block that one email address, and therefore never receive (or at least see) stuff from that source again, the fact that you have responded means that your email address in live.

        So, not only will that spammer ensure that he spams you from the next 10 50 100 email addresses he creates for his spamming, the fact that he can list a response from that address along with the address, means that he can charge premium rates for it when he sells it as a "known live" address to other spammers.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


      As perrin pointed out, no serious spammer uses valid sender addresses in their mail these days. Replying to spam will result in a bounce far more likely than confirming your email address as live. Challenge response systems won't expose you to more spam, they will however roughly at least double and may even quadruple your mail traffic (depending on how many different partners you correspond with).

      Makeshifts last the longest.

Re: OT: Spam protection
by arthas (Hermit) on Jun 10, 2003 at 08:34 UTC

    It's a bit complicated, as it might require some more time that it should for e-mails from unknown senders to get to your child. Anyhow, most legitimate people will probably reply, so your solution is probably going to work.

    The only thing is that sometimes, by replying to spammers, you let them know that you use that address, so it becomes more valuable (and used) by them, that is to say your spam amount may grow. But, then, it's a minor problem: it always grows anyway. :-(

    Michele.

      You have the same valid point as BrowserUk have in Re: OT: Spam protection and I'll take this into account. Thanks

      But, then, it's a minor problem: it always grows anyway. :-(

      Yes, it does, doesn't it... my Spam folder contains 2529 spam mails as of today.

      Update: 2530 :-(

      /brother t0mas
Re: OT: Spam protection
by sauoq (Abbot) on Jun 10, 2003 at 09:25 UTC

    Wow. What a tough problem. It's also one that I'll have to try to deal with in a few years so I especially appreciate you paving the way. ;-)

    I think the use of SpamAssassin and Mozilla is a good starting point.

    I have the same reservations about automated replies that BrowserUk voiced. But if you do go that route, FIGlet might not be the best choice. A figlet "encoded" string might be almost decipherable to someone who views it in something other than a fixed width font.

    -sauoq
    "My two cents aren't worth a dime.";
    
      A figlet "encoded" string might be almost decipherable to someone who views it in something other than a fixed width font.

      Ah, somthing else I didn't think about! I only use fixed font plain text viewing myself...

      I thought of FIGlets since they are easy to create, easy to read by the human eye, but require work to decode by a computer, and I don't think a mass-mailer would bother to do that.

      Thanks for your input.

      /brother t0mas
Re: OT: Spam protection
by wufnik (Friar) on Jun 10, 2003 at 09:32 UTC
    hello

    i have a similar problem with aunts who have requested to be shielded from the world of organ enhancement etc, and recommended they use ...hushmail, my own mail address, for legacy reasons. why?

    it uses something rather grandiosely titled a 'human authenticator system' which, instead of requiring the user to stringify a figlet, asks them to click on an image.

    your system is certainly no worse than this. as a matter of fact, with spam assasin, it is *more* rigorous. though this might not be necessary, as i now never get mail from emarketers. there is some debate as to how user-friendly 'human authenticators' are in the politech mailing list, but it gets a thumbs up from me.

    if i had discovered spamgourmet earlier, though, there might never have been need for this. using this and a human authenticator would be overkill, and besides, you'd have to keep adding spamgourmet addresses to your allowed list. but if your children are older, you might consider this instead of the above.

    best of luck, (thank god my 3yr old cannot read)

    ...wufnik

    -- in the world of the mules there are no rules --
      I've had some thoughts about having web based validation of mails instead of a reply-to validation mail, but I discarded that idea since I think it will require more work than validation mails, and I'm lazy :)

      If for some reasons (like using non-fixed fonts), a person can't read the FIGlet, a web validation link as an option will be a good idea. So now I may end up doing both.

      Thanks,

      /brother t0mas
Re: OT: Spam protection
by perrin (Chancellor) on Jun 10, 2003 at 14:50 UTC
    Since switching to a newer version of SpamAssassin, it catches nearly 100% of all spam and has had no false positives, and I get about 40 spams per day. I think it is enough for most people. However, if you want to be 100% certain, just use any of the existing response-based mail systems out there. There are lots of them. Spam is sent by cheesy automated programs. You will not get responses back from spammers. Most of their addresses will bounce.
Re: OT: Spam protection
by CountZero (Bishop) on Jun 10, 2003 at 18:17 UTC

    I would not be offended by such a "challenge/response" system.

    However, I once wrote a similar "challenge/response" program in Pegasus Mail to weed-out spam mail from a number of mailing-lists I am on.

    I seemed to have hit a raw nerve with many of the members of these mailing lists. All they had to do was sending a reply with in the body of the message "Please add me to your list".

    Far too many to count wrote pages and pages about what a disgrace it was to even dare to think that their sacrosanct messages could be considered spam! I almost had a petition started to remove me from the list.

    YMMV, but I would prepare myself for a lot of strange reactions.

    CountZero

      It is very annoying. Some chap with a challenge response system pestered everyone on the perl5-porters list with his messages. Do you have even a faint idea of how much traffic it would generate if everyone on a high-traffic list (which p5p definitely qualifies as) used such a system? On a challenge response system's Freshmeat project page, someone reported his experience trying to run it on a company mailserver to reduce the time wasted with spam, and said (besides annoying some customers and business partners) it nearly quadrupled his mail traffic due to all the extra mails a single message generated, to the point where the server was no longer able to handle it. He was forced to revert to traditional filtering methods.

      Makeshifts last the longest.

        I see, but to my defence, it was not a high traffic mail-list and the challenge went only out to "suspect" addresses ("hotmail" accounts and similar throw-away addresses)

        Of course the challenge was not posted to the list, although the answers of those who were challenged did make it to the list!

        Now I have given up on the challenge-response system and use a Bayesian (not sure about the spelling here!) filter which filters better than 98% of all spam.

        Strange as it may seem, now that my ISP has installed their own anti-spam protection scheme, the efficiency of my local filtering system dropped.

        CountZero

        "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

        It can be made less annoying if you just use the challange/responce once to verify each new email address that send to the list. Reply back with a message that has a encoded url that verifies the email address as "ok" and also maybe have a way to reply to the message as an ok trigger. to quadruple the number of messages you are sending would be impossible. Worst case you would send two times as many messages assuming that there is never a repeat poster on the list.
        it looks like this:
      • New_person@me.org sends an email to the list
      • The list notices that New_person@me.org has never been authed before.
      • It moves the message from the active queue to a hold queue
      • It adds a entry to the auth system db with the queue ID.
      • Generates a message to the sender with a url like http://mylist.org/authrequest/12314hb4ds54 and a tag in the message so he can reply to get auth and instructions.
      • New_person@me.org gets the auth request and clicks on the URL.
      • The auth DB is updated and the hold queue message is released, the "New_person@me.org" is placed in the authenticated db.
      • New_person@me.org sends another message to the list the mail server sees that the email address has been verified and the email is allowed through.
      • Daily a cron job rotates through the hold queue deleting mail that has been there for more than 1 week without being authenticated.


      • If that process puts too much load on your email server there is something wrong, There is no way that method could be more intensive than SpamAssasin or some other filter software.


        -Waswas

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://264559]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (9)
As of 2014-12-27 20:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (177 votes), past polls