http://www.perlmonks.org?node_id=569435

perl_lover_girl has asked for the wisdom of the Perl Monks concerning the following question:

This node falls below the community's threshold of quality. You may see it by logging in.

Replies are listed 'Best First'.
Re: proof of concept how to run this code
by starbolin (Hermit) on Aug 24, 2006 at 22:53 UTC

    Do you have permission from the forum owner to do what you are attempting to do? Most forum owners would not appreciate the kind of load your script would put on their system. Many forum owners have filters in place to prevent robots like these. You would most likely get your account revoked and a nasty curse placed on your children.

    If you can convince the forum owners of a legitimate need they can probably give you direct access to their database. This puts a lesser load on their servers and offers you more options for searching/retrieving.

    I am sorry to say that your code is a mess. I don't know where to start. I think that you are biteing off too much at once. I would start with some simpler tasks. Like, just grab one page and print it. You should also write separate little programs to test your subroutines before using them, so that you know if the parameters are passed correctly.

    The text you posted here contained double breaks which made the post display funny. Use paragraphs <p> instead.

    In spite of the rain I put on your parade we appreciate your participation in our site and hope you continue to visit.


    s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: proof of concept how to run this code
by jdtoronto (Prior) on Aug 24, 2006 at 20:06 UTC
    Couple of problems:

    • You have repeated the code twice in your post, can you clean that up please.
    • What are you trying to do? This looks like some sort of forum ripping robot?
    • As submitted the initialisation of the LWP::RobotUA with the  new method seems to go into lala land and just get lost, but then the invocation of the method does not appear to be in accordance with the documentation.
    What testing of this have you done so far? Where were your problems?

    jdtoronto

Re: proof of concept how to run this code
by skx (Parson) on Aug 25, 2006 at 09:43 UTC

    You don't seem to have answered the questions asked by the other monks above, so I'm not sure if you're actually having permission to do this or not.

    If you have permission then it seems that your goal of inserting all the thread data into a database is pointless - it is already in a database on the server!

    So, assuming you have permission to copy the details, etc, then you should simply ask the host to export the database for you.

    If you don't have permission then spidering the site is probably your best option - if you do it slowly and carefully.

    However down that path lies madnes..

    Steve
    --
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: proof of concept how to run this code
by cowboy (Friar) on Aug 24, 2006 at 20:48 UTC

    I'd suggest reading How (Not) To Ask A Question. There isn't much we can do to help unless you explain what it is you are trying to accomplish rather than just throwing a pile of code at us

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: proof of concept how to run this code
by mantra2006 (Hermit) on Aug 25, 2006 at 17:49 UTC
    Hello

    I have gone through your question and follow ups from you and looks to me you are expecting somebody to write a perl code for you

    programming doesn't work like that..this is where software development life cycle comes into picture

    you got to put down requirements and analyse those requirements and once requirements are confirmed then based on that you got to design the project. Then comes the coding part where one who is developing will decide to use perl or some other to develop then testing implementation and maintainence..

    And also please specify when you run this code where is the point you are getting errors and what errors

    its very hard to read the code and tell you will it work if I run type of questions


    Sridhar

    Edited by planetscape - removed unnecessary br tags and replaced with p tags