Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Sessions & Search Engines

by lostjimmy (Chaplain)
on Apr 05, 2010 at 18:22 UTC ( [id://832877]=note: print w/replies, xml ) Need Help??


in reply to Sessions & Search Engines

Figure out what the different search engine crawlers use for a user agent, then configure your web server to skip the language page for those user agent strings.

Replies are listed 'Best First'.
Re^2: Sessions & Search Engines
by ahmad (Hermit) on Apr 05, 2010 at 21:31 UTC

    I've had this idea, but this way I cannot make it crawl the content in multiple languages, I will have to set a default language only.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://832877]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (4)
As of 2024-03-28 14:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found