Quantcast
Channel: 500 – WordPress.org Forums
Viewing all articles
Browse latest Browse all 2443

anmari on "[Plugin: amr ical events lists] Semi-pagination causing hundrends of 500 errors"

$
0
0

It's a pain. Have you tried lowering the crawl rate (it's a temporary thing). Unfortunately it does happen (not just with events)

I like this writeup
http://www.blindfiveyearold.com/crawl-optimization

it made me wonder - maybe the answer is to decide how far you want google to index your events (x months) and create a few linked pages using the shortcode parameters to control what's on each page.

(EG: this 2 months, next 2 months, next year say....)

and then NOT have the semi-pagination or months navigation etc, just link the pages (it's not really pagination since it is date based)

What do you all think ? Would be interesting to have someone implement and test the idea out.

Not just us:
http://2bits.com/crawler/google-crawler-hitting-your-site-too-aggressively.html

http://www.askingbox.com/question/googlebot-crawler-comes-too-often-how-can-i-limit-the-frequency
http://forums.seochat.com/google-optimization-7/how-reduce-googlebots-traffic-without-harming-ranking-470599.html

Most advice says: welcome it and manage it (the google bot)

and maybe not always the google bot itself
http://stackoverflow.com/questions/1328436/how-to-prevent-googlebot-from-overwhelming-site

wow - 50% of web traffic in singapore is bots !
https://blogs.akamai.com/2013/06/identifying-and-mitigating-unwanted-bot-traffic.html


Viewing all articles
Browse latest Browse all 2443

Trending Articles