Re^4: Perlmonks Response Time Bottoming Out (cache-control header)

[LanX]

> One downside of having the site accessible at all is that people have again a venue to complain about the site responsiveness.

I'm not complaining, please see it as a brainstorm of options to pick from.

> I think they are unlikely to have any impact.

I'm ignorant about the bot's heuristics. Maybe they act differently on an access denied? ¹

Some other suggestions you are free to consider:

I started the chrome-console and looked into the web-response of one static page (ID:100001).

They have a cache-control max-age=0 indicating that the client shouldn't see them as static.³

So a bot might conclude it may be worth polling it again and again in short intervalls.

Various other embedded resources are also loaded with delay, and sometimes those take even longer time to load than the above initiating page

• https://perlmonks.net/index.pl?node=list%20replies%20toggle%20javascript (toggle replies JS)
• https://perlmonks.net/index.pl?node_id=204962 (CSS: Blue Web-Safe theme)

It seems like they are dynamically pulled from a node for each cached page, even if their content is super static (JS and CSS) and they have no cache control setting.²

Saying so, I'm quite inexperienced in server settings, cache control and bot behavior , so my observations may be irrelevant and my timings may be limited by the fact that I use a mobile web access.

Please ignore me if it is so ...

updates

¹) better even a HTTP-503 Service Not Available with a retry-after set to 60 min

²) When being logged in, my browser is indeed caching those embedded files, not sure whats going wrong here.

³) Apparently is at least the google-bot honoring cache settings. Probably delivering the static/cached files with cache-control headers might eventually help reducing load?