Re: [OT] HTTP downloads and caching

[kcott]

G'day Rob,

I tried this using wget on Mac OS X 10.10.3 as follows:

ken@ganymede: ~/tmp
$ wget http://www.sisyphusion.tk/ppm/Math-Float128.ppd
--2015-10-07 15:10:03--  http://www.sisyphusion.tk/ppm/Math-Float128.p
+pd
Resolving www.sisyphusion.tk (www.sisyphusion.tk)... 184.154.90.58, 18
+4.154.90.58
Connecting to www.sisyphusion.tk (www.sisyphusion.tk)|184.154.90.58|:8
+0... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4302 (4.2K) [application/vnd.cups-ppd]
Saving to: `Math-Float128.ppd'

100%[=================================================================
+=============>] 4,302       --.-K/s   in 0s      

2015-10-07 15:10:06 (216 MB/s) - `Math-Float128.ppd' saved [4302/4302]

ken@ganymede: ~/tmp
$ ls -al Math-*
-rw-r--r--  1 ken  staff  4302  5 Oct 15:23 Math-Float128.ppd
ken@ganymede: ~/tmp
$ grep 07 Math-Float128.ppd
ken@ganymede: ~/tmp
$ grep 08 Math-Float128.ppd
<SOFTPKG NAME="Math-Float128" VERSION="0,08,0,0">
    <PROVIDE NAME="Math::Float128" VERSION="0.08" />
ken@ganymede: ~/tmp
$

So, it would appear that I'm getting the "new" version.

I don't know that I can help much with the "why is this happening?" question. The following is just a shot in the dark.

I encountered a situation in the past (actually about 15 years ago) where a service provider had multiple servers and changes had been applied across their servers — except they missed one (little used) server when applying the changes. The result being that everything worked as expected most of the time but, on the rare occasions when the unchanged server was accessed, problems ensued.

This was so long ago that I really don't remember more details than that; however, that may be an avenue worth investigating. Your comment "The company that hosts the website says it can't possibly be their fault ..." brought this to mind as I seem to recall that was pretty much the initial feedback I got at the time. Even the senior "technical guys" refused to entertain the idea that the problem was at their end; however, this was for a substantial commercial project which allowed me to escalate the issue to their senior management and get all the servers checked (which resulted in the problem being found and fixed). That's not the sort of leverage I'd normally command; unless there's "big bucks" involved, you may be stuck with the "not our fault" answer.

[syphilis]

Yeah, my initial thoughts were along the lines that the hosting company had somehow stuffed up a server upgrade.
If I can find a case where someone other than me can hit the same issue then I'll consider entertaining the same thought again.
But, as it currently stands, it apparently affects only me and it happens to me *every* time I download that file over http.
OTOH, if it is affecting only me, then that doesn't really matter - that would be good news.

There are some other oddities that I didn't mention.
When I updated www.sisyphusion.tk/ppm/Math-Float128.ppd, I also updated www.sisyphusion.tk/ppm/package.xml - but there's no problem grabbing the updated version of package.xml.
It's only the updated version of Math-Float128.ppd that I can't grab.
Now that's really odd, isn't it. Two files in the same directory - no problem getting the current version of one of them, but I can only get the *previous* version of the other !

Actually, there's at least one other file (in a different directory) that was also updated, and is affected in the same way.

Anyway, thanks for checking - it helps to know that it's not *just* the hosting company that can wget the current version of the file.
And thanks for going to the additional trouble of thinking about it, and of articulating those thoughts.

I would strongly suspect that my problem was indicative of some caching on my local machine if not for the fact that it's happening on all three of my local machines - two (ie the two Linux ones) of which don't get fired up all that often.
That all three machines are affected in the same way leads me to think that it's my ISP that's throwing the spanner into the works. (My next step will probably be to see what my ISP thinks of that hypothesis :-)

Cheers,
Rob

[hippo]

That all three machines are affected in the same way leads me to think that it's my ISP that's throwing the spanner into the works. (My next step will probably be to see what my ISP thinks of that hypothesis :-)

That sounds like the obvious conclusion to me, too. You could always check whether it is a caching issue (either at the ISP or elsewhere) by appending a query string to the URL. Or indeed by using an https URL if one is available for that resource.

Good luck with your investigations.

[syphilis]

You could always check whether it is a caching issue (either at the ISP or elsewhere) by appending a query string to the URL

I don't think I would *ever* have thought of that. What good thinking !!

So ... when I append a (random) query string to the URL I get the "new" file, but when I omit the query string I get the "old" file.
This surely demonstrates that it's a caching issue, but is there a way for me to pinpoint the location of this cache ?

Afterthought: If wget is accessing a cache on the local linux machine, where would that cache be located ?

Cheers,
Rob