A typical place where Encode::Guess falls down (through no fault of its own) is in differentiating one variant of iso-8859 from another.
Who's to say if chr(250) is "Č" (ISO-8859-2) or "Θ" (ISO-8859-7)?
Without prior knowledge, you're up the creek without a paddle. So I agree wholeheartedly with Moritz's suggestion of converting everything to UTF8, while you still know what encoding it is in.
(graff - I know you're too wise a monk to have been suggesting otherwise, but I wanted to provide a simple example of just how limited Encode::Guess can be.)