Perhaps the easiest way would be to keep a record of how much of each file you have already read. Then your test for it having been written to, is just checking the current size of the file. A tied hash might be one way of persisting this information.

Perhaps the harder thing is to determine when you've read all you are going to get. Ie. Distinguishing between your reader having caught up with the writer and the writer having finished.

If you are reading the files in seperate threads, then you can afford to have the thread hang around waiting for the file to grow some minimum amount before reading the next line, and if it fails to meet that minimum within a preset amount of time, you read and process whatever is left and call it quits.