If you delete via unlink (as opposed to truncation, for example) and roll via rename (not by copying), you may not have to worry. If you unlink a file that another process is reading, the file will continue to exist until the other process closes the file. This is the basis for an old trick of opening, then unlinking, a file, so it will "disappear" even if the machine crashes. The file can be written and read using the open file handle, even though it has no visible presence in the file system. Similarly, a file that is being appended to by another process can safely be renamed within the file system. The open file is "known" by its inode number, not by the name by which it was opened, and the inode number is unaltered by a local rename.
Don't rely on atime. Many, perhaps most, linux file systems no longer maintain it.