I thought (hoped) that grep would have the behaviour you described; but even when I use a very unlikely regular expression, perl's memory usage increases at a constant rate.
The rate doesn't appear to differ depending on what regular expression I use, which seems to imply it's not the matches (of which I doubt there are any) filling memory, but just the input from /dev/urandom (which, as you mention, is endless).
The EOL delimiter is probably \n, which should occur in /dev/urandom 1 in 256 times (so long, but not that long).