If a big file happened to be created, how do I ensure that it is completely created before it is sftp-ed to the other processing server?
Create the big file in one directory, then rename it into the monitored directory. Renaming a file just edits the directory entries, so the file instantly appears in its entirely in the second directory.
You could also keep the file locked until you are done adding to it.
but I don't seem to be able to find where / how to find the filename of the newly created file.
Maybe you're suppose to read the directory's contents and compared them with what you read previously? I didn't do any research.