I need to code a daemon that track the content of log-files on multiple hosts. The "log drives" (hosts are Windows Servers) are all mounted in a single Linux host.
The tricky part is that the log-files will come (when uploaded by client of servers) and go (when archived), and don't have fixed names. They all have the naming pattern "app.DATE.log.N", and the directories are fixed (list of servers is fixed).
So I need a "tail -f" that takes a "pattern" instead of an actual name, and check regularly (every few seconds; as a near real-time monitoring) if new files have been created with this pattern, and can handle files being delete too (they are moved and compressed after 24 hours).
Ideally, I would just use existing unix tools for the "tracking", so I can code a simple daemon that reads from stdin. And the solution should be robust enough to deal with IO errors, since we are talking about drives mounted over the network.
I've found that there are tools like "lnav" and "multitail", but both seem designed for interactive use, which is not what I need.