4
votes
4answers
547 views

Using GNU Parallel With Split

I'm loading a pretty gigantic file to a postgresql database. To do this I first use split in the file to get smaller files (30Gb each) and then I load each smaller file to the database using GNU ...
4
votes
3answers
1k views

shell script to read from multiple files in parallel

I need to write a script that runs parallel and looks for a string in multiple files. I tried a lot of options but they slow down the speed of my processor.
3
votes
3answers
2k views

process files in a directory as they appear [duplicate]

Possible Duplicate: How to run a command when a directory's contents are updated? I'm trying to write a simple etl process that would look for files in a directory each minute, and if ...