3
votes
2answers
101 views

split a file, pass each piece as a param to a script, run each script in parallel

I have a words.txt with 10000 words (one to a line). I have 5,000 documents. I want to see which documents contain which of those words (with a regex pattern around the word). I have a script.sh that ...
5
votes
6answers
8k views

How to split such a string into array in bash

I have problem with output of a program. I need to launch a command in bash and take its ouput (string) and split it to add new lines in certain places. The string looks like this: battery.charge: ...
4
votes
4answers
189 views

How do I break up a file like split to stdout for piping to a command?

I have a large .sql file full of SELECT statements that contain data I want to insert into my SQL Server database. I'm looking for how I could basically take the file's contents, 100 lines at a time, ...
0
votes
2answers
523 views

Bash split a list of files

I have 200 files in a folder like: test_1_cfg.dat, test_2_cfg.dat,.... and so on. I need to include in a bash script the first 40 files as input and run some process, another script to read the next ...
1
vote
3answers
423 views

Is it possible to split file and append the pieces with same extension in one liner?

I did it in two commands --- one for splitting, and one for moving. I can't seem to find my way around it such as to have it all done with one line. What I did was: split.exe --bytes=2300k -d ...
4
votes
4answers
636 views

Using GNU Parallel With Split

I'm loading a pretty gigantic file to a postgresql database. To do this I first use split in the file to get smaller files (30Gb each) and then I load each smaller file to the database using GNU ...
5
votes
3answers
646 views

Tar piped to split piped to scp

So I'm trying to transfer a bunch of files via SCP. Some of these are too large to be stored on the recipient (Android phone, 4GB file size limit). The sender is almost out of space, so I can't ...
8
votes
4answers
31k views

Splitting string by the first occurrence of a delimiter

I have a string in the next format id;some text here with possible ; inside and want to split it to 2 strings by first occurrence of the ;. So, it should be: id and some text here with possible ; ...
7
votes
2answers
4k views

Can I automate tar's multi-volume-feature?

Ok, so I've just had a read through this page after a way to improve my current backup solution on my Debian server. Tar seems to be offering a quite nice multi-volume feature, although when I try it ...