The tag has no wiki summary.

learn more… | top users | synonyms

15
votes
4answers
19k views

What's the best way to join files again after splitting them?

If I have a large file and need to split it into 100 megabyte chunks I will do split -b 100m myImage.iso That usually give me something like xaa xab xac xad And to get them back together I have ...
13
votes
2answers
6k views

Break a large file into smaller pieces

How do I break a large, +4GB file into smaller files of about 500MB each. And how do I re-assemble them again to get the original file?
11
votes
8answers
944 views

Spreading stdin to parallel processes

I have a task that processes a list of files on stdin. The start-up time of the program is substantial, and the amount of time each file takes varies widely. I want to spawn a substantial number of ...
9
votes
4answers
37k views

Splitting string by the first occurrence of a delimiter

I have a string in the next format id;some text here with possible ; inside and want to split it to 2 strings by first occurrence of the ;. So, it should be: id and some text here with possible ; ...
8
votes
7answers
4k views

Splitting large directory tree into specified-size chunks?

I have a directory tree that I would like to back up to optical disks. Unfortunately, it exceeds the size of any one disk (it's about 60GB). I am looking for a script that would split this tree into ...
8
votes
1answer
387 views

K3b to split volume across media

How do I get K3b to split a volumes across several media? For example if I give it a folder that has 6 GB (not one file of 6 GB), it should create several discs of 4.4 GB automatically, instead of me ...
7
votes
2answers
4k views

Can I automate tar's multi-volume-feature?

Ok, so I've just had a read through this page after a way to improve my current backup solution on my Debian server. Tar seems to be offering a quite nice multi-volume feature, although when I try it ...
7
votes
4answers
787 views

Creating a single output stream out of three other streams produced in parallel

I have three kinds of data that are in different formats; for each data type, there is a Python script that transforms it into a single unified format. This Python script is slow and CPU-bound (to a ...
7
votes
3answers
1k views

Split: how to split into different percentages?

How can I split a text file into 70% and 30% using the split command ?
6
votes
4answers
2k views

Split PDF into documents with several pages each

There are several resources on the web explaining how one can split a PDF into many files with on page per file. But how can you split them into chunks of, say, five pages each? I have looked into ...
6
votes
4answers
308 views

How to split text between separator into multiple files?

I have a text file contaning the following: "random textA" "random random textB" The separator is " How can I split the containt into multiple file as follow using a bash command? File 1 : ...
6
votes
1answer
188 views

Zip Archive With Limited Number of Files

What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual ...
6
votes
6answers
766 views

Split large file into chunks without splitting entry

I have a rather large .msg file formatted in the UIEE format. $ wc -l big_db.msg 8726593 big_db.msg Essentially, the file is made up of entries of various length that look something like this: ...
5
votes
4answers
8k views

Split a file by line and have control over resulting files extension

There is a standard command for file splitting - split. For example, if I want to split a words file in several chunks of 10000 lines, I can use: split -dl 10000 words wrd and it would generate ...
5
votes
6answers
11k views

How to split such a string into array in bash

I have problem with output of a program. I need to launch a command in bash and take its ouput (string) and split it to add new lines in certain places. The string looks like this: battery.charge: ...
5
votes
3answers
727 views

Tar piped to split piped to scp

So I'm trying to transfer a bunch of files via SCP. Some of these are too large to be stored on the recipient (Android phone, 4GB file size limit). The sender is almost out of space, so I can't ...
5
votes
3answers
156 views

Robustly segmenting a csv file containing noisy data

Say I have a csv file with these columns: timestamp,cpu,memory An example of the shape of the actual data would be this chart: Having the data visualized in this way, it's easy for the human eye ...
5
votes
2answers
69 views

How to wait for execution of parallelized processes and stitch together the outputs?

Quite new to doing things on Unix, looking to make a script that does the following things in order: Take main .tsv file, split into X number of files with Y lines each Run each split file through a ...
5
votes
3answers
208 views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] ...
4
votes
5answers
1k views

How to split a file by using keyword boundaries

I have a vcf file that contains numerous vcards. When importing the vcf file to outlook it seems to only import the first vcard. Hence I want to split them up. Given that a vcard starts with ...
4
votes
2answers
1k views

What's a safe and portable way to split a string in shell programming?

When writing a shell script, I often want to split a string. Here's a very simple example: for dir in $(echo $PATH | tr : " "); do [[ -x "$dir"/"$1" ]] && echo $dir done This will ...
4
votes
3answers
2k views

Split file into multiple small files, separate by the newline symbol

Is there a utility that split file by newline symbol? e.g if a file contains the following lines, aa bbb cccc If I want to split it to 3 files, the desired output would be: aa, bbb And cccc (in 3 ...
4
votes
4answers
259 views

How do I break up a file like split to stdout for piping to a command?

I have a large .sql file full of SELECT statements that contain data I want to insert into my SQL Server database. I'm looking for how I could basically take the file's contents, 100 lines at a time, ...
4
votes
3answers
728 views

Using GNU Parallel With Split

I'm loading a pretty gigantic file to a postgresql database. To do this I first use split in the file to get smaller files (30Gb each) and then I load each smaller file to the database using GNU ...
4
votes
6answers
1k views

How can I split a large text file into chunks of 500 words or so?

I know there's the command split which can split a file into several chunks based on file size, and then there's the command wc which can count words. I just don't know how to use the two together.
4
votes
3answers
513 views

Is it possible to use split to make character chunks out of Chinese unicode bytes?

For a while, I've been dealing with Chinese unicode text. Of course, the usual rules apply. I can grep for characters the same way I'd do so for words. This is very useful to me. But there's one ...
3
votes
2answers
185 views

Script using “split” in a loop over large files

I'm attempting to get a bash script working, and running into terrible errors along the way. The aim of the script is to break several large text files, into multiple files with a line count read from ...
3
votes
2answers
167 views

split a file, pass each piece as a param to a script, run each script in parallel

I have a words.txt with 10000 words (one to a line). I have 5,000 documents. I want to see which documents contain which of those words (with a regex pattern around the word). I have a script.sh that ...
3
votes
3answers
1k views

Splitting a large binary file into sections determined by context patterns

I have a large (2GB) file that looks like this: ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ <binary data> ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ <binary data> ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ ...
3
votes
3answers
318 views

Split the lines of a text file into separate files

I have a text file that looks something like this: foo bar zip rar tar I need to use a bash script on OSX to make a new text file after every new line like this: cat text1.txt foo cat text2.txt ...
3
votes
1answer
62 views

Split input file into X parts of size total lines / X

Currently, my bash script splits by number of lines. However, I'd like to split a file into X pieces, each of those having total lines equal to the file length divided by X. The script is run as ...
3
votes
2answers
112 views

TAR: better to skip directory or use split

I've got a file.tar.gz that I create and backup to Amazon S3. It's gotten bigger than Amazon's limits and thus has not been backing up. This is my current command: tar -cpzf file.tar.gz directory/ ...
3
votes
0answers
70 views

gnu screen: automatic session on split

Inactive region appears after 'split' command in gnu screen now. You must go to the new region and type :screen. Only after that shell session appears in this region. Is it possible to make new ...
2
votes
2answers
92 views

What are the differences between split and rar?

I am used to raring files -- but I am looking for something faster. I see that there is a "split" command. How does this command compare to rar? What is are the differences between split and rar?
2
votes
2answers
119 views

How to split an image vertically using the comand line?

So, say I have a large 800x5000 image, how would I split that into 5 separate images with dimensions 800x1000 using the command line?
2
votes
1answer
935 views

How to split a ddrescue disk image and how to use it again?

I have a 500GB external HDD that I need to rescue the contents of. Unfortunately I only have two 400GB partitions to save the contents to. Can I split the disk image as: ~$ cd /mnt/part1/Recovery/ ...
2
votes
2answers
641 views

Is there a way to split a HTML file without losing all the formatting within each segment?

So... the split command allows me to split a file into pieces. The problem with that, though, is that only one of the files is going to contain the HTML headers, and only one of them is going to ...
2
votes
1answer
163 views

Emacs - finding the cursor in multiple windows

I've started splitting my emacs frame into two side-by-side windows, but often I'm unable to tell at a glance which window the cursor is currently in. I've bound the other-window command to a special ...
2
votes
0answers
15 views

screen & xterm: how to select text with the mouse in one only pane when window is divided vertically

Interestingly enough, this has also been a tmux issue for some: http://daemonforums.org/showthread.php?t=7311 But currently I am sticking to screen, and I'd like to know how I can make screen ...
2
votes
2answers
951 views

splitting the array and send the values to hashset in awk

I am getting var as argument from shell script to awk awk -v var = "blah1|blah2|blah3" -f awk_script.awk I need to put each blah's into a hashset, currently, I am doing it like this. I am getting ...
1
vote
4answers
263 views

How to split a line by multiple characters?

Output of w | head -1 can give 2 different results for my machine: 16:34:42 up 32 days, 3:02, 1 user, load average: 0.14, 0.11, 0.07 14:10:55 up 32 days, 39 min, 1 user, load average: 0.07, 0.13, ...
1
vote
4answers
153 views

How to split text file into records by first non-blank charecter using awk and print only records containing string

I have a text file with information separated in blocks. I would like to separate these blocks into records so that I can handle them separately. This is my text file (file.txt): Alarm Stats: ...
1
vote
3answers
647 views

Is it possible to split file and append the pieces with same extension in one liner?

I did it in two commands --- one for splitting, and one for moving. I can't seem to find my way around it such as to have it all done with one line. What I did was: split.exe --bytes=2300k -d ...
1
vote
2answers
663 views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
1
vote
1answer
220 views

Split is not available?

I've just tried to run a simple shell script on my Readynas server running Debian, and it appears that the split command is not available. This is a server which I can't fiddle too much with, so if ...
1
vote
2answers
58 views

Specifying height for vertical splits in vim

Usually I am editing two files, A and B, split horizontally. I set it up, so that one buffer would only occupy the first 2 lines at the top of the screen, and the rest is for the other buffer. ...
1
vote
1answer
58 views

What does `{\} {}` mean in `find` command?

Please explain this command in detail. I am using this command to find large files above 6MB and split in to 5MB chunks in the same folder. Then it removes the original files (larger than 6MB) in the ...
1
vote
1answer
97 views

Split very big xml file into little pieces with specific line number count

This is how I did to extract the first 100000 lines from my big xml file (2gb): head source.xml -n 100000 > part.xml How can I keep splitting them to 100000 line (or specific file size chunks) ...
1
vote
2answers
229 views

printing special characters in awk output

I have a large file that contains lot MQ messages with RFH2 header. Each message in the file is separated by a blank line. Now I need to split this large file into small files each containing single ...
1
vote
1answer
399 views

Can I split a avi file into 2 valid (watchable) files with only standard unix commands?

The question is self explanatory. I have a very big movie file, I want to split it in 2. How can I do this?