Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

3
votes
1answer
34 views

Split input file into X parts of size total lines / X

Currently, my bash script splits by number of lines. However, I'd like to split a file into X pieces, each of those having total lines equal to the file length divided by X. The script is run as ...
1
vote
1answer
52 views

What does `{\} {}` mean in `find` command?

Please explain this command in detail. I am using this command to find large files above 6MB and split in to 5MB chunks in the same folder. Then it removes the original files (larger than 6MB) in the ...
1
vote
1answer
35 views

Split very big xml file into little pieces with specific line number count

This is how I did to extract the first 100000 lines from my big xml file (2gb): head source.xml -n 100000 > part.xml How can I keep splitting them to 100000 line (or specific file size chunks) ...
6
votes
1answer
168 views

Zip Archive With Limited Number of Files

What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual ...
3
votes
3answers
174 views

Split the lines of a text file into separate files

I have a text file that looks something like this: foo bar zip rar tar I need to use a bash script on OSX to make a new text file after every new line like this: cat text1.txt foo cat text2.txt ...
5
votes
2answers
66 views

How to wait for execution of parallelized processes and stitch together the outputs?

Quite new to doing things on Unix, looking to make a script that does the following things in order: Take main .tsv file, split into X number of files with Y lines each Run each split file through a ...
0
votes
3answers
56 views

Split a file into two

I have a big file and need to split into two files. Suppose in the first file the 1000 lines should be selected and put into another file and delete those lines in the first file. I tried using split ...
1
vote
4answers
245 views

How to split a line by multiple characters?

Output of w | head -1 can give 2 different results for my machine: 16:34:42 up 32 days, 3:02, 1 user, load average: 0.14, 0.11, 0.07 14:10:55 up 32 days, 39 min, 1 user, load average: 0.07, 0.13, ...
1
vote
4answers
136 views

How to split text file into records by first non-blank charecter using awk and print only records containing string

I have a text file with information separated in blocks. I would like to separate these blocks into records so that I can handle them separately. This is my text file (file.txt): Alarm Stats: ...
-1
votes
3answers
47 views

Split a file into multiple based on selected lines [duplicate]

I have a file in which more than 50,000 lines are there. How can I split the file into two or more based on the selected lines? Suppose I want to split a file from line 10,000 to line 40,000.
1
vote
2answers
49 views

Specifying height for vertical splits in vim

Usually I am editing two files, A and B, split horizontally. I set it up, so that one buffer would only occupy the first 2 lines at the top of the screen, and the rest is for the other buffer. ...
0
votes
1answer
573 views

FFmpeg - Split video multiple parts

I read the following thread, and the solution works if I want to split a video into pieces every two minutes: Split video file into pieces with ffmpeg The problem is that I want to split a video into ...
3
votes
2answers
172 views

Script using “split” in a loop over large files

I'm attempting to get a bash script working, and running into terrible errors along the way. The aim of the script is to break several large text files, into multiple files with a line count read from ...
3
votes
2answers
132 views

split a file, pass each piece as a param to a script, run each script in parallel

I have a words.txt with 10000 words (one to a line). I have 5,000 documents. I want to see which documents contain which of those words (with a regex pattern around the word). I have a script.sh that ...
1
vote
0answers
45 views

How do I convert a TTA file and its CUE sheet into several FLAC files?

I have a sound file encoded in TTA format and a CUE sheet with its metadata. Which tools and commands do I need to convert it and split it into several FLAC files? I tried using ffmpeg to convert the ...
6
votes
6answers
700 views

Split large file into chunks without splitting entry

I have a rather large .msg file formatted in the UIEE format. $ wc -l big_db.msg 8726593 big_db.msg Essentially, the file is made up of entries of various length that look something like this: ...
1
vote
2answers
216 views

printing special characters in awk output

I have a large file that contains lot MQ messages with RFH2 header. Each message in the file is separated by a blank line. Now I need to split this large file into small files each containing single ...
5
votes
6answers
10k views

How to split such a string into array in bash

I have problem with output of a program. I need to launch a command in bash and take its ouput (string) and split it to add new lines in certain places. The string looks like this: battery.charge: ...
1
vote
2answers
498 views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
-1
votes
2answers
184 views

Split files and rename the files

I need to split a large file into 5 pieces and rename them. For Example, file Test needs to be split into Test1, Test2 & Test3. However, the split command splits as xaa,xab,xbc.
4
votes
4answers
216 views

How do I break up a file like split to stdout for piping to a command?

I have a large .sql file full of SELECT statements that contain data I want to insert into my SQL Server database. I'm looking for how I could basically take the file's contents, 100 lines at a time, ...
0
votes
2answers
639 views

Bash split a list of files

I have 200 files in a folder like: test_1_cfg.dat, test_2_cfg.dat,.... and so on. I need to include in a bash script the first 40 files as input and run some process, another script to read the next ...
3
votes
2answers
100 views

TAR: better to skip directory or use split

I've got a file.tar.gz that I create and backup to Amazon S3. It's gotten bigger than Amazon's limits and thus has not been backing up. This is my current command: tar -cpzf file.tar.gz directory/ ...
5
votes
3answers
197 views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] ...
1
vote
3answers
537 views

Is it possible to split file and append the pieces with same extension in one liner?

I did it in two commands --- one for splitting, and one for moving. I can't seem to find my way around it such as to have it all done with one line. What I did was: split.exe --bytes=2300k -d ...
1
vote
1answer
304 views

Can I split a avi file into 2 valid (watchable) files with only standard unix commands?

The question is self explanatory. I have a very big movie file, I want to split it in 2. How can I do this?
2
votes
2answers
92 views

What are the differences between split and rar?

I am used to raring files -- but I am looking for something faster. I see that there is a "split" command. How does this command compare to rar? What is are the differences between split and rar?
5
votes
3answers
151 views

Robustly segmenting a csv file containing noisy data

Say I have a csv file with these columns: timestamp,cpu,memory An example of the shape of the actual data would be this chart: Having the data visualized in this way, it's easy for the human eye ...
1
vote
1answer
282 views

Binary search for a failing line

git has bisect run to figure out which revision an error was introduced. I have a big file (100GB) and at least one of the lines is bad, but the program I have to check it with will not tell me which ...
2
votes
1answer
143 views

Emacs - finding the cursor in multiple windows

I've started splitting my emacs frame into two side-by-side windows, but often I'm unable to tell at a glance which window the cursor is currently in. I've bound the other-window command to a special ...
3
votes
3answers
1k views

Splitting a large binary file into sections determined by context patterns

I have a large (2GB) file that looks like this: ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ <binary data> ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ <binary data> ^%%-=-=-=-=-=-=-=-=-=-=-=-=-=-%%^ ...
4
votes
5answers
1k views

How to split a file by using keyword boundaries

I have a vcf file that contains numerous vcards. When importing the vcf file to outlook it seems to only import the first vcard. Hence I want to split them up. Given that a vcard starts with ...
4
votes
3answers
2k views

Split file into multiple small files, separate by the newline symbol

Is there a utility that split file by newline symbol? e.g if a file contains the following lines, aa bbb cccc If I want to split it to 3 files, the desired output would be: aa, bbb And cccc (in 3 ...
6
votes
4answers
2k views

Split PDF into documents with several pages each

There are several resources on the web explaining how one can split a PDF into many files with on page per file. But how can you split them into chunks of, say, five pages each? I have looked into ...
4
votes
6answers
1k views

How can I split a large text file into chunks of 500 words or so?

I know there's the command split which can split a file into several chunks based on file size, and then there's the command wc which can count words. I just don't know how to use the two together.
4
votes
3answers
677 views

Using GNU Parallel With Split

I'm loading a pretty gigantic file to a postgresql database. To do this I first use split in the file to get smaller files (30Gb each) and then I load each smaller file to the database using GNU ...
0
votes
2answers
1k views

Why screen is not splitting my GNOME terminal?

I am working in a remote Linux box using VNC.I have a single terminal with lots of tabs opened in my system. How can I split them using the screen utility? Does the screen utility work with already ...
4
votes
2answers
1k views

What's a safe and portable way to split a string in shell programming?

When writing a shell script, I often want to split a string. Here's a very simple example: for dir in $(echo $PATH | tr : " "); do [[ -x "$dir"/"$1" ]] && echo $dir done This will ...
5
votes
3answers
692 views

Tar piped to split piped to scp

So I'm trying to transfer a bunch of files via SCP. Some of these are too large to be stored on the recipient (Android phone, 4GB file size limit). The sender is almost out of space, so I can't ...
1
vote
1answer
890 views

Reading the contents of the file and splitting using ksh

We're using a ksh script for installing one product. I've another config file, I'd need to read this configuration file from my main script Content of the Configuration file: ...
2
votes
1answer
902 views

How to split a ddrescue disk image and how to use it again?

I have a 500GB external HDD that I need to rescue the contents of. Unfortunately I only have two 400GB partitions to save the contents to. Can I split the disk image as: ~$ cd /mnt/part1/Recovery/ ...
3
votes
0answers
70 views

gnu screen: automatic session on split

Inactive region appears after 'split' command in gnu screen now. You must go to the new region and type :screen. Only after that shell session appears in this region. Is it possible to make new ...
0
votes
2answers
615 views

Split large txt file into certain small file

I have a txt file which is formatted as 250,000 * 3600 (3600 rows and 250,000 column) I am going to split it into 3600 small txt which each file is 250,000 * 1. I understand the split can basically ...
8
votes
4answers
34k views

Splitting string by the first occurrence of a delimiter

I have a string in the next format id;some text here with possible ; inside and want to split it to 2 strings by first occurrence of the ;. So, it should be: id and some text here with possible ; ...
7
votes
4answers
749 views

Creating a single output stream out of three other streams produced in parallel

I have three kinds of data that are in different formats; for each data type, there is a Python script that transforms it into a single unified format. This Python script is slow and CPU-bound (to a ...
11
votes
8answers
922 views

Spreading stdin to parallel processes

I have a task that processes a list of files on stdin. The start-up time of the program is substantial, and the amount of time each file takes varies widely. I want to spawn a substantial number of ...
2
votes
2answers
919 views

splitting the array and send the values to hashset in awk

I am getting var as argument from shell script to awk awk -v var = "blah1|blah2|blah3" -f awk_script.awk I need to put each blah's into a hashset, currently, I am doing it like this. I am getting ...
5
votes
4answers
7k views

Split a file by line and have control over resulting files extension

There is a standard command for file splitting - split. For example, if I want to split a words file in several chunks of 10000 lines, I can use: split -dl 10000 words wrd and it would generate ...
0
votes
0answers
48 views

Backing up to DVD [duplicate]

Possible Duplicate: Splitting large directory tree into specified-size chunks? I have a partition that is about 14G and I need to back it up to DVD in such a way that I can use each disc to ...
4
votes
3answers
484 views

Is it possible to use split to make character chunks out of Chinese unicode bytes?

For a while, I've been dealing with Chinese unicode text. Of course, the usual rules apply. I can grep for characters the same way I'd do so for words. This is very useful to me. But there's one ...