6
votes
6answers
417 views

Split large file into chunks without splitting entry

I have a rather large .msg file formatted in the UIEE format. $ wc -l big_db.msg 8726593 big_db.msg Essentially, the file is made up of entries of various length that look something like this: ...
1
vote
2answers
181 views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
1
vote
1answer
197 views

Binary search for a failing line

git has bisect run to figure out which revision an error was introduced. I have a big file (100GB) and at least one of the lines is bad, but the program I have to check it with will not tell me which ...
4
votes
3answers
1k views

Split file into multiple small files, separate by the newline symbol

Is there a utility that split file by newline symbol? e.g if a file contains the following lines, aa bbb cccc If I want to split it to 3 files, the desired output would be: aa, bbb And cccc (in 3 ...
1
vote
1answer
755 views

Reading the contents of the file and splitting using ksh

We're using a ksh script for installing one product. I've another config file, I'd need to read this configuration file from my main script Content of the Configuration file: ...
7
votes
4answers
663 views

Creating a single output stream out of three other streams produced in parallel

I have three kinds of data that are in different formats; for each data type, there is a Python script that transforms it into a single unified format. This Python script is slow and CPU-bound (to a ...
4
votes
3answers
453 views

Is it possible to use split to make character chunks out of Chinese unicode bytes?

For a while, I've been dealing with Chinese unicode text. Of course, the usual rules apply. I can grep for characters the same way I'd do so for words. This is very useful to me. But there's one ...