Manipulation of text by programs, scripts, etc.
4
votes
2answers
210 views
AWK help for filling up rest of the columns with   in file
I'm trying to get AWK filling up following text file for "empty" columns".
Basic idea of finding longest line with AWK NF and checking which line has no value on that column. Then adding x times N/A ...
1
vote
1answer
102 views
Using join with two files fails on larger file sizes
I am having some issues with a script that is using join to join two files. Eaxmple input files contains lines like this:
Here are the input file and the output of the join command:
...
9
votes
11answers
3k views
How can I split a text file into multiple text files?
I have a text file called entry.txt that contains the following:
[ entry1 ]
1239 1240 1242 1391 1392 1394 1486 1487 1489 1600
1601 1603 1657 1658 1660 2075 2076 2078 2322 2323
2325 2740 2741 2743 ...
6
votes
7answers
255 views
Swapping an unlimited number of columns
I have a file with columns. See below for an example:
a b c ... z
1 2 3 ... 26
I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc..
z y x ...
3
votes
1answer
194 views
grep list of names and information from bigger file
I have two files: one with list of names (500 entries) and other having some more information for each entry in A.txt and extra entries too.
File A.txt (each line is starting with > (fasta format) ...
3
votes
3answers
186 views
How to make and apply (patch) one side diff?
I want to make a shell script that updates a config file.
The file present on the server has information, like various IP addresses.
The new file has more code, from new configs added to the system, ...
3
votes
2answers
104 views
Column mismatch and substituting
input.txt (tab-delimted)
TTTTOTTT00000000008 RTTTT899 5.00E-28
TTTTOTTT00000000046 RTTTWRR 3.00E-31
TTTTOTTT00000000051 2.00E-11
TTTTOTTT00000000051 7.00E-12
TTTTOTTT00000000054 ...
5
votes
3answers
350 views
Extracting lines by key from very large file
I have a 42M line text file. Th first nine characters of each line are a numeric key. What is the most efficient way to extract only the lines whose key exists in another list of about 1.5M keys? Both ...
4
votes
4answers
2k views
Compare two files: lines present in one, not in the other, by one column comparison
I need to compare 2 files. Column 1 is the same in both files. Column 2 is what I want to compare: I want all lines in file 1 that are not in file 2 when comparing column 2. Column 3 is different in ...
2
votes
2answers
80 views
reformat a text file to have all text on one line using CentOS 5.6
I have a 210-line block of HTML that I have stored in a HTML for debugging purposes.
The HTML, as you would expect, is formatted accordingly so is stored in the file with spaces and tabs. Is it ...
2
votes
1answer
1k views
Delete last character in a word but only if the character is there - in bash
How to remove last character only if it's there?
input:
OpenOffice.org/m
openOffice.org/ozm
Pers.
Pfg.
phil.
Prof.
resp.
Roonstr./m
roonstr./ozm
desired output:
OpenOffice.org
openOffice.org
...
2
votes
2answers
146 views
How to Wrap a file with Tags in Bash
I have an xml file named test.xml, without a root tag like this:
<cat></cat>
<name></name>
<id></id>
<cat></cat>
<name></name>
...
3
votes
4answers
231 views
Sort input file by the results of a regex
I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C
@property (nonatomic, strong) id <AlbumArtDelegate, ...
1
vote
3answers
101 views
Merge files using zipper method / late merge
I am searching for a nice method of merging two or more files line by line using the zipper method (also called late merge). Assuming we have three files, the result should look like this:
line1 ...
3
votes
1answer
1k views
How to do multiline search and replace text (delete) recursively
I want to go recursively through a directory and search for specific content in all files. The content should then be replaced if found. With a single line I would make something like this:
find . ...
1
vote
1answer
232 views
What's the most appropriate way of parsing values from this output?
I have some netstat output I'm scripting and want to parse. What's the most efficient way of parsing values from the output below? It's important to point out that I'm using the old /bin/sh shell ...
2
votes
4answers
2k views
awk-Printing column value without new line and adding comma
input.txt
EN1
EN2
EN3
EN4
EN5
output
EN1,EN2,EN3,EN4,EN5
I have tried awk.But it is not printing with comma
awk 'BEGIN { OFS = ","} { printf $1}' input.txt
I have GNU Awk 4.0.0 version
3
votes
2answers
1k views
AWK-Find maximum value in a row & print with header
I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...
5
votes
1answer
179 views
How to adjust numeric fields in a text file
Suppose I have the following <Tab> separated text file:
file name size owner
file1.txt 12.345 root
file2.txt 0.172222 user1
file3.txt 2.46e2 user2
file4.txt 12345 ...
6
votes
4answers
703 views
A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1
Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ?
I have my ...
6
votes
1answer
493 views
Find any lines exceeding a certain length
Is it possible to find any lines in a file that exceed 79 characters?
3
votes
1answer
383 views
How to do df only on root partition?
How can get df results only for / partition. The partition name/identification (/dev/sda2, /dev/cciss/c0d0p1) could vary on different computers.
1
vote
4answers
103 views
“Multipass” scripted modification of large file in-place (file-system level)?
I have just came to the problem of having to cut some lines from a large (gigabyte) sized file, and being aware of potential CPU hog trying to read it in memory, I wanted to edit it in-place ...
2
votes
1answer
188 views
text processing-perl
I am a biologist. I would like help of computer experts to help in text file editing.
Here is what the input file looks like:
##dsfsd2
##sdf-sdf sasg 5.6.3
gi34_ex Gen CDS 161 317 . ...
2
votes
2answers
255 views
remove only specific text occurrences from string using sed
I have a text file that contains many rows of this sort of thing:
/*[17:51:27][1 ms]*/ UPDATE `country` SET `region_id` = '4' WHERE `country_id` = '36';
Is there a way that I can use sed to remove ...
2
votes
1answer
195 views
Duplicate selected words with sed and replace it
I have to modify a document containing xml data; one modification I couldn't set up with sed.
I have to modify the following expressions from:
<Bild ...
4
votes
2answers
652 views
delete first line of file only if blank using sed
I have a solution for this in awk:
awk '{if (NR==1 && NF==0) next};1' somefile
but was unable to find one that worked in sed. E.g.,
sed -i.bak '/^$/{1,1d;}' somefile
ended up deleting ...
3
votes
3answers
319 views
Using CSV line as command parameters
I have a CSV file like:
Name,Age,Address
Daniel Dvorkin,28,Some Address St. 1234
(... N ...)
Foo Bar,90,Other Address Av. 3210
And I have a command that take this parameters:
./mycommand ...
2
votes
4answers
423 views
shorten long lines in a log file
I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
3
votes
2answers
169 views
Subtracting one text string from another leaving unique characters, perhaps using awk
Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg.
Input
ab a
cd d ...
2
votes
1answer
280 views
How can I use unix tools with Cyrillic text?
I recently started processing Cyrillic text, and it's been really difficult.
I couldn't get my Python scripts to work with it at all. And I tried.
PHP worked well, but I don't know PHP. I just ...
1
vote
2answers
184 views
How to split vCards lines
Related to How to join vCards lines, vCard does a weird kind of line splitting: If a line contains more than 75 characters, insert a "CR, LF, space" sequence. Thus the following line:
...
2
votes
1answer
214 views
How to join vCards lines
vCard uses a special way to split long lines: At 75 characters, insert a DOS newline and a space. Joining therefore means to replace all occurrences of the sequence "CR, LF, space" with an empty ...
1
vote
2answers
115 views
How do I create a bunch of files with different parameters in the body of each file?
So, say, I want to create a bunch of files named this way (where I have an original file "albedo10solar100" with text inside), where I vary albedo from 10 to 60 in steps of 10 and solar from 90 to 100 ...
2
votes
2answers
529 views
How to do a regex search in a UTF-16LE file while in a UTF-8 locale?
EDIT: Due to a comment Warren Young made, it made me realize that I was not clear on one quite relevant point. My search string is already in UTF-16LE order (not in Unicode Codepoint order, which is ...
3
votes
4answers
481 views
Strip spaces after single capital letters with sed
I am writing a bash script to automatically generate some other files, and I have to format some strings a certain way. Specifically, the last problem I'm having is formatting a string that has ...
3
votes
2answers
554 views
In emacs (or other editor) how to display the byte offset of the cursor?
The question indicates my preference to use emacs, but the overriding issue is that I want to be able to do a normal text search and somehow see/copy-paste the byte-offset of the matched text.
To ...
5
votes
3answers
1k views
How can sed output be formatted like printf's formatted printing?
Can sed replace text with a string formatted like printf's formatted printing?
The following sed command replaces a line starting with the current value of "$domain" with several values specified in ...
3
votes
2answers
354 views
How to count the number of lines in a UTF-16LE/CR-LF/BOM file?
The immediate thought is wc, but then the next not-so-immediate thought is... Is *nix's wc purely for *nix line endings \x0a?... It seems so.
I've semi-wangled my way around it, but I feel there ...
2
votes
1answer
116 views
Cal no space delimiter
I am an OpenBSD user and I am writing an awk script which automatically generates TeX course calendars for all courses that I teach. To obtain actual calendar out of the system I use Unix cal command. ...
4
votes
2answers
909 views
Collapsing a stream of multiple lines into one line
I have a file containing a a large number of lines, each of which contains a bunch of numbers which are separated by spaces. I process this data in a pipe in some way, and then I want to collapse the ...
4
votes
1answer
238 views
inexact text search
Is there any utility like grep or even uniq but for inexact search or I should write it myself?
I mean it will look at 90% (number may vary) matching, or smth like that.
For example I have file with ...
2
votes
2answers
156 views
How to make a list and count LaTeX keywords in a source file?
I am a beginner, trying to make lists (and eventually count and make statistics) the LaTeX keywords appearing in some documents.
Basically a LaTeX "valid" keyword can be identified by:
The fact ...
0
votes
1answer
82 views
Why `paste file1 file2 > file1` will replace content of file 1 with that of file2? [duplicate]
Possible Duplicate:
Why does sed act differently depending on the output file?
I wonder why this command will simply replace the content of file1 with the content of file2, instead of the ...
2
votes
2answers
1k views
Why does xargs strip quotes from input?
Why does xargs strip quotes from input text?
Here is a simplified example:
echo "/Place/='http://www.google.com'" | xargs echo
outputs
/Place/=http://www.google.com
Is there any way to ...
3
votes
3answers
130 views
Regarding separate a single file into multiple files according to line separation
Currently, I have a plain text file, A, such as
lowest priority
very high significance.
outstanding
very novel
In this file, every line contains a sentence. I want to separate this file into ...
3
votes
2answers
5k views
How do I delete the first n lines of an ascii file using shell commands?
I have multiple files that contain ascii text information in the first 5-10 lines, followed by well-tabulated matrix information. In a shell script, I want to remove these first few lines of text so ...
5
votes
2answers
108 views
Sort fields inline
I'm trying to sort within a line of input over an unknown number of fields:
Input:
ab bc
bc ab
cd ef bc
bc cd ef
cd bc ab
ef ab bc cd gh
Output:
ab bc
ab bc
bc cd ef
bc cd ef
ab cb cd
ab bc cd ...
4
votes
4answers
907 views
concatenate two files without adding a newline
If I nano two files, one of which reads 'this' without me entering a newline, and one of which reads 'is' without me entering a newline, I want to be able to then cat the two files together into ...
1
vote
1answer
75 views
What happens for the last line when using N command with sed
Here is my sed command:
echo -e "AB\nCD\nEF\nGH" | sed 'N; D;'
It prints:
GH
while sed processing input line by line, what happens with the last line? (When after that sed saw End-Of-File)? I ...
