Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

6
votes
4answers
5k views

Compare two files: lines present in one, not in the other, by one column comparison

I need to compare 2 files. Column 1 is the same in both files. Column 2 is what I want to compare: I want all lines in file 1 that are not in file 2 when comparing column 2. Column 3 is different in ...
2
votes
2answers
123 views

reformat a text file to have all text on one line using CentOS 5.6

I have a 210-line block of HTML that I have stored in a HTML for debugging purposes. The HTML, as you would expect, is formatted accordingly so is stored in the file with spaces and tabs. Is it ...
4
votes
1answer
6k views

Delete last character in a word but only if the character is there - in bash

How to remove last character only if it's there? input: OpenOffice.org/m openOffice.org/ozm Pers. Pfg. phil. Prof. resp. Roonstr./m roonstr./ozm desired output: OpenOffice.org openOffice.org ...
2
votes
2answers
489 views

How to Wrap a file with Tags in Bash

I have an xml file named test.xml, without a root tag like this: <cat></cat> <name></name> <id></id> <cat></cat> <name></name> ...
4
votes
4answers
1k views

Sort input file by the results of a regex

I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C @property (nonatomic, strong) id <AlbumArtDelegate, ...
4
votes
3answers
228 views

Merge files using zipper method / late merge

I am searching for a nice method of merging two or more files line by line using the zipper method (also called late merge). Assuming we have three files, the result should look like this: line1 ...
3
votes
1answer
3k views

How to do multiline search and replace text (delete) recursively

I want to go recursively through a directory and search for specific content in all files. The content should then be replaced if found. With a single line I would make something like this: find . ...
1
vote
1answer
485 views

What's the most appropriate way of parsing values from this output?

I have some netstat output I'm scripting and want to parse. What's the most efficient way of parsing values from the output below? It's important to point out that I'm using the old /bin/sh shell ...
2
votes
5answers
14k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version
3
votes
2answers
4k views

AWK-Find maximum value in a row & print with header

I need to read a file, find the maximum value in each row, and print the ID of the row, the column the maximum value came from (sno#), and the value from the associated lc# column. If the maximum ...
5
votes
1answer
312 views

How to adjust numeric fields in a text file

Suppose I have the following <Tab> separated text file: file name size owner file1.txt 12.345 root file2.txt 0.172222 user1 file3.txt 2.46e2 user2 file4.txt 12345 ...
26
votes
11answers
9k views

A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1

Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ? I have my ...
9
votes
1answer
10k views

Find any lines exceeding a certain length

Is it possible to find any lines in a file that exceed 79 characters?
3
votes
1answer
1k views

How to do df only on root partition?

How can get df results only for / partition. The partition name/identification (/dev/sda2, /dev/cciss/c0d0p1) could vary on different computers.
1
vote
4answers
138 views

“Multipass” scripted modification of large file in-place (file-system level)?

I have just came to the problem of having to cut some lines from a large (gigabyte) sized file, and being aware of potential CPU hog trying to read it in memory, I wanted to edit it in-place ...
2
votes
1answer
246 views

text processing-perl

I am a biologist. I would like help of computer experts to help in text file editing. Here is what the input file looks like: ##dsfsd2 ##sdf-sdf sasg 5.6.3 gi34_ex Gen CDS 161 317 . ...
3
votes
2answers
755 views

remove only specific text occurrences from string using sed

I have a text file that contains many rows of this sort of thing: /*[17:51:27][1 ms]*/ UPDATE `country` SET `region_id` = '4' WHERE `country_id` = '36'; Is there a way that I can use sed to remove ...
2
votes
1answer
696 views

Duplicate selected words with sed and replace it

I have to modify a document containing xml data; one modification I couldn't set up with sed. I have to modify the following expressions from: <Bild ...
4
votes
2answers
2k views

delete first line of file only if blank using sed

I have a solution for this in awk: awk '{if (NR==1 && NF==0) next};1' somefile but was unable to find one that worked in sed. E.g., sed -i.bak '/^$/{1,1d;}' somefile ended up deleting ...
4
votes
3answers
968 views

Using CSV line as command parameters

I have a CSV file like: Name,Age,Address Daniel Dvorkin,28,Some Address St. 1234 (... N ...) Foo Bar,90,Other Address Av. 3210 And I have a command that take this parameters: ./mycommand ...
4
votes
4answers
1k views

shorten long lines in a log file

I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
0
votes
2answers
7k views

Replacing one test string with another in particular column

I would like to replace "CC" with "C" and "AA" with A" in a particular column of Tab delimited file (using awk probably).
3
votes
2answers
399 views

Subtracting one text string from another leaving unique characters, perhaps using awk

Is there any way to subtract one column of data containing text from another column containing text and get third column containing unique charcters, for example using awk eg. Input ab a cd d ...
2
votes
1answer
686 views

How can I use unix tools with Cyrillic text?

I recently started processing Cyrillic text, and it's been really difficult. I couldn't get my Python scripts to work with it at all. And I tried. PHP worked well, but I don't know PHP. I just ...
1
vote
2answers
311 views

How to split vCards lines

Related to How to join vCards lines, vCard does a weird kind of line splitting: If a line contains more than 75 characters, insert a "CR, LF, space" sequence. Thus the following line: ...
2
votes
1answer
398 views

How to join vCards lines

vCard uses a special way to split long lines: At 75 characters, insert a DOS newline and a space. Joining therefore means to replace all occurrences of the sequence "CR, LF, space" with an empty ...
1
vote
2answers
208 views

How do I create a bunch of files with different parameters in the body of each file?

So, say, I want to create a bunch of files named this way (where I have an original file "albedo10solar100" with text inside), where I vary albedo from 10 to 60 in steps of 10 and solar from 90 to 100 ...
2
votes
2answers
2k views

How to do a regex search in a UTF-16LE file while in a UTF-8 locale?

EDIT: Due to a comment Warren Young made, it made me realize that I was not clear on one quite relevant point. My search string is already in UTF-16LE order (not in Unicode Codepoint order, which is ...
3
votes
4answers
994 views

Strip spaces after single capital letters with sed

I am writing a bash script to automatically generate some other files, and I have to format some strings a certain way. Specifically, the last problem I'm having is formatting a string that has ...
5
votes
3answers
4k views

How can sed output be formatted like printf's formatted printing?

Can sed replace text with a string formatted like printf's formatted printing? The following sed command replaces a line starting with the current value of "$domain" with several values specified in ...
4
votes
2answers
931 views

How to count the number of lines in a UTF-16LE/CR-LF/BOM file?

The immediate thought is wc, but then the next not-so-immediate thought is... Is *nix's wc purely for *nix line endings \x0a?... It seems so. I've semi-wangled my way around it, but I feel there ...
2
votes
1answer
191 views

Cal no space delimiter

I am an OpenBSD user and I am writing an awk script which automatically generates TeX course calendars for all courses that I teach. To obtain actual calendar out of the system I use Unix cal command. ...
5
votes
3answers
4k views

Collapsing a stream of multiple lines into one line

I have a file containing a a large number of lines, each of which contains a bunch of numbers which are separated by spaces. I process this data in a pipe in some way, and then I want to collapse the ...
6
votes
1answer
654 views

inexact text search

Is there any utility like grep or even uniq but for inexact search or I should write it myself? I mean it will look at 90% (number may vary) matching, or smth like that. For example I have file with ...
1
vote
2answers
632 views

Remove first instance of lines with duplicate field value

I have sha1 summed all the image files on my storage server and put the results in a text file in the form of: sha1sum filename I've sorted the file and removed all all unique sha1sum entries. So ...
2
votes
2answers
240 views

How to make a list and count LaTeX keywords in a source file?

I am a beginner, trying to make lists (and eventually count and make statistics) the LaTeX keywords appearing in some documents. Basically a LaTeX "valid" keyword can be identified by: The fact ...
0
votes
1answer
183 views

Why `paste file1 file2 > file1` will replace content of file 1 with that of file2? [duplicate]

Possible Duplicate: Why does sed act differently depending on the output file? I wonder why this command will simply replace the content of file1 with the content of file2, instead of the ...
4
votes
2answers
4k views

Why does xargs strip quotes from input?

Why does xargs strip quotes from input text? Here is a simplified example: echo "/Place/='http://www.google.com'" | xargs echo outputs /Place/=http://www.google.com Is there any way to ...
3
votes
3answers
475 views

Regarding separate a single file into multiple files according to line separation

Currently, I have a plain text file, A, such as lowest priority very high significance. outstanding very novel In this file, every line contains a sentence. I want to separate this file into ...
18
votes
2answers
43k views

How do I delete the first n lines of an ascii file using shell commands?

I have multiple files that contain ascii text information in the first 5-10 lines, followed by well-tabulated matrix information. In a shell script, I want to remove these first few lines of text so ...
7
votes
4answers
391 views

Sort fields inline

I'm trying to sort within a line of input over an unknown number of fields: Input: ab bc bc ab cd ef bc bc cd ef cd bc ab ef ab bc cd gh Output: ab bc ab bc bc cd ef bc cd ef ab cb cd ab bc cd ...
4
votes
4answers
4k views

concatenate two files without adding a newline

If I nano two files, one of which reads 'this' without me entering a newline, and one of which reads 'is' without me entering a newline, I want to be able to then cat the two files together into ...
1
vote
1answer
90 views

What happens for the last line when using N command with sed

Here is my sed command: echo -e "AB\nCD\nEF\nGH" | sed 'N; D;' It prints: GH while sed processing input line by line, what happens with the last line? (When after that sed saw End-Of-File)? I ...
2
votes
1answer
285 views

Delete repeated words between brackets inline

Our input looks something like 2012-04-17 [GBPGBP] 2012-04-13 [GBP GBP] 2012-04-13 [GBP] 2012-04-11 [GBPGBP] 2012-04-11 [GBP GBP] 2012-04-10 [GBPGBP] 2012-04-06 [GBP GBP GBP] 2012-04-17 ...
1
vote
1answer
790 views

Multiply certain numbers in a text -file by certain constant

I want an alternative solution to this jQuery hack, tasting a bit reinventing the wheel -- look I am sure I could do this with some one-liner using basic *ix tools. Look much easier and ...
1
vote
2answers
711 views

Is there any simple way to change one line in a lot of files?

I'm trying to use PHP CodeSniffer, and here's the result: ----------------------------------------------------------------- A TOTAL OF 3008 ERROR(S) AND 380 WARNING(S) WERE FOUND IN 46 FILE(S) ...
4
votes
3answers
726 views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...
2
votes
3answers
790 views

loop to paste specific files in different directories

I some directories that contain a similarly named file eg (*Sample_name*.base.coverage.txt). And I would like to paste all of the *base.coverage.txt files together. I have something written, but its ...
4
votes
1answer
350 views

command like wc but with tee behaviour

I want to backup a database using psql and COPY command. here is my script: psql "user=${USERNAME} host=${HOSTNAME} dbname=${DBNAME} password=${PASSWORD}" -c \ "COPY (SELECT * FROM tbl) ORDER BY id ...
2
votes
2answers
1k views

How can I delete lines that has an empty first column?

INPUT: CC2352345|m,safnlasndfmnyxdcvyxcvyxcv |klasjdf |3|lasdjflasdf| |2345567356 CC3543353|asdfasdffghntz |klasjdf |3|lsajdfl | ...