Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

10
votes
10answers
1k views

How can I quickly sum all numbers in a file?

Each line contains text and numbers in one column. I need to calculate the sum of the numbers in each row. How can I do that? Thx example.log contains: time=31sec time=192sec time=18sec time=543sec ...
0
votes
1answer
21 views

Using Perl to modify specified lines of an ASCII file

Background: (1) I have a data set of scientific notation: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.503832e-05, 1.433673e-05, 2.557402e-05, 3.081098e-05, 4.044465e-05, 2.480817e-05, 2.681778e-05, ...
0
votes
2answers
29 views

sed replace ;“ content with ';”

I have files that contain ;" and I need it replaced with ';" The following sed command does not work: sed -i "s/;"/';"/g" /root/bashy.sh How can I do this? The solution doesn't have to be sed.
2
votes
2answers
103 views

how to print field between the double quote

I have the following file conf.txt: perl /home/site/gen.pl "GG - W C1 Test - sai1" "15072" && ssh 13.4.93.103 perl /home/site/gen.pl "GG - W C1 Test - sai2" "15072" && ssh ...
-1
votes
1answer
30 views

How to duplicate lines in a text (xpm) file?

I'm having some pixmaps like this (36, to be exact): /* XPM */ static char * hide_active_xpm[] = { "12 14 3 1", " c None", ". c #EDEDED s active_color_2", "+ c #313739 s active_text_color_2", " ...
9
votes
5answers
521 views

Combining large amount of files

I have ±10,000 files (res.1 - res.10000) all consisting of one column, and an equal number of rows. What I want is, in essence, simple; merge all files column-wise in a new file final.res. I have ...
1
vote
3answers
55 views

Find the complement set of an array?

Following from this question, where I wish to extract 10 random lines from a file, I now wish to also have the remaining 90 lines as a separate file. Since the document has 100 lines, indexing from 1 ...
2
votes
3answers
53 views

Specific column level substitution

If I have a file with columnar entries, is it possible to substitute specific entries from a specific columns only and save changes to the same file?? Example a file abc.txt: SR|FRUITS|COLOR| ...
12
votes
5answers
10k views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
0
votes
1answer
30 views

Move a range of words to the end of the string

I have the following string: "5 S comment=whatever Unspecific text fruit=apple animal=mouse sky=blue" I need to move to end of the string, all that begin with comment to the word text. The start ...
0
votes
1answer
26 views

.txt File from Mac not converting properly

I have a .txt file from my Mac that when I send it over to my Raspberry Pi running Raspbian and open it in nano it converts weirdly. Example: Text file in Mac OSX: ...
2
votes
2answers
68 views

Delete blank columns

This must be a common problem, but I couldn't find a solution. I have several files which have different sets of columns which are completely blank (for all rows except header) , how can I delete ...
8
votes
8answers
14k views

How to print all lines after a match up to the end of the file?

Input file1 is: dog 123 4335 cat 13123 23424 deer 2131 213132 bear 2313 21313 I give the match the pattern from in other file ( like dog 123 4335 from file2). I match the pattern of the line ...
0
votes
4answers
525 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
1
vote
2answers
801 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
1
vote
2answers
50 views

How to show searched lines as well as number of occurrence of searched string?

I am working on a script which will show all the results and their count on the console based on a particular string . Example: Suppose I searched for a string called "ERROR", I got the below ...
2
votes
4answers
80 views

Extract two values from the output of a command

I'm using the following command in a script to get synthetic wifi info: echo "$( iw dev wlp1s0 link | grep '^\s*SSID:\s' | sed -r 's/^\s*SSID:\s//' ) $( iw dev wlp1s0 link | grep ...
4
votes
3answers
124 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
1
vote
3answers
41 views

Data extraction from text file

I have a text file in the following format:- $DATA1 SOURCE='HSPICE' VERSION='I-2013.12-SP2-1 32-BIT' .TITLE 'cmos inverter transfer characteristics' index t_r t_f ...
0
votes
1answer
89 views

need help merging two Unix files

I have 2 pipe delimited files say file1 and file2. Where file1 has say 33 columns and file2 may have 34/35/36 columns. So let's not consider how many columns we have. What I want to do is compare the ...
1
vote
1answer
33 views

How to filter a basic csv by columns

How do I filter by third column of the following file list.txt [[ list of people ]] billy baxter - @baxter - b.baxter@woods.com james woods - @woods - j.woods@someother.com I currently have cat ...
0
votes
1answer
50 views

Delete customers domain extension only (in .csv file)

I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command: sed -i '/.br/d' customers.csv But that would also delete a ...
-1
votes
0answers
34 views

How to merge these three task in a single command ( grep | sed | echo)

I am trying something where I parse out from the tags in a file then add back the HTML. I see everyone suggest not to do such with regEx (grep). but here it is grep -m 1 -oP ...
20
votes
6answers
37k views

combine text files column-wise

I have two text files. The first one has content: Languages Recursively enumerable Regular while the second one has content: Minimal automaton Turing machine Finite I want to combine them into ...
8
votes
9answers
20k views

How to print only last column?

echo -e 'one two three\nfour five six\nseven eight nine' one two three four five six seven eight nine how can I do some "MAGIC" do get this output?: three six nine UPDATE: I don't need it in this ...
0
votes
4answers
52 views

Remove first 2 columns from first row of a file

I am looking for a command to remove first 2 columns from the first row of a file. The delimiter for the columns is a pipe. Number of columns vary. we just need to blindly remove first 2 columns. ...
2
votes
3answers
94 views

Remove duplicate entries from a CSV file

I've got a [csv] file with duplicate datum reprinted ie the same data printed twice. I've tried using sort's uniq by sort myfile.csv | uniq -u however there is no change in the myfile.csv, also I've ...
7
votes
5answers
673 views

Modify a file without creating another file

Is there a way to modify a file without writing the contents to another file, without sed and awk? For example: $ cat test.txt aaa aaa bbb ccc ddd Replacing using sed with -i option, sed -i ...
0
votes
2answers
63 views

Print text between two patterns not containing a particular word

I wanted to print some text between two patterns which doesn't contain a particular word input text is HEADER asdf asd asd COW assd TAIL sdfsdfs HEADER asdf asd sdfsd DOG sdfsdfsdf TAIL sdfsdfs ...
5
votes
2answers
147 views

Transpose a file, and replace missing values

I have Readings for Names, from Machines and sometimes these readings are replicated. Where a reading was not found, it is left as blank. Name Instrument Rep R1 R2 R3 N1 I1 1 1 2 3 N2 I1 1 1 3 4 ...
6
votes
2answers
3k views

awk: forcing a return status?

This is a followup to my earlier question. I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
2
votes
1answer
28 views

Adding consecutive characters to duplicate words according to order they appear

So I have files like this but for hundreds of different genes per file. >lcl|accession_cds_proteinaccession [gene=dnaA] [protein=description] [protein_id=accession] [location=location] ...
1
vote
2answers
55 views

How to do operation on individual matches in a full line

I have a Verilog line like this: if (i2_0&!(i2_1)) (posedge i0_0 => (o:1'b1))=(0, 0); How can I convert it to a line like: if (i2_0 == 1'b1 & i2_1 == 1'b0) (posedge i0_0 => (o:1'b1))=(0, 0); ...
-1
votes
3answers
48 views

make average row with the same pattern [duplicate]

I have a huge table (10 columns and 6000 rows) and I have to make an average of all the rows only if they have the same pattern in column 3 (ko). My table contains numbers and strings. Here's an ...
1
vote
3answers
143 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
5
votes
5answers
8k views

Concatenate multiple files with same header

I have multiple files with the same header and different vectors below that. I need to concatenate all of them but I want only the header of first file to be concatenated and I don't want other ...
5
votes
2answers
88 views

How to search for text in a file ignoring newlines?

I'd like to search for text that may be split over several lines in a file. A grep that would ignore line breaks and return the matching span of lines. e.g. I would be searching for is an example ...
-1
votes
2answers
34 views

Removing unwanted lines from the file [closed]

I have eight file's which start with: SEALS-* Each file has a header and data: #Col_1\Col_2\Col_3\col_4 --- header \\\\\\\\\\ --- data If it is a null entry, I need to delete ...
3
votes
3answers
246 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
1
vote
2answers
32 views

Extract some columns from grep output

I've run the command grep -a --binary-file=text "DLDM" /home/path/ldm.log-$1* | grep -a "$line" and the output is something like this :- /home/path/ldm.log2015:"unix","20150518 ...
10
votes
4answers
765 views

How to report “sed” in-place changes

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)? For instance, how can I change the command line ...
30
votes
8answers
6k views

Is there a way to modify a file in-place?

I have a fairly large file (35Gb), and I would like to filter this file in situ (i.e. I don't have enough disk space for another file), specifically I want to grep and ignore some patterns — is there ...
27
votes
6answers
12k views

How to obtain inverse behavior for `tail` and `head`?

Is there a way to head/tail a document and get the reverse output; because you don't know how many lines there are in a document? I.e. I just want to get everything but the first 2 lines of foo.txt ...
2
votes
4answers
36 views

Join multiple files with two blank lines?

I'm trying to join Markdown files with two newlines between each file. I've tried using the following command: paste -sd '\n\n' file1.md file2.md This joins the files with a newline, but not two ...
0
votes
4answers
65 views

Hexadecimal number sequence checker in Linux?

I have a list of hexadecimal number. I would like to check whether they are in sequence or not. That is, they should be consecutive numbers, in increasing order. In other words, there should be an ...
8
votes
3answers
3k views

How to delete commands in history matching a given string?

I need to delete all commands in my history matching a string. I've tried: $ history | grep searchstring | cut -d" " -f2 | history -d -bash: history: -d: option requires an argument $ history | grep ...
0
votes
2answers
37 views

How to match and replace an attribute value inside a tag

I am trying to replace the value of following node using sed command. <KeyValue key="EMPTY_SEARCH_CRITERIA" type="String" value="CS_CODE" comment="Configured empty search criteria."/> I am ...
3
votes
2answers
79 views

Create a file of pairs from pairs

Say I have the following file: A 1 B 2 CC 33 And I want to create a file containing every combination of two from the previous file, like this: AA 11 AB 12 ACC 133 BA 21 BB 22 BCC 233 CCA 331 CCB ...
1
vote
3answers
45 views

How to use sed or awk to obtain just a directory

3:ââsdb1 8:17 1 984M 0 part /media/(NAMEOFDRIVEHERE) I want to just have the /media/(NAMEOFDRIVEHERE) location so I can store it in a variable. I've tried some sed commands and have ...
2
votes
1answer
22 views

Perform one of many substitutions in many files at two specific locations

I have about a hundred thousand files, for each file I want to do the following: Between the fifth and sixth character in the file with the following ascii code: 0x1f, a string is present. Now I want ...