Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
2answers
76 views

Getting Group Names for List of Users

I have a text files with a list of users formatted like: name1 name2 name3 ... Calling the command groups gives the primary group name for that user. ex: bash$ groups name1 name1: ...
-3
votes
1answer
76 views

AWK sum every two rows

I want to sum two rows until end of file: 1 2 3 4 0 1 2 3 0 4 5 6 2 3 5 6 Output should be 1 3 5 7 2 7 10 12 I have tried the following, but it only works for the 1st row awk ...
19
votes
15answers
1k views

How to find line with least characters

I am writing a shell script, using any general UNIX commands. I have to retrieve the line that has the least characters (whitespace included). There can be up to around 20 lines. I know I can use ...
5
votes
6answers
361 views

sed command to swap characters

My input file layout is: mm/dd/yyyy,hh:mm,other fields I need to format it as: yyyy-mm-dd hh:mm:00,other fields sample input: 01/02/1998,09:30,0.4571,0.4613,0.4529,0.4592,6042175 ...
4
votes
4answers
303 views

Merge two files

I have this file1.txt deiauk 9 kespaul 8 luktol 7 titkur 6 and other file2.txt kespaul b deiauk a And I want to merge both files in one by first value, so my result should be deiauk 9 a ...
2
votes
3answers
258 views

How to sort data in the file using script?

I have a file logs.txt with numbers which repeat: 1 QWE 1 ASD 1 QWE 1 QWE 1 QWE 1 qwe 12 22 qwe 2 aaa 2 vcc 2 xxx 4 asa 44 qwe 4 gfd 4 bcx 5 6 kuy 7 76 lol 5 4 ggg 23 ...
2
votes
1answer
32 views

Trouble with horizontial merge of csv files under Cygwin

I have two csv files: test1.csv 1 2 3 4 test2.csv 6 7 8 9 I want to horizontally merge these two files. To do this I use paste -d , test1.csv test2.csv > paste.csv If I open this file in ...
2
votes
2answers
62 views

Awk: compare two files

I have a file1.txt USA Joe 123.123.123 Russia Marry 458.786.892 Canada Greg 151.844.165 Latvia Grace 125.895.688 and file2.txt 1 123.123.123 2 151.844.165 3 465.879.515 and I want to create a ...
4
votes
6answers
171 views

Extracting lines based on conditions

Each line in a comma-separated file has 5 fields. a,b,c,d,e f,g,c,i, j,k,c,m,n o,p,c,r,s t,u,c,w, x,y,z,aa,bb How can I extract the lines which have c in the 3rd field and their 5th field is NOT ...
0
votes
1answer
37 views

Using Perl to modify specified lines of an ASCII file

Background: (1) I have a data set of scientific notation: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.503832e-05, 1.433673e-05, 2.557402e-05, 3.081098e-05, 4.044465e-05, 2.480817e-05, 2.681778e-05, ...
0
votes
2answers
37 views

sed replace ;“ content with ';”

I have files that contain ;" and I need it replaced with ';" The following sed command does not work: sed -i "s/;"/';"/g" /root/bashy.sh How can I do this? The solution doesn't have to be sed.
-1
votes
1answer
45 views

How to duplicate lines in a text (xpm) file?

I'm having some pixmaps like this (36, to be exact): /* XPM */ static char * hide_active_xpm[] = { "12 14 3 1", " c None", ". c #EDEDED s active_color_2", "+ c #313739 s active_text_color_2", " ...
3
votes
2answers
133 views

how to print field between the double quote

I have the following file conf.txt: perl /home/site/gen.pl "GG - W C1 Test - sai1" "15072" && ssh 13.4.93.103 perl /home/site/gen.pl "GG - W C1 Test - sai2" "15072" && ssh ...
15
votes
10answers
2k views

How can I quickly sum all numbers in a file?

Each line contains text and numbers in one column. I need to calculate the sum of the numbers in each row. How can I do that? Thx example.log contains: time=31sec time=192sec time=18sec time=543sec ...
1
vote
3answers
76 views

Find the complement set of an array?

Following from this question, where I wish to extract 10 random lines from a file, I now wish to also have the remaining 90 lines as a separate file. Since the document has 100 lines, indexing from 1 ...
0
votes
1answer
37 views

Move a range of words to the end of the string

I have the following string: "5 S comment=whatever Unspecific text fruit=apple animal=mouse sky=blue" I need to move to end of the string, all that begin with comment to the word text. The start ...
3
votes
3answers
88 views

Specific column level substitution

If I have a file with columnar entries, is it possible to substitute specific entries from a specific columns only and save changes to the same file?? Example a file abc.txt: SR|FRUITS|COLOR| ...
0
votes
1answer
28 views

.txt File from Mac not converting properly

I have a .txt file from my Mac that when I send it over to my Raspberry Pi running Raspbian and open it in nano it converts weirdly. Example: Text file in Mac OSX: ...
2
votes
2answers
105 views

Delete blank columns

This must be a common problem, but I couldn't find a solution. I have several files which have different sets of columns which are completely blank (for all rows except header) , how can I delete ...
10
votes
5answers
615 views

Combining large amount of files

I have ±10,000 files (res.1 - res.10000) all consisting of one column, and an equal number of rows. What I want is, in essence, simple; merge all files column-wise in a new file final.res. I have ...
1
vote
3answers
64 views

Data extraction from text file

I have a text file in the following format:- $DATA1 SOURCE='HSPICE' VERSION='I-2013.12-SP2-1 32-BIT' .TITLE 'cmos inverter transfer characteristics' index t_r t_f ...
2
votes
4answers
102 views

Extract two values from the output of a command

I'm using the following command in a script to get synthetic wifi info: echo "$( iw dev wlp1s0 link | grep '^\s*SSID:\s' | sed -r 's/^\s*SSID:\s//' ) $( iw dev wlp1s0 link | grep ...
0
votes
4answers
106 views

Remove first 2 columns from first row of a file

I am looking for a command to remove first 2 columns from the first row of a file. The delimiter for the columns is a pipe. Number of columns vary. we just need to blindly remove first 2 columns. ...
0
votes
2answers
93 views

Print text between two patterns not containing a particular word

I wanted to print some text between two patterns which doesn't contain a particular word input text is HEADER asdf asd asd COW assd TAIL sdfsdfs HEADER asdf asd sdfsd DOG sdfsdfsdf TAIL sdfsdfs ...
-1
votes
3answers
56 views

make average row with the same pattern [duplicate]

I have a huge table (10 columns and 6000 rows) and I have to make an average of all the rows only if they have the same pattern in column 3 (ko). My table contains numbers and strings. Here's an ...
-1
votes
2answers
43 views

Removing unwanted lines from the file [closed]

I have eight file's which start with: SEALS-* Each file has a header and data: #Col_1\Col_2\Col_3\col_4 --- header \\\\\\\\\\ --- data If it is a null entry, I need to delete ...
2
votes
1answer
31 views

Adding consecutive characters to duplicate words according to order they appear

So I have files like this but for hundreds of different genes per file. >lcl|accession_cds_proteinaccession [gene=dnaA] [protein=description] [protein_id=accession] [location=location] ...
4
votes
3answers
317 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
5
votes
3answers
149 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
1
vote
2answers
60 views

How to do operation on individual matches in a full line

I have a Verilog line like this: if (i2_0&!(i2_1)) (posedge i0_0 => (o:1'b1))=(0, 0); How can I convert it to a line like: if (i2_0 == 1'b1 & i2_1 == 1'b0) (posedge i0_0 => (o:1'b1))=(0, 0); ...
1
vote
2answers
68 views

Extract some columns from grep output

I've run the command grep -a --binary-file=text "DLDM" /home/path/ldm.log-$1* | grep -a "$line" and the output is something like this :- /home/path/ldm.log2015:"unix","20150518 ...
4
votes
6answers
85 views

Join multiple files with two blank lines?

I'm trying to join Markdown files with two newlines between each file. I've tried using the following command: paste -sd '\n\n' file1.md file2.md This joins the files with a newline, but not two ...
0
votes
1answer
63 views

Delete customers domain extension only (in .csv file)

I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command: sed -i '/.br/d' customers.csv But that would also delete a ...
5
votes
2answers
192 views

How to search for text in a file ignoring newlines?

I'd like to search for text that may be split over several lines in a file. A grep that would ignore line breaks and return the matching span of lines. e.g. I would be searching for is an example ...
0
votes
4answers
122 views

Hexadecimal number sequence checker in Linux?

I have a list of hexadecimal number. I would like to check whether they are in sequence or not. That is, they should be consecutive numbers, in increasing order. In other words, there should be an ...
1
vote
3answers
163 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
0
votes
2answers
67 views

How to match and replace an attribute value inside a tag

I am trying to replace the value of following node using sed command. <KeyValue key="EMPTY_SEARCH_CRITERIA" type="String" value="CS_CODE" comment="Configured empty search criteria."/> I am ...
1
vote
2answers
54 views

How to show searched lines as well as number of occurrence of searched string?

I am working on a script which will show all the results and their count on the console based on a particular string . Example: Suppose I searched for a string called "ERROR", I got the below ...
1
vote
3answers
48 views

How to use sed or awk to obtain just a directory

3:ââsdb1 8:17 1 984M 0 part /media/(NAMEOFDRIVEHERE) I want to just have the /media/(NAMEOFDRIVEHERE) location so I can store it in a variable. I've tried some sed commands and have ...
2
votes
1answer
23 views

Perform one of many substitutions in many files at two specific locations

I have about a hundred thousand files, for each file I want to do the following: Between the fifth and sixth character in the file with the following ascii code: 0x1f, a string is present. Now I want ...
3
votes
2answers
88 views

Create a file of pairs from pairs

Say I have the following file: A 1 B 2 CC 33 And I want to create a file containing every combination of two from the previous file, like this: AA 11 AB 12 ACC 133 BA 21 BB 22 BCC 233 CCA 331 CCB ...
0
votes
2answers
19 views

joining a text file in groups of 3 entries

I have a text file like this: X0122 Yellow Car $1200 YK321 Black Car $400 BZ321132 Red Boat $2344 every line is separated by a \n I want to join every 3 elements in a line separated by ;. So, the ...
1
vote
1answer
59 views

Search and sorts error logs based on a particular string

How to search through all the errors.zip as well as from errors.log files filter by "Error" string and sort by date? What about totals of errors? requirement : here are 3 components are installed ...
3
votes
1answer
62 views

Counting number of entries in column 2 where column 1 is similar

I am trying to count the number of entries in column 2 for the same column1 and also number of times the first entry in column2 for similar column1 occur. For example my input TSV file is: C1 NC01 ...
1
vote
0answers
73 views

Grepping multiple instances of a string, with context, from a file

I have a MySQL dump file, which is mostly INSERT statements. I want to grep out instances of the string 'media' from a field in a particular table, with +/- ~10 characters of context. The string might ...
0
votes
2answers
60 views

compare two files and fetch data

I have two files: File A sample >> AT5G46880.1|PACid:19669936 # score bias c-Evalue i-Evalue hmmfrom hmm to alifrom ali to envfrom env to acc --- ------ ----- --------- ...
1
vote
3answers
76 views

Remove special pattern ^M from script which got appended after FTP from windows to Unix

I send a windows file using FTP to an unix system and got appended ^M wherever a new line was intended, and I just want to remove them. One method which I can opt is to run dos2unix command. Can ...
1
vote
5answers
101 views

How to remove a string of characters after and before a specific character?

I have a file with thousands of lines that start with: >Miriam132_38138 Otu32555|1 I need to remove 'Miriam*********' so that each lines begins with: >Otu32555|1 The first string of ...
1
vote
2answers
78 views

Remove string from file name to leave variable?

I have the following list of files: Dorn_Triatomine_A5201_sequence_1_unmappedforTdim_tdim.alleles.tsv Dorn_Triatomine_A5201_sequence_1_unmappedforTdim_tdim.matches.tsv ...
3
votes
4answers
253 views

Replacing new line character with a pipe and a new line character

I am using ksh. I have a file temp.txt with some pipe delimited data. one|two|three|four|five abc|def|pqr|lmn|xyz As clear from example, the record ends with a new line character after the data ...