Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
6answers
125 views

Scripting to split a single CSV row into multiple

I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario: ...
1
vote
3answers
76 views

Filter csv file based on extended column values

I have the following csv file: ID,PDBID,FirstResidue,SecondResidue,ThirdResidue,FourthResidue,Pattern RZ_AUTO_1,4tov,1404,1405,1518,1519,CG/AA Canonical ribose-zipper ...
2
votes
2answers
39 views

how to print the closest column when looking for a specific strings

I have a file with around 12 000 lines. ...
0
votes
2answers
48 views

How to add a column if line contains string?

If I have a tab-delimited file containing the following lines: Sample1-Baboon 1 2 3 4 Sample2-Human 1 2 3 4 Sample3-Human 1 2 3 4 Sample4-Baboon 1 2 3 4 How can I add a new column ...
0
votes
1answer
30 views

Filter by column value and write to file

I have a table like this: BB1520_v1.0_0103 exon 3329 3601 BB1520_v1.0_0103 exon 3650 4040 BB1520_v1.0_0089 exon 1 862 BB1520_v1.0_0089 exon 1312 ...
1
vote
1answer
54 views

sed regex not working for hidden characters

I have a list that has for some reason got some non alphanumeric characters appended to the end. I would have thought it would be a simple case of running : sed 's/[^A-Za-z0-9]$//' To replace ...
0
votes
5answers
68 views

How to Read string from file and compare to each line of second file

I'm making the post OS installation script that will include the Hardening section . And in this hardening section it will read the Kernel parameters from file A and compare to file /etc/sysctl.conf ...
-1
votes
1answer
40 views

How to add a line number follow by a tab to a text file

I tried using nl -ba -s '\t' full_media > full_media2 But it really did not add the tab, but it added actually the "\t" text after the line number. The file im editing is a CSV file and here by ...
1
vote
1answer
148 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
-1
votes
1answer
58 views

Script to `grep` a file with variable [duplicate]

while read wholeline do echo ${wholeline} --outputs John Jones #grab all the lines that begin with these values #grep '^John Jones' workfile2 --works, prints contents of workfile2 where the ...
0
votes
2answers
55 views

How to run a script to get certain contents of a file

So say a file has the login data for a bunch of users: bobby activityname 2014-05-12(bobbymiller@gmail.com) bill activityname 2014-06-12(bobbymiller@gmail.com) bobby activityname ...
0
votes
0answers
50 views

Bash - Convert escape character colored text to BBCode

I want to convert a terminal escape-character colored text file to BBCode colored text. For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color ...
2
votes
4answers
72 views

Substitute all occurrences of pattern with first matched group

I have to go from this: text match$something_here and match$xxx blablabla text match$something_else_here and match$xxx blablabla ... to this: text match$something_here and match$something_here ...
2
votes
4answers
86 views

Tool for escaping all quotes in a string/file

I have a weird problem that I don't have a good way around. My company, because reasons, stores scripting code in a database table. Don't ask. Whenever I have to insert a new version of the code, I ...
5
votes
3answers
115 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
2answers
59 views

Multiple grep commands with sort

I have a text file containing a list of student grades: Andrew cooking A Andrew cooking B Andrew computing F Steven java C Steven python E Ben C++ D Ben python C Jamie java E Jamie C++ E Jamie C++ B ...
1
vote
2answers
43 views

Generate Sequence number for each line at the end .. modified t he post can someone help&suggest on this

I am new to linux I have got this query sed '/./=' abc.txt| sed '/./N; s/\n/, /' >> as.dat source file has 3 rows like below a b c When use the following command it gives me result like ...
7
votes
8answers
635 views

Using 'grep' to find lines that contain all of three specified characters in any order [duplicate]

We have a document which contains lines and we have to find if [a|b|c] appears at least once in every line no matter which order. For example: Input: abc bca cab hhfdhdhfabjfdjdjff acjfdjdfjdf ...
3
votes
6answers
163 views

redirect output to specific line number

I want to grep certain lines by keyword and redirect the output to specific line number of an existing file. Command grep "key" temp_file >> desired.txt What I need is that I can append the ...
0
votes
1answer
114 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
1
vote
1answer
37 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
0
votes
2answers
74 views

Using AWK to Print Output of Combination of Unspecified Number of Files

My objective is to move into a subdirectory, call it SUB, which has many text files with 5 columns each and always the same number of rows. I want one large 'sum' file that adds all columns from 2-5, ...
20
votes
2answers
1k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
1
vote
2answers
54 views

add a new line to a delimited file

So I have a file name test.txt inside that file I have about 20 lines of text that are delimited by pipe | Example: John|freshman|seatle|math|4|fulltime Bob|senior|Tacoma|biology|4|part-time I ...
3
votes
2answers
190 views

Find the line number which contains the pattern with custom delimiter

As in example I'm trying to get line numbers which contains the pattern. My pattern contains slashes so I wanted to add custom delimiter. This simple one works: sed -n '/file/=' temp.txt Using ...
0
votes
1answer
37 views

Compare two files [closed]

I want to compare two files in which the data is delimited by a pipe |. Both the files have the same results but only few fields might vary. File 1: A|B|C|D File 2: A|B|F|D I want to compare ...
2
votes
3answers
136 views

Writing a program for editing .txt data - Python or Unix?

I only have little experience in programming, and I'm currently working on improving my skills. Basically, I need to write a program, that can do some specific processes to some data in a .txt file. ...
1
vote
2answers
73 views

Copy the first 2 lines from a file and then remove the lines

I need to process a file using a shell script. The file arrives with the data as well as some embedded information releavant to how it is processed. The file is in the following format; ...
3
votes
5answers
214 views

How to replace only the third occurrence of the file in unix using only sed command?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
1
vote
3answers
56 views

Display distinct values of a list and number of occurences

I have a program that writes on the standard output a list of string values, one per line, and I would like to display in real time the list of distinct values along with the number of occurences for ...
3
votes
3answers
63 views

Unix: Add unique names after signs in text

I have a file looking like the following: abcdefghijklmnopqrst // abcdefghijklmnopqrst // abcdefghijklmnopqrst I use the following code to insert a line with // to the beginning of the document (as ...
3
votes
7answers
586 views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
1
vote
3answers
85 views

Replace date with current date

I have a file abc.fil whose content is in the format yyyymmdd.xyz.doc. I want to cat that file abc.fil such that I get the output as 20150127.xyz.doc. Can any one please help me to resolve this ...
2
votes
3answers
165 views

Replace multiline string in files

I have a number of files I want to update by replacing one multi-line string with another multi-line string. Something along the lines of: * Some text, * something else * another thing And I want ...
2
votes
4answers
63 views

Removing an empty line and the line above it

I have a large .fna file containing millions of DNA sequences. Due to some write errors after truncating my primers, I'm left with some sequence headers with no sequence following it. An example is ...
0
votes
1answer
72 views

Extracting info from blocks inside text file

I have a text file which has blocks like dir1/dir2/dir3/name_run_number1: line1_run_number1_part1 line2_run_number1_part2 line3_run_number1_part3... Each block is separated with a blank line and ...
0
votes
3answers
60 views

how to replace a character within in a multiple double quoted words within a line using sed?

I have text file including text format like this. 012345,"[ThinkPadT2/3Gband,Mac]",Lenovo,"iPhone3G,A1241" How do i replace comma inside words only within double quote by pipe(|) operator using ...
0
votes
2answers
37 views

Multiply a column using awk

How do I multiply a column by an integer (say 3) and replace the older value by new one, using awk? Input: Data 9390.900391 10573.089844 80.000000 200.000000 2.700000 Data ...
-1
votes
1answer
71 views

grep and print all pattern in one lane

I have a file with a list of pattern like this K00001 K00003 K00005 I want to grep and print my pattern in a tab delimited table (the original table is without blank line) like this: K00001 ...
0
votes
2answers
39 views

How do I download a PDF, then output a chunk of text?

My city water bill is emailed to me every month as a link to the current PDF hosted on their server. So I can easily download this PDF any time I want. How do I take that PDF and extract my account ...
1
vote
2answers
83 views

Formatting ifconfig using sed/awk

I'm using ifconfig on OpenSUSE. When I run ifconfig eth0 I get eth0 Link encap:Ethernet HWaddr CE:FD:75:DF:A5:6D inet addr:172.16.4.177 Bcast:172.16.5.255 Mask:255.255.254.0 inet6 ...
2
votes
5answers
177 views

Edit Nth occurence interactively in a search in vim

Given a text with multiple occurrences of the word foo, I would like to do an interactive search through the text, and upon the Nth occurrence of foo, edit it. I can only find the search and replace ...
1
vote
4answers
44 views

Two operations with sed on the same pattern

I use this following sed command for display names of files with a specific form: ls -1 *|sed 's/^\(.*\).png/\1\/\1,/g' If I have two files named BOH_Contour.png and BOV_Web.png, I obtain ...
1
vote
1answer
520 views

How to remove junk characters from the file generated by script command in linux [duplicate]

I am using script command to record everything from terminal. But when I am opening the generated file its having lots of junk character. Can anyone help me to remove these junk characters from file ...
3
votes
1answer
57 views

Show exact number of CRLF occurrences in file

grep -c $'\r\n' file returns the exact same number as grep -c $'\n' file, it shows the number of LF in the file instead of the number of CRLF. (The file definitely does not contain that number of ...
-4
votes
1answer
191 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
2
votes
2answers
331 views

How to remove all CRLF in file (not replace with LF)

I'd like to remove all carriage returns followed by line feeds (CRLF), such as \r\n in a file. How can I do that? I can't use dos2unix because that replaces CRLF with LF. And I can't use tr because ...
1
vote
2answers
38 views

Help with Script to Process txt file to CSV

I am trying/attempting to write a script to turn this: Mikrotik01#show unique routing entry | 192.168.255.232 | inc ID Thu June 10 15:21:32.808 CST Unique Entry: 192.168.255.21 Unique Entry: ...
2
votes
1answer
235 views

How do I print hyphens if a variable is empty?

I am collecting windows system information using wmic on Linux system. For that purpose, I make various wmic calls and their output is stored in some variables. After required data is collected, I ...
1
vote
5answers
90 views

Using sed to cut a string starting with n spaces and ending with n spaces

How to use sed to cut text starting with x spaces and ending with y spaces? For instance This is my string: kkk 111 fff aaabbb 5d98 ccc mmmppp 9369d and I want to get this output: ...