Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

-1
votes
0answers
17 views

using unix command, how to add column which as same name for example add salary which as same name? [duplicate]

name salary pooja 1000 yashu 2000 gagan 3000 pooja 2000 yash 3000 pooja 2000 yashu 2000 Answer should be pooja [sum of all salary], yashu [sum of all salary]... conti
4
votes
3answers
178 views

Extract multiple lines if match

I have a command that outputs information about all DIMM slots in blocks like the following: ID SIZE TYPE 44 105 SMB_TYPE_MEMDEVICE (type 17) (memory device) Manufacturer: NO DIMM Serial ...
0
votes
2answers
25 views

Occurence Count

184.13.80.102 - - [07/Jan/2019:00:33:10 -0500] "GET /?p=454 HTTP/1.1" 200 8822 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0" 184.13.80.102 - - [07/Jan/2019:20:16:...
-1
votes
4answers
52 views

SQL operation on csv file using bash or shell

This is my input file 0164318,001449,001452,001922 0164318,001456,001457,001922 0842179,002115,002118,001485 0846354,001512,001513,001590 0841422,001221,001224,001860 0841422,001227,...
-1
votes
0answers
18 views

Parse a txt file into different files based upon tags [duplicate]

I need not parse below file into three files starting with tag [] until next tag is found in file. File is below: [AAA] SERIALNUMBER=2E:00:11:5A:FD:BA:16 SERIALNUMBER=2E:00:11:B1:30:CC:AF ...
0
votes
0answers
23 views

How can I regex match-and-replace _all_ on-screen text?

I need a way to replace an on-screen pattern with another string, regardless of what places that text on the screen. Maybe I can live without user input being replaced, but anything that comes from ...
-2
votes
2answers
28 views

Fetch line which contains only some specific symbol and number

I Need only those records from below file which contains Symbol like "+" and number and characters. 285627 AA283244 278178## 295456+ asdfasdf{ asdfasdfasdf I tried below code but it gives all ...
0
votes
2answers
32 views

How to print a new line if a pattern from a column matches in 2 lines?

I have the following file: transcr_15824 - YML042W - transcr_18545 - YCR105W + transcr_16934 + YDR032C + transcr_15237 + YDL008W - I do need something that looks like this: transcr_15824 - YML042W -...
0
votes
4answers
38 views

How can I remove repeated lines taking into consideration the text from just a column?

I have the following file: ICR1 + ICR1+1+3199 + ICR1+2526+2828 + IRT1 + IRT1+1+1489 + IRT1+713+937 + LSR1 - LSR1+1+1175 - LSR1+366+638 - NME1 + NME1+1+340 + NME1+2+118 + PWR1 - PWR1+1+941 - PWR1+724+...
0
votes
4answers
63 views

How remove duplicate names and print array after unique names

How to collapse KO categories with the same name and print gene names that were assigned to each category in array, as in example below. I have this: K00002 gene_65472 K00002 gene_212051 K00002 ...
0
votes
5answers
78 views

How to separate each line of text by comma and put them in an array?

I have a text file and I want to create an array out of it like this: array=["line1","line2",...]. Here are the first lines of my text: Hamlet William Shakespeare Edited Barbara B Mowat Paul ...
0
votes
2answers
31 views

Find all lines containing only the same character and replace with the same amount of another character

My goal is to convert a NEWS format (the output of appstream-util appdata-to-news) to a markdown format (to use on GitHub/GitLab). A good approximation for me would be to just go from this: Version ...
0
votes
4answers
72 views

How to compare the content inside a txt file?

I have a txt file which named as output.txt: output.txt 2128,4.4 2128,5.5 I wish to have a compare script that first line first number before comma is equal to second line first number, then it ...
0
votes
1answer
20 views

Reading multiple files and operating on stored Arrays

I have 2 text files "${LinkP}" and "${QuestionP}. I want to read these files and store each complete line in the respective array, IFS=$'\r\n' GLOBIGNORE='*' command eval "LinkA=($(cat "${LinkP}"))" ...
0
votes
3answers
40 views

Oneliner for outputting to file only if string is matched (bash)?

I am making a script where I curl a URL and output to a file like this: curl http://example.com/$1 > $1 Is there any way to make it dismiss anything that doesn't include, say <head>? I ...
2
votes
2answers
31 views

CSV to html with dynamic color coding

I have written the shell script below to convert csv to HTML with color coding for a single file and it's working properly. The condition for color coding is different for different reports (csv files)...
0
votes
3answers
52 views

awk + count strings in file

we have huge file like this this is partial list in the file Topic: Ho_HTR_bvt Partition: 31 Leader: 1007 Replicas: 1007,1008,1009 Isr: 1009,1007,1008 Topic: Ho_HTR_bvt Partition:...
0
votes
1answer
36 views

Compare columns in 2 files and put information according to file 1 [duplicate]

I have 2 files, file 1 and file 2 and I want to compare column 1 in file to column 1 in file 2 and have additional column generated with column 2 in file 3 File 1: 9001 Whitby On 9002 ...
3
votes
2answers
39 views

Join two files based on a common field

I'm trying to join two files together based on their common field, and output it to a file, using the join command. File 1: C01:Nancy:Jones:njones@abc.com C02:Barbara:Madison:bmadison@bcd.com C03:...
3
votes
3answers
417 views

grep specific number from lines 5 but not 25 or 52 and so on

I need to find a user for a userid. The return from the authentication system is as follows. Now with a bash script I need to extract the line with the exact number. Not 5 and 25 .. ------------ ID ...
1
vote
4answers
139 views

Awk to print lines in file with multiple delimiters

I have a file that looks like: chr1 1197592 . C A . . DP=67;ECNT=1;NLOD=8.12 GT:AD:AF:F1R2 0/1:37,2:0.063:13,0 chr1 1355707 . G T . . DP=69;ECNT=1;NLOD=4.51 GT:AD:AF:...
2
votes
2answers
71 views

How to replace a single similar line only AFTER a match [closed]

I am trying to search for a specific item [7_addons] and ONLY replace the NEXT entry that appears from enabled=0 to enabled=1. in this example: sed -i -e 's#enabled=0#enabled=1#g' /etc/file.txt .....
4
votes
1answer
105 views

Sort multiple delimited file lexicographically by one column, numerically by another

I wish to sort the TSV file below (called min_ex) by the first column lexicographically and by the second column numerically. A X, N 2.2 A, N 5.7 A, A 5.8 A, N 2.1 A, T 0.2 B G, M 2....
2
votes
4answers
234 views

Parse all strings of specific length?

I've exported my email archive of 10 years which is very large. I want to parse all the text for any string that is 64 characters long in search of a bitcoin private key. How can I parse strings of ...
1
vote
1answer
45 views

Using sed to replace the hexadecimal code for URL and to insert new SVG codes after SVG tag in all SVG files

I would like to ... replace the hexadecimal code for URL (id) in all SVG files insert/add the new lines of SVG codes after <svg ...>, using Regex. Input: <svg xmlns="http://www.w3.org/...
0
votes
3answers
48 views

Find the min and max value of column 3 and print whole row

I have huuuge sets of data consisting of 3 columns. I'd like to be able to find max and min values in the third column (there is mix of zeros and then negative values, so I want to find max negative ...
6
votes
8answers
372 views

Count lines containing word

I have a file with multiple lines. I want to know, for each word that appears in the total file, how many lines contain that word, for example: 0 hello world the man is world 1 this is the world 2 a ...
7
votes
3answers
395 views

Awk - replace one character only in a certain column

I have a file like this: 2018.01.02;1.5;comment 1 2018.01.04;2.75;comment 2 2018.01.07;5.25;comment 4 2018.01.09;1.25;comment 7 I want to replace all dots . in the second column with a comma , as I ...
1
vote
5answers
45 views

Delete empty line if there is only one between two lines, if there are two empty lines remove only one

I am getting db2 output as of below. this is testing 1 this is testing 2 this is testing 4 The db2 provides output as, output for a query and a empty row and if there is no ...
0
votes
1answer
75 views

how can I merge two text files together

I have a text file like this sp|QBWMM1-2|PDCI_Mouse(741-770) 8864=mil (25/2.20/2.50) sp|Pm345|Hisf_Mouse(613-640) 776=mil (25/2.20/2.50) sp|P0065-2|Hila_Mouse(344-393)6543=mil (25/2.20/2.50) sp|...
2
votes
1answer
29 views

Awk - convert time value to decimal value

How can I use awk to convert a time value to a decimal value. I have been using this command for the other way round (-> from): awk '{printf "%d:%02d", ($1 * 60 / 60), ($1 * 60 % 60)}' <<< ...
4
votes
6answers
107 views

How to edit the entire file after match a grep pattern?

To simplify, I want to edit the whole file after matched pattern, example of a file: $ cat file ip=x.x.x.a mask=255.0.0.0 host=a ip=x.x.x.b mask=255.0.0.0 host=b ip=x.x.x.c mask=255.0.0.0 host=c ip=x....
-1
votes
0answers
36 views

Shell script required for collection/compiling of data

I wanted to collect some stats in a specific format (of previous day) from my server log files using a cronjob everyday. Below is the sample list of log files:- -rw-rw-r-- 1 lpt tkg 2.0M **Jan 3**...
1
vote
3answers
39 views

Find empty value for any name in file

I have a file in the below format: Name=Dan Age=45 Name=Alex Age=35 Cellphone=12345678901 Name=John Age= Cellphone=12345678901 This file has got 1000's of data items in above format, Name=Value. If ...
3
votes
2answers
77 views

Remove even numbered lines of text file [duplicate]

I have a text file on a Linux system, and would like to remove every second line, which is "even" numbered. Example: I would like to remove the second line of the file, the fourth, sixth, eighth line, ...
2
votes
1answer
23 views

How to change data file format from Gnuplot to Tikz errorbars

I would like to change the format of Gnuplot errorbar data to LaTeX/Tikz format. For example, the Gnuplot format data is: 5 74.92725 74.7028332590511 75.1516667409489 112.5 112.55897 112....
1
vote
2answers
49 views

How to start a new column after every nth row?

I have a file (comma separated) on a Linux system with 3 columns. I want to start new column after every 4th row. Input: col1,col2,col3 1,disease1,high 1,disease2,low 1,disease3,high col1,col2,...
2
votes
4answers
108 views

Replace AWORD or BWORD with CWORD in sed

So I have a list of random websites of the following kind: rapido21655bonk.a.sweetpotato.net rapido26230bonk.a.sourpotato.net rapido29926bonk.b.sourpotato.net rapido29926bonk.b....
0
votes
0answers
13 views

Interaction of sed and for loop [duplicate]

for i in {1..25}; do cat file$((i)) | sed 's/.*/$((i))/'; done doesn't print the current value of i but just $((i)). How can I make it print the value of i instead?
0
votes
1answer
37 views

Unix command to search with a string of 6 numbers for any 3 matching numbers per line

Does anyone know of a UNIX command or series of commands to list lines that contains 3 matching numbers to a search with 6 numbers? For example, searching with 38 39 40 41 42 43 I would like to see ...
0
votes
0answers
24 views

Creating Plain Text MAN Pages: Removing overstrike characters [duplicate]

I want to create plain text files of certain man pages such as the "ls" command. Simply performing a man ls > ls.txt, and then running file ls.txt will inform me that this is full of 'ASCII text, ...
1
vote
1answer
56 views

Remove all lines that contain $PWD

I have a file that contains paths - looks like this: /Users/a/Desktop /Users/a/Documents /Users/a/Documents/Work What would be the easiest way to remove all lines that contain the current directory (...
2
votes
4answers
39 views

Linux: how to extract rows of certain IDs and the IDs are writtein in another files?

I have a csv file of IDs. 1 3 8 I have another file of information, not only for the IDs, but also others. If I want to extract the information only of the IDs, how to do it? 1 96 283 4 2 87 918 6 ...
2
votes
4answers
49 views

repeat a list of data in the same file/column

I have a list of data in a file such as -0.2947990000000000 0.1722064000000000 0.1722064000000000 0.1722064000000000 0.0315420000000000 and I would like to repeat this sequence of data 35 times ...
3
votes
5answers
270 views

copy certain spaces from a file

I have a file which looks like this 18DMA H 9996 0.886 5.687 5.320 18DMA H 9997 1.019 5.764 5.247 18DMA Np 9998 0.947 5.584 5.151 18DMA H 9999 1.033 ...
0
votes
1answer
55 views

sed or awk for formatting

I have one text file which consists of group of contact numbers with names like below. sam=12345,john=98765,bruce=67890,jack=54321,nick=23456,noah=67894,vision=65432,thor=98768,tony=89769,cap=...
3
votes
2answers
93 views

print the first line of those files in current directory which are having last line with a pattern

I have multiple log files in a directory and want to print first line of those files which are having pattern vanished on last line. Could it be done with awk for *.logs having also printed name of ...
0
votes
1answer
44 views

how can I merge two txt files by one similar string [closed]

I have a data like this as an example sp|O15304|SIVA_HUMAN ...
0
votes
1answer
53 views

how can I pass over a string on several txt files

I have a data like this but much bigger. So I have a df1.txt as follow sp|O15304|SIVA_HUMAN ...
0
votes
2answers
24 views

Using awk how to compare 2 fields and print something out

Let say I have file1 with the following content: 1,2,3,4,12,6 1,2,3,5,333,6 1,2,3,12,5,6 1,2,3,44,101,6 While file2 has the following content: 22,4,111,1,6,7,12 22,4,56,55,6,7,12 22,4,12,12,6,7,12 ...