Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

0
votes
0answers
3 views

Find consecutive integers from list of numbers

With a file of integers separated by newlines, I would like to search for consecutive integers, then list the amount of integers that are consecutive per uninterrupted sequence, as well as the ...
2
votes
1answer
23 views

Count number of repeated multi-character strings in file in bash

I am trying to search for consecutive, repeated instances of numbers in a file, with each number on a different line. It's easiest to show what I mean in an example. For example, my input would be: ...
2
votes
1answer
26 views

Replace text using sed and also add single quote to it

I answered this question, but I am stuck on the part where I have to add a single quote also to it. Like original file is: $$BATCHCTRL=TEST-012017 $$STATE=CA AZ $$FROM_DATE=01/10/2017 $$TO_DATE=01/...
-1
votes
2answers
43 views

Find String and Replace with SUBSTRING

I have the file like this. ex.(test.txt) $$BATCHCTRL=TEST-012017 $$STATE=CA AZ $$FROM_DATE=01/10/2017 $$TO_DATE=01/30/2017 All I need to do is replace this $$STATE=CA AZ with first TWO bytes of this ...
1
vote
1answer
29 views

unix shell script : Merging data(lines) of 2 different text files

files have many entries but for sample posting only 2 entries each. I want to merge file1 first line content with file 2's 1st line content and so on.example is as below. file1.txt Start Date: 2019-...
0
votes
2answers
36 views

Enter IP at the end of a specific line

I have the following file.txt that follows the same pattern and I want to modify it where this file is by adding an ip: # gfhfhgfh gfhfghgfhgfhgfh MACs # access USER CONSOLA *,!10.249.247.3,!10.249....
6
votes
5answers
347 views

How to sort by odd lines then remove repeated values?

I have the following type of file: transcr_25793 + YAL039C - transcr_25793 + YAL037C-B - transcr_20649 + YBL100C - transcr_7135 + YBL029C-A - transcr_11317 + YBL067C - transcr_25793 + YAL038W + ...
-1
votes
0answers
20 views

using unix command, how to add column which as same name for example add salary which as same name? [duplicate]

name salary pooja 1000 yashu 2000 gagan 3000 pooja 2000 yash 3000 pooja 2000 yashu 2000 Answer should be pooja [sum of all salary], yashu [sum of all salary]... conti
4
votes
3answers
194 views

Extract multiple lines if match

I have a command that outputs information about all DIMM slots in blocks like the following: ID SIZE TYPE 44 105 SMB_TYPE_MEMDEVICE (type 17) (memory device) Manufacturer: NO DIMM Serial ...
0
votes
2answers
27 views

Occurence Count

184.13.80.102 - - [07/Jan/2019:00:33:10 -0500] "GET /?p=454 HTTP/1.1" 200 8822 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0" 184.13.80.102 - - [07/Jan/2019:20:16:...
-1
votes
4answers
58 views

SQL operation on csv file using bash or shell

This is my input file 0164318,001449,001452,001922 0164318,001456,001457,001922 0842179,002115,002118,001485 0846354,001512,001513,001590 0841422,001221,001224,001860 0841422,001227,...
-1
votes
0answers
18 views

Parse a txt file into different files based upon tags [duplicate]

I need not parse below file into three files starting with tag [] until next tag is found in file. File is below: [AAA] SERIALNUMBER=2E:00:11:5A:FD:BA:16 SERIALNUMBER=2E:00:11:B1:30:CC:AF ...
0
votes
0answers
25 views

How can I regex match-and-replace _all_ on-screen text?

I need a way to replace an on-screen pattern with another string, regardless of what places that text on the screen. Maybe I can live without user input being replaced, but anything that comes from ...
-2
votes
2answers
29 views

Fetch line which contains only some specific symbol and number

I Need only those records from below file which contains Symbol like "+" and number and characters. 285627 AA283244 278178## 295456+ asdfasdf{ asdfasdfasdf I tried below code but it gives all ...
0
votes
2answers
35 views

How to print a new line if a pattern from a column matches in 2 lines?

I have the following file: transcr_15824 - YML042W - transcr_18545 - YCR105W + transcr_16934 + YDR032C + transcr_15237 + YDL008W - I do need something that looks like this: transcr_15824 - YML042W -...
0
votes
4answers
41 views

How can I remove repeated lines taking into consideration the text from just a column?

I have the following file: ICR1 + ICR1+1+3199 + ICR1+2526+2828 + IRT1 + IRT1+1+1489 + IRT1+713+937 + LSR1 - LSR1+1+1175 - LSR1+366+638 - NME1 + NME1+1+340 + NME1+2+118 + PWR1 - PWR1+1+941 - PWR1+724+...
0
votes
4answers
70 views

How remove duplicate names and print array after unique names

How to collapse KO categories with the same name and print gene names that were assigned to each category in array, as in example below. I have this: K00002 gene_65472 K00002 gene_212051 K00002 ...
0
votes
5answers
78 views

How to separate each line of text by comma and put them in an array?

I have a text file and I want to create an array out of it like this: array=["line1","line2",...]. Here are the first lines of my text: Hamlet William Shakespeare Edited Barbara B Mowat Paul ...
0
votes
2answers
32 views

Find all lines containing only the same character and replace with the same amount of another character

My goal is to convert a NEWS format (the output of appstream-util appdata-to-news) to a markdown format (to use on GitHub/GitLab). A good approximation for me would be to just go from this: Version ...
0
votes
4answers
75 views

How to compare the content inside a txt file?

I have a txt file which named as output.txt: output.txt 2128,4.4 2128,5.5 I wish to have a compare script that first line first number before comma is equal to second line first number, then it ...
0
votes
1answer
20 views

Reading multiple files and operating on stored Arrays

I have 2 text files "${LinkP}" and "${QuestionP}. I want to read these files and store each complete line in the respective array, IFS=$'\r\n' GLOBIGNORE='*' command eval "LinkA=($(cat "${LinkP}"))" ...
0
votes
3answers
40 views

Oneliner for outputting to file only if string is matched (bash)?

I am making a script where I curl a URL and output to a file like this: curl http://example.com/$1 > $1 Is there any way to make it dismiss anything that doesn't include, say <head>? I ...
2
votes
2answers
31 views

CSV to html with dynamic color coding

I have written the shell script below to convert csv to HTML with color coding for a single file and it's working properly. The condition for color coding is different for different reports (csv files)...
0
votes
3answers
52 views

awk + count strings in file

we have huge file like this this is partial list in the file Topic: Ho_HTR_bvt Partition: 31 Leader: 1007 Replicas: 1007,1008,1009 Isr: 1009,1007,1008 Topic: Ho_HTR_bvt Partition:...
0
votes
1answer
36 views

Compare columns in 2 files and put information according to file 1 [duplicate]

I have 2 files, file 1 and file 2 and I want to compare column 1 in file to column 1 in file 2 and have additional column generated with column 2 in file 3 File 1: 9001 Whitby On 9002 ...
4
votes
2answers
39 views

Join two files based on a common field

I'm trying to join two files together based on their common field, and output it to a file, using the join command. File 1: C01:Nancy:Jones:njones@abc.com C02:Barbara:Madison:bmadison@bcd.com C03:...
3
votes
3answers
417 views

grep specific number from lines 5 but not 25 or 52 and so on

I need to find a user for a userid. The return from the authentication system is as follows. Now with a bash script I need to extract the line with the exact number. Not 5 and 25 .. ------------ ID ...
1
vote
4answers
141 views

Awk to print lines in file with multiple delimiters

I have a file that looks like: chr1 1197592 . C A . . DP=67;ECNT=1;NLOD=8.12 GT:AD:AF:F1R2 0/1:37,2:0.063:13,0 chr1 1355707 . G T . . DP=69;ECNT=1;NLOD=4.51 GT:AD:AF:...
2
votes
2answers
72 views

How to replace a single similar line only AFTER a match [closed]

I am trying to search for a specific item [7_addons] and ONLY replace the NEXT entry that appears from enabled=0 to enabled=1. in this example: sed -i -e 's#enabled=0#enabled=1#g' /etc/file.txt .....
4
votes
1answer
108 views

Sort multiple delimited file lexicographically by one column, numerically by another

I wish to sort the TSV file below (called min_ex) by the first column lexicographically and by the second column numerically. A X, N 2.2 A, N 5.7 A, A 5.8 A, N 2.1 A, T 0.2 B G, M 2....
2
votes
4answers
241 views

Parse all strings of specific length?

I've exported my email archive of 10 years which is very large. I want to parse all the text for any string that is 64 characters long in search of a bitcoin private key. How can I parse strings of ...
1
vote
1answer
45 views

Using sed to replace the hexadecimal code for URL and to insert new SVG codes after SVG tag in all SVG files

I would like to ... replace the hexadecimal code for URL (id) in all SVG files insert/add the new lines of SVG codes after <svg ...>, using Regex. Input: <svg xmlns="http://www.w3.org/...
0
votes
3answers
48 views

Find the min and max value of column 3 and print whole row

I have huuuge sets of data consisting of 3 columns. I'd like to be able to find max and min values in the third column (there is mix of zeros and then negative values, so I want to find max negative ...
6
votes
8answers
373 views

Count lines containing word

I have a file with multiple lines. I want to know, for each word that appears in the total file, how many lines contain that word, for example: 0 hello world the man is world 1 this is the world 2 a ...
7
votes
3answers
397 views

Awk - replace one character only in a certain column

I have a file like this: 2018.01.02;1.5;comment 1 2018.01.04;2.75;comment 2 2018.01.07;5.25;comment 4 2018.01.09;1.25;comment 7 I want to replace all dots . in the second column with a comma , as I ...
1
vote
5answers
46 views

Delete empty line if there is only one between two lines, if there are two empty lines remove only one

I am getting db2 output as of below. this is testing 1 this is testing 2 this is testing 4 The db2 provides output as, output for a query and a empty row and if there is no ...
0
votes
1answer
75 views

how can I merge two text files together

I have a text file like this sp|QBWMM1-2|PDCI_Mouse(741-770) 8864=mil (25/2.20/2.50) sp|Pm345|Hisf_Mouse(613-640) 776=mil (25/2.20/2.50) sp|P0065-2|Hila_Mouse(344-393)6543=mil (25/2.20/2.50) sp|...
2
votes
1answer
29 views

Awk - convert time value to decimal value

How can I use awk to convert a time value to a decimal value. I have been using this command for the other way round (-> from): awk '{printf "%d:%02d", ($1 * 60 / 60), ($1 * 60 % 60)}' <<< ...
4
votes
6answers
108 views

How to edit the entire file after match a grep pattern?

To simplify, I want to edit the whole file after matched pattern, example of a file: $ cat file ip=x.x.x.a mask=255.0.0.0 host=a ip=x.x.x.b mask=255.0.0.0 host=b ip=x.x.x.c mask=255.0.0.0 host=c ip=x....
-1
votes
0answers
36 views

Shell script required for collection/compiling of data

I wanted to collect some stats in a specific format (of previous day) from my server log files using a cronjob everyday. Below is the sample list of log files:- -rw-rw-r-- 1 lpt tkg 2.0M **Jan 3**...
1
vote
3answers
39 views

Find empty value for any name in file

I have a file in the below format: Name=Dan Age=45 Name=Alex Age=35 Cellphone=12345678901 Name=John Age= Cellphone=12345678901 This file has got 1000's of data items in above format, Name=Value. If ...
3
votes
2answers
77 views

Remove even numbered lines of text file [duplicate]

I have a text file on a Linux system, and would like to remove every second line, which is "even" numbered. Example: I would like to remove the second line of the file, the fourth, sixth, eighth line, ...
2
votes
1answer
23 views

How to change data file format from Gnuplot to Tikz errorbars

I would like to change the format of Gnuplot errorbar data to LaTeX/Tikz format. For example, the Gnuplot format data is: 5 74.92725 74.7028332590511 75.1516667409489 112.5 112.55897 112....
1
vote
2answers
49 views

How to start a new column after every nth row?

I have a file (comma separated) on a Linux system with 3 columns. I want to start new column after every 4th row. Input: col1,col2,col3 1,disease1,high 1,disease2,low 1,disease3,high col1,col2,...
2
votes
4answers
109 views

Replace AWORD or BWORD with CWORD in sed

So I have a list of random websites of the following kind: rapido21655bonk.a.sweetpotato.net rapido26230bonk.a.sourpotato.net rapido29926bonk.b.sourpotato.net rapido29926bonk.b....
0
votes
0answers
13 views

Interaction of sed and for loop [duplicate]

for i in {1..25}; do cat file$((i)) | sed 's/.*/$((i))/'; done doesn't print the current value of i but just $((i)). How can I make it print the value of i instead?
0
votes
1answer
37 views

Unix command to search with a string of 6 numbers for any 3 matching numbers per line

Does anyone know of a UNIX command or series of commands to list lines that contains 3 matching numbers to a search with 6 numbers? For example, searching with 38 39 40 41 42 43 I would like to see ...
0
votes
0answers
24 views

Creating Plain Text MAN Pages: Removing overstrike characters [duplicate]

I want to create plain text files of certain man pages such as the "ls" command. Simply performing a man ls > ls.txt, and then running file ls.txt will inform me that this is full of 'ASCII text, ...
1
vote
1answer
56 views

Remove all lines that contain $PWD

I have a file that contains paths - looks like this: /Users/a/Desktop /Users/a/Documents /Users/a/Documents/Work What would be the easiest way to remove all lines that contain the current directory (...
2
votes
4answers
39 views

Linux: how to extract rows of certain IDs and the IDs are writtein in another files?

I have a csv file of IDs. 1 3 8 I have another file of information, not only for the IDs, but also others. If I want to extract the information only of the IDs, how to do it? 1 96 283 4 2 87 918 6 ...