Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

1
vote
2answers
19 views

Remove all matches containing a string until its next space

I want to delete all matches in a text following this pattern: Each match starts with linux- Each match ends with a space (Remove linux-* until immediate next space) EXAMPLE: From this text: ...
-3
votes
0answers
29 views

Compare columns and find the difference from a table dynamically

I have a table with 3 columns,first column is some parameter and remaining 3 are last 1 week count of those parameter and has content similar like below. first row is table columns from oracle. I have ...
0
votes
2answers
23 views

Get permissions and concatenate them in one line

Here is the result of listing the files in a directory. total 4 -rw-r--r-x 1 y_wc y_wc 6828641 dez 24 18:21 file1 -rw-rw-rw- 1 y_wc y_wc 2051577 dez 24 18:13 file2 -rw-rwxr-x 1 y_wc y_wc 1874334 dez ...
2
votes
3answers
46 views

BUILD LOGIC - what could be better than using AWK?

Below is how a log looks like: Company=XYZ Req_id=1234 Time_taken=10 sec Status=Success Company=ABC Req_id=3456 Time_taken=200 sec Status=Failure Company=DFG Req_id=3001 Time_taken=15 sec Status=...
-5
votes
1answer
48 views

Need Desire output using shell [on hold]

I need to get desired output using a shell script. Input: aaabbcaaabbcc Desired Output: a3b2ca3b2c2 Command should print number of occurrence of each alphabet only when it's greater than ...
0
votes
0answers
42 views

Number of characters in test.txt is 20 Number of words in test.txt is 3 Number of lines in test.txt is 3 [on hold]

Write a shell script which displays the total number of words, characters, lines in a file. Assume the file name is given as command line argument. file=$1 if [ $# -ne 1 ] then echo "$0 fileName" ...
-1
votes
0answers
27 views

awk text processing for multiple pattern pairs

This is a repost of my previous query which was well answered by Gerard, search pattern pair using awk Now I have another addition to this query, So here is my query. Below is input string: Policy ...
-1
votes
1answer
14 views

find in a file where sign of a column entry changes and print first line where this happens

I have multiply files, each with three columns. I would like to monitor where a given column entries first change sign (i.e negative to positive) so that I save that row where this first happens to ...
0
votes
1answer
12 views

delete rows with duplications in first column in bash

I have a tab delimited .txt file and want to delete rows that have duplications in the first column. If this can be done without sort, that would be amazing, but I've kind of come to terms with that I ...
-1
votes
0answers
47 views

How to get department wise total from text file

I have a text file as below: sales|100 accounts|500 marketing|200 sales|50 sales|70 accounts|300 marketing|40 accounts|150 marketing|200 sales|80 with the following script, I have generated ...
0
votes
1answer
32 views

How to grep against a list of domains without using a bash script [on hold]

I have been given a list of disposable emails and other emails that my company does not want in our mailing list and asked to remove them. I know grep -v or awk !/xxx/' but this is against a list of ...
0
votes
1answer
36 views

Distinguish ascii from UTF-8 characters in the same file

On Ubuntu 18.04, I created a dummy text file with just one UTF-8 character, è. The other characters are all ascii: $ cat dummytext Hello Helloè This is the resulting hexdump: $ hexdump -C dummyfile ...
1
vote
1answer
25 views

Determine Missing DIMM

When a memory module in one of my servers fails the event log will often report the wrong DIMM slot or a non existent DIMM slot altogether. The best way we have come up with to determine the failed ...
1
vote
5answers
42 views

creating new columns

i have two input files like this file 1 a 1 b 2 c 3 d 4 file 2 a a b c c c d d how can i create a shell script to have the following output a 1 a 1 b 2 c 3 ...
1
vote
3answers
34 views

adding a column to an output file with multiple entries

I have 2 files with the following data: file 1 22:50:48] Return_M 22:50:49] Return_A 22:50:50] Return_F 22:50:51] Return_K 22:50:52] Return_Y file 2 22:50:48] <0> X led_required 22:50:48]...
7
votes
7answers
502 views

Grepping string, but include all non-blank lines following each grep match

Consider the following toy example: this is a line this line contains FOO this line is not blank This line also contains FOO Some random text This line contains FOO too Not blank Also not blank ...
-1
votes
2answers
47 views

filtering output

for i in $(cat HPE_FRAMES_IP); do ssh Reports@$i showsys | awk '{print $2,$5}' | column -t ; ssh Reports@$i showpd -degraded -failed | awk '{print $1,$2,$3,$5}' ; done | grep -v "MiB" | grep -v "...
1
vote
1answer
36 views

Calculate difference in numbers and save in colon-separated time format with bash script

I have a tab-separated file, e.g., 0018803 01 1710 2050 002571 0018951 01 1934 2525 003277 0019362 02 2404 2415 002829 0019392 01 2621 2820 001924 0019542 01 ...
0
votes
1answer
43 views

Extracting rows from tabular data based on date range [closed]

I have a table with 10 columns. One of them is a date column and I want to get data for a specific date range. For example, I have data from the year 2016 and onwards, but I want to fetch data only ...
0
votes
1answer
33 views

bash script to replace script tags in html with their content

is it possible to use sed to match the source of a script tag within html (which is not valid xml) and replace the whole match with the files' contents? e.g. say the html contained <link rel='...
1
vote
1answer
15 views

Add values when X columns exist

I have a file that looks something like this InvoiceDate Bill Person BAN Type Amount Amount-Expenses GreenFee TaxAmount FinalTotal 1524753593 Jennifer Doe 39193 S ...
1
vote
4answers
51 views

Format Date fields in Pipe Delimited File

I require assistance with this I have a pipe delimited file that is formatted as such Location|1111|222222|333333|Doe|John|19900723|19900724|19900725 The last 3 fields are Date fields, but I can't ...
0
votes
3answers
35 views

How can I use AWK to break line content into multiple lines using a separator

Given an input like this: field1,field2,field3,field4 I would like to get an output like this field1 field2 field3 field4 How can I do this with awk?
2
votes
1answer
50 views

bash remove common lines from two files

I have two files, (no blank lines/Spaces/Tabs) /tmp/all aa bb cc hello SearchText.json xyz.txt /tmp/required SearchText.json and the end output I want is : (all uncommon lines from /...
1
vote
1answer
15 views

Save Request url in Curl output

The command below cats a file full of urls and feeds it into gnu parallel. I would like the output in curlresponse.txt to have also the original request url. How can it be done? cat urls.txt | ...
-2
votes
3answers
49 views

Grouping data based on the second column

I have a file with the following lines: 1 a 2 a 3 a 1 b 2 b 1 c 2 c 3 c 4 c 1 d I want to get the result as: a 1 2 3 b 1 2 c 1 2 3 4 d 1
10
votes
6answers
632 views

Bash - pair each line of file

This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not ...
3
votes
4answers
161 views

Replace four times with sed

I want to replace the second, third, fourth and fifth dots in this string 2019-03-17T11:32:28.143343Z;1234.5678;901.234;567.89012;3456.78;192.168.0.1 with a commas, to get this result: 2019-03-...
0
votes
2answers
52 views

shell - extracting date & time from logs

I have web server log files that look like this: 2001:67c:1220:80c:d4:985a:df2c:d717 - - [22/Feb/2019:07:49:01 +0100] "GET / HTTP/1.1" 200 58266 "-" "curl/7.61.1" 2001:67c:1220:80c:d4:985a:df2c:d717 -...
1
vote
2answers
49 views

Compare every nth line in 2 files and save (asymmetric) context

I have 2 files, each containing the same amount of 4-lines records, in the same order: fileA: record1 line1=header record1 line2 X <= this is different but should be ignored record1 line3 id ABC ...
0
votes
1answer
37 views

Find and Replace in Unix - XML File

I have output XML file which has string in each line icn="1", this number inside the double quotes will be changing for every line. But this number is not incrementing in order, so i have to make ...
0
votes
1answer
33 views

merge multiple rows based on a key column

My csv file looks like the following: 8 id1 xyz 1 id1 abcdef 2 id1 mv 1 id1 tzfl 6 id2 xyz 1 id2 abcdef 1 id2 mv 5 id3 xyz 2 id3 abcdef ... 8 id100 xyz 1 id100 abcdef 2 id100 mv 1 id100 tzfl 4 ...
0
votes
2answers
62 views

Echo Results of a Variable in Sed Replacement

I have a file called LoginExcInternal.txt which has data that looks like the following: Name1 OpenFin/111.111.111.111 Name2 OpenFin/111.111.111.112 Name3 Chrome/111.111.111.111 These two ...
2
votes
2answers
160 views

Bash replace string at multiple places in a file from command line

I have a file with the below content. I want to replace chrome with chromeBackup but not in all places. Name=chrome XXXXXXXXXXX Run=bin -app "chrome/chrome.sh" XXXXXXXXXXX Vol.apps/chrome=chrome ...
1
vote
2answers
50 views

Compare data in 5th column with 6th column and print the least difference element

I have a file with below data: APPLE,FRUIT,1KG,RED,1660.00|1680.00|1700.00|1720.00|1740.00|1760.00|1800.00|1900.00|1920.00|1520.00|1320.00|1340.00|1360.00|1380.00|1400.00|1420.00|1440.00|1460.00|1480....
2
votes
1answer
29 views

How to reflect data points across their median line?

Sample Input file name 0.00 -1.0000 number1 0.00 -0.8000 number2 0.00 -0.6000 number3 0.00 -0.4000 number4 0.00 -0.2000 number5 0.00 0.0000 number6 0.00 0.2000 number7 0....
0
votes
2answers
52 views

Add line breaks to Grep Commands

How can I add a few line breaks? I want it to show OpenFin category, have a line break, Chrome category, another line break, then memoryInfo. Here is what I have so far: cat LoginExInternal.txt | ...
3
votes
2answers
47 views

Merge every two lines after processing each of them with external shell command

I have a file of the following structure: #1552342335 ls #1552342359 date #1552342380 cat .bash_history i.e. timestamp followed by a text line. I want to process every timestamp line using shell's ...
0
votes
1answer
24 views

Text file: find string, save string field to var, find 2nd string, replace field with var, repeat to end

I think this would best be done with AWK, but not sure. Its been stumping me all day how to do this. I have a text file with * delimiters between the fields on the lines. I need to search for lines ...
0
votes
2answers
53 views

Splitting text file into CSV with multiple delimiters in bash?

Trying to parse a text file into CSV. Catch is that I have multiple delimiters currently, that I'd ideally like to use as column headers but can strip out of the csv results. Ideally would rather ...
1
vote
3answers
42 views

Sort File with word

I have a text file that looks something like this Name1 OpenFin Name2 Chrome Name3 OpenFin Name4 Chrome Name5 OpenFin Name6 OpenFin I want to sort this is a way where all lines with OpenFin appears ...
0
votes
1answer
15 views

Remove specific words within lines

I have a text file that has something like this: loginName: Name1 userAgent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/ (KHTML like Gecko) Chrome/61.0.3163.104 OpenFin/9.61.31.74 ...
1
vote
4answers
42 views

Get a specific words from a specific line from a text file in NetBSD

How does one proceed to extract member names from a list like the one below and output them to a new file? Group-name supernatural Members admin, Admin1, John, sam, dean Group-id 405 expected out-...
0
votes
2answers
39 views

Remove data in between 2 words

I want to remove all data no matter what it is in between 2 words. The two words are user and Gecko) , exactly as it appears. For example: abcd: efgh user jfslkdj ajskdlfj askldjf Gecko) print ijkl:...
-1
votes
1answer
60 views

“unterminated `s' command” when doing substitution with sed [closed]

I'm trying to run the following sed command: sed 's?^?'`pwd`'/VOCdevkit/VOC2012/JPEGImages/?; s?$?.jpg?' test.txt > voc.2012.test But it is responding with an error message: sed: -e expression #...
0
votes
3answers
44 views

Looking for bash script which can insert input from 2 files

I have 2 files with different data. How can I put data in those files and print it? I was trying for loop, but it is only good for 1 variable. FileA https://xyx.com/test-posts/ https://www.abc.com/...
1
vote
1answer
37 views

How to parse openssl certificate output into hostname and verification codes?

I have this shell code that reads lines of host names from input file, performs an openssl command to print ssl certificate info, and outputs the results in the terminal. while read x; do echo $x; ...
0
votes
2answers
54 views

When I add 0 to 0.0, how do I make sure the result is 0.0 not 0?

There are two columns of numbers in a file, the first line like this: 0 0.0 I want to add the numbers in column 1 to those in column 2, and I want to keep the results floats, not ...
1
vote
3answers
54 views

Extracting text from a file when I know its format

I am running a script which will output a file called scServer.scs which looks something like this: 49138@bber0501u29b.bb2.cluster 2 Solution 1 Fluid Flow (Fluent) Solution ...
4
votes
2answers
291 views

Searching for a string that contains the file name

I have a large set of files in a directory. The files contains arbitrary text. I want to search for the file name inside that particular file text. To clarify, I have file1.py.txt (yeas, two dots .py....