Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
1
vote
2answers
19 views
Remove all matches containing a string until its next space
I want to delete all matches in a text following this pattern:
Each match starts with linux-
Each match ends with a space
(Remove linux-* until immediate next space)
EXAMPLE:
From this text:
...
-3
votes
0answers
29 views
Compare columns and find the difference from a table dynamically
I have a table with 3 columns,first column is some parameter and remaining 3 are last 1 week count of those parameter and has content similar like below. first row is table columns from oracle. I have ...
0
votes
2answers
23 views
Get permissions and concatenate them in one line
Here is the result of listing the files in a directory.
total 4
-rw-r--r-x 1 y_wc y_wc 6828641 dez 24 18:21 file1
-rw-rw-rw- 1 y_wc y_wc 2051577 dez 24 18:13 file2
-rw-rwxr-x 1 y_wc y_wc 1874334 dez ...
2
votes
3answers
46 views
BUILD LOGIC - what could be better than using AWK?
Below is how a log looks like:
Company=XYZ
Req_id=1234
Time_taken=10 sec
Status=Success
Company=ABC
Req_id=3456
Time_taken=200 sec
Status=Failure
Company=DFG
Req_id=3001
Time_taken=15 sec
Status=...
-5
votes
1answer
48 views
Need Desire output using shell [on hold]
I need to get desired output using a shell script.
Input:
aaabbcaaabbcc
Desired Output:
a3b2ca3b2c2
Command should print number of occurrence of each alphabet only when it's greater than ...
0
votes
0answers
42 views
Number of characters in test.txt is 20 Number of words in test.txt is 3 Number of lines in test.txt is 3 [on hold]
Write a shell script which displays the total number of words, characters, lines in a file. Assume the file name is given as command line argument.
file=$1
if [ $# -ne 1 ]
then
echo "$0 fileName"
...
-1
votes
0answers
27 views
awk text processing for multiple pattern pairs
This is a repost of my previous query which was well answered by Gerard,
search pattern pair using awk
Now I have another addition to this query, So here is my query.
Below is input string:
Policy ...
-1
votes
1answer
14 views
find in a file where sign of a column entry changes and print first line where this happens
I have multiply files, each with three columns. I would like to monitor where a given column entries first change sign (i.e negative to positive) so that I save that row where this first happens to ...
0
votes
1answer
12 views
delete rows with duplications in first column in bash
I have a tab delimited .txt file and want to delete rows that have duplications in the first column. If this can be done without sort, that would be amazing, but I've kind of come to terms with that I ...
-1
votes
0answers
47 views
How to get department wise total from text file
I have a text file as below:
sales|100
accounts|500
marketing|200
sales|50
sales|70
accounts|300
marketing|40
accounts|150
marketing|200
sales|80
with the following script, I have generated ...
0
votes
1answer
32 views
How to grep against a list of domains without using a bash script [on hold]
I have been given a list of disposable emails and other emails that my company does not want in our mailing list and asked to remove them. I know grep -v or awk !/xxx/' but this is against a list of ...
0
votes
1answer
36 views
Distinguish ascii from UTF-8 characters in the same file
On Ubuntu 18.04, I created a dummy text file with just one UTF-8 character, è. The other characters are all ascii:
$ cat dummytext
Hello
Helloè
This is the resulting hexdump:
$ hexdump -C dummyfile
...
1
vote
1answer
25 views
Determine Missing DIMM
When a memory module in one of my servers fails the event log will often report the wrong DIMM slot or a non existent DIMM slot altogether. The best way we have come up with to determine the failed ...
1
vote
5answers
42 views
creating new columns
i have two input files like this
file 1
a 1
b 2
c 3
d 4
file 2
a
a
b
c
c
c
d
d
how can i create a shell script to have the following output
a 1
a 1
b 2
c 3
...
1
vote
3answers
34 views
adding a column to an output file with multiple entries
I have 2 files with the following data:
file 1
22:50:48] Return_M
22:50:49] Return_A
22:50:50] Return_F
22:50:51] Return_K
22:50:52] Return_Y
file 2
22:50:48] <0> X led_required
22:50:48]...
7
votes
7answers
502 views
Grepping string, but include all non-blank lines following each grep match
Consider the following toy example:
this is a line
this line contains FOO
this line is not blank
This line also contains FOO
Some random text
This line contains FOO too
Not blank
Also not blank
...
-1
votes
2answers
47 views
filtering output
for i in $(cat HPE_FRAMES_IP); do ssh Reports@$i showsys | awk '{print $2,$5}' | column -t ; ssh Reports@$i showpd -degraded -failed | awk '{print $1,$2,$3,$5}' ; done | grep -v "MiB" | grep -v "...
1
vote
1answer
36 views
Calculate difference in numbers and save in colon-separated time format with bash script
I have a tab-separated file, e.g.,
0018803 01 1710 2050 002571
0018951 01 1934 2525 003277
0019362 02 2404 2415 002829
0019392 01 2621 2820 001924
0019542 01 ...
0
votes
1answer
43 views
Extracting rows from tabular data based on date range [closed]
I have a table with 10 columns. One of them is a date column and I want to get data for a specific date range. For example, I have data from the year 2016 and onwards, but I want to fetch data only ...
0
votes
1answer
33 views
bash script to replace script tags in html with their content
is it possible to use sed to match the source of a script tag within html (which is not valid xml) and replace the whole match with the files' contents?
e.g. say the html contained
<link rel='...
1
vote
1answer
15 views
Add values when X columns exist
I have a file that looks something like this
InvoiceDate Bill Person BAN Type Amount Amount-Expenses GreenFee TaxAmount FinalTotal
1524753593 Jennifer Doe 39193 S ...
1
vote
4answers
51 views
Format Date fields in Pipe Delimited File
I require assistance with this I have a pipe delimited file that is formatted as such
Location|1111|222222|333333|Doe|John|19900723|19900724|19900725
The last 3 fields are Date fields, but I can't ...
0
votes
3answers
35 views
How can I use AWK to break line content into multiple lines using a separator
Given an input like this:
field1,field2,field3,field4
I would like to get an output like this
field1
field2
field3
field4
How can I do this with awk?
2
votes
1answer
50 views
bash remove common lines from two files
I have two files, (no blank lines/Spaces/Tabs)
/tmp/all
aa
bb
cc
hello
SearchText.json
xyz.txt
/tmp/required
SearchText.json
and the end output I want is : (all uncommon lines from /...
1
vote
1answer
15 views
Save Request url in Curl output
The command below cats a file full of urls and feeds it into gnu parallel. I would like the output in curlresponse.txt to have also the original request url. How can it be done?
cat urls.txt | ...
-2
votes
3answers
49 views
Grouping data based on the second column
I have a file with the following lines:
1 a
2 a
3 a
1 b
2 b
1 c
2 c
3 c
4 c
1 d
I want to get the result as:
a 1 2 3
b 1 2
c 1 2 3 4
d 1
10
votes
6answers
632 views
Bash - pair each line of file
This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not ...
3
votes
4answers
161 views
Replace four times with sed
I want to replace the second, third, fourth and fifth dots in this string
2019-03-17T11:32:28.143343Z;1234.5678;901.234;567.89012;3456.78;192.168.0.1
with a commas, to get this result:
2019-03-...
0
votes
2answers
52 views
shell - extracting date & time from logs
I have web server log files that look like this:
2001:67c:1220:80c:d4:985a:df2c:d717 - - [22/Feb/2019:07:49:01 +0100] "GET / HTTP/1.1" 200 58266 "-" "curl/7.61.1"
2001:67c:1220:80c:d4:985a:df2c:d717 -...
1
vote
2answers
49 views
Compare every nth line in 2 files and save (asymmetric) context
I have 2 files, each containing the same amount of 4-lines records, in the same order:
fileA:
record1 line1=header
record1 line2 X <= this is different but should be ignored
record1 line3 id ABC
...
0
votes
1answer
37 views
Find and Replace in Unix - XML File
I have output XML file which has string in each line icn="1", this number inside the double quotes will be changing for every line. But this number is not incrementing in order, so i have to make ...
0
votes
1answer
33 views
merge multiple rows based on a key column
My csv file looks like the following:
8 id1 xyz
1 id1 abcdef
2 id1 mv
1 id1 tzfl
6 id2 xyz
1 id2 abcdef
1 id2 mv
5 id3 xyz
2 id3 abcdef
...
8 id100 xyz
1 id100 abcdef
2 id100 mv
1 id100 tzfl
4 ...
0
votes
2answers
62 views
Echo Results of a Variable in Sed Replacement
I have a file called LoginExcInternal.txt which has data that looks like the following:
Name1 OpenFin/111.111.111.111
Name2 OpenFin/111.111.111.112
Name3 Chrome/111.111.111.111
These two ...
2
votes
2answers
160 views
Bash replace string at multiple places in a file from command line
I have a file with the below content. I want to replace chrome with chromeBackup but not in all places.
Name=chrome
XXXXXXXXXXX
Run=bin -app "chrome/chrome.sh"
XXXXXXXXXXX
Vol.apps/chrome=chrome
...
1
vote
2answers
50 views
Compare data in 5th column with 6th column and print the least difference element
I have a file with below data:
APPLE,FRUIT,1KG,RED,1660.00|1680.00|1700.00|1720.00|1740.00|1760.00|1800.00|1900.00|1920.00|1520.00|1320.00|1340.00|1360.00|1380.00|1400.00|1420.00|1440.00|1460.00|1480....
2
votes
1answer
29 views
How to reflect data points across their median line?
Sample Input
file name
0.00 -1.0000 number1
0.00 -0.8000 number2
0.00 -0.6000 number3
0.00 -0.4000 number4
0.00 -0.2000 number5
0.00 0.0000 number6
0.00 0.2000 number7
0....
0
votes
2answers
52 views
Add line breaks to Grep Commands
How can I add a few line breaks? I want it to show OpenFin category, have a line break, Chrome category, another line break, then memoryInfo.
Here is what I have so far:
cat LoginExInternal.txt | ...
3
votes
2answers
47 views
Merge every two lines after processing each of them with external shell command
I have a file of the following structure:
#1552342335
ls
#1552342359
date
#1552342380
cat .bash_history
i.e. timestamp followed by a text line. I want to process every timestamp line using shell's ...
0
votes
1answer
24 views
Text file: find string, save string field to var, find 2nd string, replace field with var, repeat to end
I think this would best be done with AWK, but not sure. Its been stumping me all day how to do this. I have a text file with * delimiters between the fields on the lines. I need to search for lines ...
0
votes
2answers
53 views
Splitting text file into CSV with multiple delimiters in bash?
Trying to parse a text file into CSV. Catch is that I have multiple delimiters currently, that I'd ideally like to use as column headers but can strip out of the csv results. Ideally would rather ...
1
vote
3answers
42 views
Sort File with word
I have a text file that looks something like this
Name1 OpenFin
Name2 Chrome
Name3 OpenFin
Name4 Chrome
Name5 OpenFin
Name6 OpenFin
I want to sort this is a way where all lines with OpenFin appears ...
0
votes
1answer
15 views
Remove specific words within lines
I have a text file that has something like this:
loginName: Name1 userAgent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/ (KHTML like Gecko) Chrome/61.0.3163.104 OpenFin/9.61.31.74
...
1
vote
4answers
42 views
Get a specific words from a specific line from a text file in NetBSD
How does one proceed to extract member names from a list like the one below and output them to a new file?
Group-name supernatural
Members admin, Admin1, John, sam, dean
Group-id 405
expected out-...
0
votes
2answers
39 views
Remove data in between 2 words
I want to remove all data no matter what it is in between 2 words. The two words are user and Gecko) , exactly as it appears.
For example:
abcd: efgh user jfslkdj ajskdlfj askldjf Gecko) print
ijkl:...
-1
votes
1answer
60 views
“unterminated `s' command” when doing substitution with sed [closed]
I'm trying to run the following sed command:
sed 's?^?'`pwd`'/VOCdevkit/VOC2012/JPEGImages/?; s?$?.jpg?' test.txt > voc.2012.test
But it is responding with an error message:
sed: -e expression #...
0
votes
3answers
44 views
Looking for bash script which can insert input from 2 files
I have 2 files with different data. How can I put data in those files and print it? I was trying for loop, but it is only good for 1 variable.
FileA
https://xyx.com/test-posts/
https://www.abc.com/...
1
vote
1answer
37 views
How to parse openssl certificate output into hostname and verification codes?
I have this shell code that reads lines of host names from input file, performs an openssl command to print ssl certificate info, and outputs the results in the terminal.
while read x;
do
echo $x;
...
0
votes
2answers
54 views
When I add 0 to 0.0, how do I make sure the result is 0.0 not 0?
There are two columns of numbers in a file, the first line like this:
0 0.0
I want to add the numbers in column 1 to those in column 2, and I want to keep the results floats, not ...
1
vote
3answers
54 views
Extracting text from a file when I know its format
I am running a script which will output a file called scServer.scs which looks something like this:
49138@bber0501u29b.bb2.cluster
2
Solution 1
Fluid Flow (Fluent)
Solution
...
4
votes
2answers
291 views
Searching for a string that contains the file name
I have a large set of files in a directory. The files contains arbitrary text.
I want to search for the file name inside that particular file text. To clarify, I have file1.py.txt (yeas, two dots .py....
