Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
8,240
questions
-3
votes
1
answer
24
views
How to select specific rows and their specific columns in bash?
For example
I have this data, I want to get output as
serNo: ASDFAFDAFD3223
profileType": "Asset Tag: profileValue": "12341233
corrected ouput as serNo: ...
2
votes
4
answers
155
views
How to extract columns in which their name containing the word "chronic" from CSV file
I have a big csv file ( around 1000 columns ) and I want to extract to a new file only columns that contain the word "chronic" in their header name. How can I do that ?
For example if I have:...
- 29
-2
votes
4
answers
58
views
Printing parts of certain lines
I want to read a file and print all lines that contain
## DF <anything>
or else end with
<something> # DF <anything>
Then print from DF onwards
Here are some examples
local ...
- 29
0
votes
2
answers
24
views
Two Columns For Text And One For Footnotes In Groff
I'm new to groff, so excuse me if it's a silly question. I can't find an appropriate solution for have two-columned content with one-columned footnotes. I've messed with .1C and .2C macros to get the ...
-3
votes
1
answer
33
views
Find hostname in a text file within directory, and rename the directory to the hostname
The hostname is included in the text file /sys/info with the format
Host Name: xxxxxxx
I want to fetch the hostname (in this case xxxxxxx) from /sys/info and rename the parent directory to be the ...
- 1
0
votes
3
answers
59
views
Using "sed" to insert text at the end of a field in TSV file
I want to insert characters in TSV files using the sed command with wildcard expressions:
I have a file like this:
Marker Pvalue Trait Chr Pos
S3_16887238 6.172847e-09 Total_Soil_S 3 ...
- 15
0
votes
1
answer
46
views
Is it possible to create such a tool that allows to save a being read text from a PDF reader into a string for handling this text afterwards?
The Linux Mint has a tool for PDF reading that is called the "Xreader" and installed by default together with the operation system installation.
Is it possible to create such a tool that ...
- 101
0
votes
1
answer
52
views
unterminated s command with seed on special characters with sed [duplicate]
i have special characters in the content of $VALUE which I would to add in file ($FILE).
When I run this cammand :
sed -i "s|T_VALUE|$VALUE|" "$FILE"
i have an error unterminated ...
- 99
0
votes
0
answers
24
views
Text processing involving shell variables doesn't evaluate the variable correctly [duplicate]
I have special characters in the content of $VALUE which I would to add in file ($FILE).
When I do :
sed -i 's|T_VALUE|$VALUE|g' "$FILE"
my file contains the literal string $VALUE instead ...
- 99
1
vote
4
answers
73
views
Grep find text in a file and print the text 3 lines before it
I have a file and will be looking for multiple file names in the file. Example search text intl_reg.jcl. Once I find it I need to get the text that is 3 lines above it. For example below is 4 lines ...
- 15
0
votes
2
answers
37
views
I want list only list the text using grep or any other option using shell script
I have folder called rules/resources, in that I have the sub folders lets say A, B and C. Each sub-folder contains constraint.yaml.
Now I want to grep the constraint.yaml files which contain the ...
- 1
0
votes
4
answers
64
views
Remove last line of file by any user, owned by root
I have a file with permissions 666 so users can edit it with a different script. This works.
I need to script to remove the last line of the file, however this is not working as using something like ...
- 452
0
votes
1
answer
45
views
Help with yq assignment
I'm trying to use yq to update the static IP on an ubuntu machine via the netcfg yaml file.
My file:
network:
version: 2
renderer: networkd
ethernets:
enp0s1:
dhcp4: no
addresses:...
- 35.1k
4
votes
3
answers
199
views
AWK print lines between two patterns, only need to print last occurrence of matching lines
I want to filter a log file and print some lines between two matches and print only the last occurrence.
Sample file content:
2023-03-08 11:12:44,306 - Code Deploy - INFO - Received signal
2023-03-08 ...
- 399
0
votes
3
answers
71
views
How to print the content between the third and the forth double quotation marks
I have some logs from a web service on a Linux machine. The logs look like this:
{"log":"[2023-03-09T06:39:10.669Z] \"GET /server/prod?blank=true HTTP/1.1\" 200 - 0 874 1 1 \&...
- 3,009
0
votes
5
answers
88
views
Remove three lines from file until a match
982
01:25:09,473 --> 01:25:10,978
Stay with me.
983
01:25:09,473 --> 01:25:10,978
Stay with me.
984
01:25:15,390 --> 01:25:18,484
( MAJESTIC MUSIC )
I want to delete three lines below 984 (...
1
vote
4
answers
67
views
Extract lines from indented output
I'd like to parse parses the output below (from ddcutil) in a shell command:
Model: MQ780
MCCS version: 2.1
Commands:
Op Code: 01 (VCP Request)
Op Code: 02 (VCP Response)
Op Code: 03 (VCP Set)...
- 121
0
votes
1
answer
75
views
evaluate a string that is a bash if-statement
My script has a function do_command() that prints a command and then executes it. I want to put my commands in a bash array of strings, and then print _ execute them with this function. But when I try ...
- 539
0
votes
3
answers
86
views
Create SQL script from file having table names as first column and fields as second column
I have a text file containing two columns separated by a space:
Foo Param1
Foo Param2
Foo Param3
Foo Param4
Foo Param5
Bar Stuff1
Bar Stuff2
Bar Param3
Bar Stuff4
etc
The first column is a group name,...
- 317
-2
votes
1
answer
48
views
Split/extract street name from street number [duplicate]
I have some addresses.csv in this formats
Street 1
Street 10
Street 100
Street 1000
Straße 1b
Straße1b
Street 1 B
Street, 1B
The Street 1B
The-Street 1B
The'Street 1B
The&Street 1B
The Str. 1B
...
- 157
1
vote
4
answers
93
views
How to change a range of instances of a string
I need to change some instances of a sub-string on every line of a text file. These instances are all consecutive such as 3 to 6 or 2 to 5 etc. What command do I need for this operation? The nearest I ...
- 33
0
votes
1
answer
103
views
The ultimate tool to split/extract street name from street number
I have some addresses.csv in different international formats
Example Street 1
Teststraße 2
Teststr. 1-5
Baker Street 221b
221B Baker Street
19th Ave 3B
3B 2nd Ave
1-3 2nd Mount x Ave
105 Lock St # 219
...
- 157
1
vote
1
answer
81
views
How to insert file on matching pattern with filename?
Suppose I have 3 files:
index.html
And here is foo:
<!-- include: includes/foo.html -->
This can be done similarly:
<code>
<!-- include: includes/bar.txt -->
</code>
includes/...
- 75
0
votes
2
answers
100
views
Identify only the duplicate records from a file and print only the duplicate records into a new file
I have many 10 columns in a CSV file, out of which column 1, 3 and 4 have duplicate records in it. Need to identify these duplicate records and store them into a separate TXT file.
File is a ',' ...
- 15
1
vote
4
answers
80
views
Summing up the values in the 5th line of every 6-line record in a file
I have a txt file with blocks of text like this:
17-01-2023
Purchase AAA
Apple Pay John Doe
Full Payment
-11,34€
0,11€
30-01-2023
Purchase BBB
Mastercard Jane Doe
Installment
-23,90€
0,24€
So, we ...
- 4,354
3
votes
3
answers
181
views
Calculate decimals input piped to awk using two values which both occur on different rows and repeat every nth row?
I have input piped to awk on a macOS system. I want the sum of two decimal values. One occurs every second row, the other is on every fourth row. I want the result of the sum on every fifth row. The ...
- 33
4
votes
6
answers
916
views
How to delete all UUID from fstab but not the UUID of boot filesystem
on the following example fstab file, we want to delete all lines that start with UUID , but except the UUID line with boot word
/dev/mapper/VG100-lv_root / xfs defaults ...
- 12.2k
2
votes
1
answer
36
views
Convert grouped log entry to single line
I have one log (generated by Unison file syncronized) thats look like this:
UNISON 2.51.3 (OCAML 4.11.1) started propagating changes at 15:25:10.21 on 27 Feb 2023
[BGN] Copying TestFolder/01/...
- 23
0
votes
2
answers
57
views
Remove and add at the same time with sed
I have an variable in my script which I want to change:
read -p "Enter the name of city: " city
if [[ $city =~ Liverpool ]]; then
sed -i -e "/\$type\_of\_city =/ s/= .*/= ${city}/&...
- 17
2
votes
2
answers
88
views
Unpivoting Delimited String to get the parent child relationship
I have a scenario where in need to unpivot the data to achieve the parent child relationship.
My source data is as below:
Key_Col|Hierarchy
1|a,b,c,d
2|a,b,c,d,e
My Expected Output below:
Key Col
...
- 43
0
votes
2
answers
68
views
"grep -A" but with some lines removed
The problem
The output of grep -A 4 davidson file.txt gives me:
davidson 0 27 |r|= 0.00244 e= [-0.04073962 0.42777949 0.44157777 0.45997235 0.45997235 0.51613738
0.55005554 0.5538866 0....
- 237
0
votes
0
answers
36
views
datamash crosstab with counts per row, per column; post-sorted
For a huge list of pairs each looking like
incident_N, name_M
...I need to create an "advanced" cross-table with datamash:
for each incident, it should also show count of names involved
...
- 63
-1
votes
2
answers
39
views
Modify LDAP data given list of identifiers
I have a file with below content, where few lines are of length 15 while rest are less than 15.
AAAAAAAAAAAAA
BBBBBBBB
cccc
I need an output in below way.
Delete Subscriber:
AAAAAAAAAAAAA
in
IMSI=...
- 45
4
votes
4
answers
151
views
sed - Print lines matched by a pattern range if one line matches a condition with ^
I found this question and it solved 99% of my issue but I can't figure out to match the end pattern with ^.
sed - Print lines matched by a pattern range if one line matches a condition
Here are my ...
5
votes
3
answers
229
views
Processing a continuous single line of data with stream processing in bash pipeline?
I am debugging an embedded server that outputs a continuous single line of text to a specified network port. There is no newline anywhere in the stream but it is text data and I would like to format ...
- 193
4
votes
5
answers
577
views
How can I transform dates expressed with month names to numerical month designations?
I use MacOS and I have variable date in format
3.Jan.2023,
12.Nov.2017,
9.Apr.2022,
...
I need to change in
03.01.2023,
12.11.2017,
09.04.2022,
...
- 43
1
vote
2
answers
76
views
Match date pattern that precedes SED range match and include the matching range in output
Having issues with trying to set variable (example: str_pattern) with multiple values and then using those in sed command to match range in reverse to previous blank line or back to ExecuteThread.
...
- 11
0
votes
2
answers
47
views
how to use grep to only extract a certain column from a linux command output
I want to use grep to only see a certain column from the output of a Linux command. When I use this command, I get the following output:
$ ps ax | grep sshd | grep 'priv' | sort -k 3
4886 ? ...
- 3
0
votes
5
answers
82
views
How to bring the next line to the end of the first line using awk, separated by a comma?
I have downloaded some sequences from a publicly available database in .fa format. I want to generate a .csv file that contains the name of the sequence, and its length, separated by a comma.
This is ...
-5
votes
1
answer
33
views
Read line from 2 files and print values of both files in one line shell script
Need help for unix shell script.
I have 2 files:
file1:
abc
def
ghi
file2:
123
456
789
I need output as below:
abc 123
def 456
ghi 789
Request you to please help asap
-1
votes
6
answers
107
views
Output everything before the first slash '/' in a line [duplicate]
I have a set of URLs and I'm really only interested in anything up to the first /.
How can I capture this info to a text file?
Input (foo.txt):
apple.com/nothing.js
t1.msn.com/cookie=22
happy.net/...
- 1
0
votes
2
answers
107
views
Replace multiple lines and words using sed in a for loop
I have 2 files (mac and uid) with data on a Linux machine, that I want to carry to a "template", with a list, using sed to replace the data with variables.
list.txt:
CP00
CP01
mac.txt:
FF-...
- 11
0
votes
0
answers
49
views
Surrounding line with quotes using sed in WSL2 Ubuntu not working as expected
The command: sed -e 's/.*/\"&\",/'
I tested the regex at https://sed.js.org/ and it worked correctly, so if I had input like this:
George Washington, 1789-1797
John Adams, 1797-1801
...
0
votes
1
answer
74
views
Ensuring begin and end sections
I have files with sections defined by the starting section composed oy TITLE, SUBTITLE, followed by comma separated KEYWORD.
## TITLE [SUBTITLE] KEYWORD,KEYWORD
The ending is done using
## END OF ...
- 13
4
votes
2
answers
140
views
How to select rows according to the frequency of elements in columns
I have a file, as you can see, which is tab-separated data, about 4,000 lines, ten columns.
The second column of the file records the different tissues.
samples tissue_s tissue_e tissue_d ...
- 53
0
votes
3
answers
81
views
how to change a word in a specific group using sed?
I need to change a word in a specific define service section where service_description is NRPE - NTP_TIME and where contact_groups is opsgenie-sla1, I need to change to opsgenie-sla2 and I would like ...
- 13
1
vote
3
answers
56
views
How do I pick only one record per ID based on the earliest date in another column?
I have a file containing multiple lines, with fields separated by tab:
ID Code Date
1 XX 23/1/2018
1 XX 11/3/2021
2 XX 14/5/2011
2 XX 20/9/2013
3 XX 08/7/2014
3 XX 11/9/2016
3 XX ...
- 15
-2
votes
3
answers
141
views
How to combine cat, grep, awk and sed commands?
When I do enter those commands individualy, I get the results I want:
cat /var/log/apt/history.log | grep 'apt install' > installation.txt
grep -v "nvidia" installation.txt > tmpfile &...
user330163
0
votes
1
answer
52
views
Text processing - Modify based off two different files
I have two files named one.txt containing
"297","237","14",NULL,"51.195.14.201","02:00:00:1a:1c:06","255.0.0.0","10.0.10.1","...
- 2,793
0
votes
3
answers
178
views
How can I test if a string has any ASCII white space characters in it?
How can I check if a string has any whitespace characters in it? I do not have to worry about things outside of ASCII for example unicode zero width characters etc. You can assume that the string is ...
- 4,743