Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
-3 votes
1 answer
24 views

How to select specific rows and their specific columns in bash?

For example I have this data, I want to get output as serNo: ASDFAFDAFD3223 profileType": "Asset Tag: profileValue": "12341233 corrected ouput as serNo: ...
2 votes
4 answers
155 views

How to extract columns in which their name containing the word "chronic" from CSV file

I have a big csv file ( around 1000 columns ) and I want to extract to a new file only columns that contain the word "chronic" in their header name. How can I do that ? For example if I have:...
-2 votes
4 answers
58 views

Printing parts of certain lines

I want to read a file and print all lines that contain ## DF <anything> or else end with <something> # DF <anything> Then print from DF onwards Here are some examples local ...
  • 29
0 votes
2 answers
24 views

Two Columns For Text And One For Footnotes In Groff

I'm new to groff, so excuse me if it's a silly question. I can't find an appropriate solution for have two-columned content with one-columned footnotes. I've messed with .1C and .2C macros to get the ...
-3 votes
1 answer
33 views

Find hostname in a text file within directory, and rename the directory to the hostname

The hostname is included in the text file /sys/info with the format Host Name: xxxxxxx I want to fetch the hostname (in this case xxxxxxx) from /sys/info and rename the parent directory to be the ...
  • 1
0 votes
3 answers
59 views

Using "sed" to insert text at the end of a field in TSV file

I want to insert characters in TSV files using the sed command with wildcard expressions: I have a file like this: Marker Pvalue Trait Chr Pos S3_16887238 6.172847e-09 Total_Soil_S 3 ...
0 votes
1 answer
46 views

Is it possible to create such a tool that allows to save a being read text from a PDF reader into a string for handling this text afterwards?

The Linux Mint has a tool for PDF reading that is called the "Xreader" and installed by default together with the operation system installation. Is it possible to create such a tool that ...
  • 101
0 votes
1 answer
52 views

unterminated s command with seed on special characters with sed [duplicate]

i have special characters in the content of $VALUE which I would to add in file ($FILE). When I run this cammand : sed -i "s|T_VALUE|$VALUE|" "$FILE" i have an error unterminated ...
  • 99
0 votes
0 answers
24 views

Text processing involving shell variables doesn't evaluate the variable correctly [duplicate]

I have special characters in the content of $VALUE which I would to add in file ($FILE). When I do : sed -i 's|T_VALUE|$VALUE|g' "$FILE" my file contains the literal string $VALUE instead ...
  • 99
1 vote
4 answers
73 views

Grep find text in a file and print the text 3 lines before it

I have a file and will be looking for multiple file names in the file. Example search text intl_reg.jcl. Once I find it I need to get the text that is 3 lines above it. For example below is 4 lines ...
0 votes
2 answers
37 views

I want list only list the text using grep or any other option using shell script

I have folder called rules/resources, in that I have the sub folders lets say A, B and C. Each sub-folder contains constraint.yaml. Now I want to grep the constraint.yaml files which contain the ...
  • 1
0 votes
4 answers
64 views

Remove last line of file by any user, owned by root

I have a file with permissions 666 so users can edit it with a different script. This works. I need to script to remove the last line of the file, however this is not working as using something like ...
  • 452
0 votes
1 answer
45 views

Help with yq assignment

I'm trying to use yq to update the static IP on an ubuntu machine via the netcfg yaml file. My file: network: version: 2 renderer: networkd ethernets: enp0s1: dhcp4: no addresses:...
  • 35.1k
4 votes
3 answers
199 views

AWK print lines between two patterns, only need to print last occurrence of matching lines

I want to filter a log file and print some lines between two matches and print only the last occurrence. Sample file content: 2023-03-08 11:12:44,306 - Code Deploy - INFO - Received signal 2023-03-08 ...
0 votes
3 answers
71 views

How to print the content between the third and the forth double quotation marks

I have some logs from a web service on a Linux machine. The logs look like this: {"log":"[2023-03-09T06:39:10.669Z] \"GET /server/prod?blank=true HTTP/1.1\" 200 - 0 874 1 1 \&...
  • 3,009
0 votes
5 answers
88 views

Remove three lines from file until a match

982 01:25:09,473 --> 01:25:10,978 Stay with me. 983 01:25:09,473 --> 01:25:10,978 Stay with me. 984 01:25:15,390 --> 01:25:18,484 ( MAJESTIC MUSIC ) I want to delete three lines below 984 (...
1 vote
4 answers
67 views

Extract lines from indented output

I'd like to parse parses the output below (from ddcutil) in a shell command: Model: MQ780 MCCS version: 2.1 Commands: Op Code: 01 (VCP Request) Op Code: 02 (VCP Response) Op Code: 03 (VCP Set)...
  • 121
0 votes
1 answer
75 views

evaluate a string that is a bash if-statement

My script has a function do_command() that prints a command and then executes it. I want to put my commands in a bash array of strings, and then print _ execute them with this function. But when I try ...
0 votes
3 answers
86 views

Create SQL script from file having table names as first column and fields as second column

I have a text file containing two columns separated by a space: Foo Param1 Foo Param2 Foo Param3 Foo Param4 Foo Param5 Bar Stuff1 Bar Stuff2 Bar Param3 Bar Stuff4 etc The first column is a group name,...
  • 317
-2 votes
1 answer
48 views

Split/extract street name from street number [duplicate]

I have some addresses.csv in this formats Street 1 Street 10 Street 100 Street 1000 Straße 1b Straße1b Street 1 B Street, 1B The Street 1B The-Street 1B The'Street 1B The&Street 1B The Str. 1B ...
  • 157
1 vote
4 answers
93 views

How to change a range of instances of a string

I need to change some instances of a sub-string on every line of a text file. These instances are all consecutive such as 3 to 6 or 2 to 5 etc. What command do I need for this operation? The nearest I ...
  • 33
0 votes
1 answer
103 views

The ultimate tool to split/extract street name from street number

I have some addresses.csv in different international formats Example Street 1 Teststraße 2 Teststr. 1-5 Baker Street 221b 221B Baker Street 19th Ave 3B 3B 2nd Ave 1-3 2nd Mount x Ave 105 Lock St # 219 ...
  • 157
1 vote
1 answer
81 views

How to insert file on matching pattern with filename?

Suppose I have 3 files: index.html And here is foo: <!-- include: includes/foo.html --> This can be done similarly: <code> <!-- include: includes/bar.txt --> </code> includes/...
0 votes
2 answers
100 views

Identify only the duplicate records from a file and print only the duplicate records into a new file

I have many 10 columns in a CSV file, out of which column 1, 3 and 4 have duplicate records in it. Need to identify these duplicate records and store them into a separate TXT file. File is a ',' ...
1 vote
4 answers
80 views

Summing up the values in the 5th line of every 6-line record in a file

I have a txt file with blocks of text like this: 17-01-2023 Purchase AAA Apple Pay John Doe Full Payment -11,34€ 0,11€ 30-01-2023 Purchase BBB Mastercard Jane Doe Installment -23,90€ 0,24€ So, we ...
  • 4,354
3 votes
3 answers
181 views

Calculate decimals input piped to awk using two values which both occur on different rows and repeat every nth row?

I have input piped to awk on a macOS system. I want the sum of two decimal values. One occurs every second row, the other is on every fourth row. I want the result of the sum on every fifth row. The ...
  • 33
4 votes
6 answers
916 views

How to delete all UUID from fstab but not the UUID of boot filesystem

on the following example fstab file, we want to delete all lines that start with UUID , but except the UUID line with boot word /dev/mapper/VG100-lv_root / xfs defaults ...
  • 12.2k
2 votes
1 answer
36 views

Convert grouped log entry to single line

I have one log (generated by Unison file syncronized) thats look like this: UNISON 2.51.3 (OCAML 4.11.1) started propagating changes at 15:25:10.21 on 27 Feb 2023 [BGN] Copying TestFolder/01/...
  • 23
0 votes
2 answers
57 views

Remove and add at the same time with sed

I have an variable in my script which I want to change: read -p "Enter the name of city: " city if [[ $city =~ Liverpool ]]; then sed -i -e "/\$type\_of\_city =/ s/= .*/= ${city}/&...
  • 17
2 votes
2 answers
88 views

Unpivoting Delimited String to get the parent child relationship

I have a scenario where in need to unpivot the data to achieve the parent child relationship. My source data is as below: Key_Col|Hierarchy 1|a,b,c,d 2|a,b,c,d,e My Expected Output below: Key Col ...
  • 43
0 votes
2 answers
68 views

"grep -A" but with some lines removed

The problem The output of grep -A 4 davidson file.txt gives me: davidson 0 27 |r|= 0.00244 e= [-0.04073962 0.42777949 0.44157777 0.45997235 0.45997235 0.51613738 0.55005554 0.5538866 0....
0 votes
0 answers
36 views

datamash crosstab with counts per row, per column; post-sorted

For a huge list of pairs each looking like incident_N, name_M ...I need to create an "advanced" cross-table with datamash: for each incident, it should also show count of names involved ...
-1 votes
2 answers
39 views

Modify LDAP data given list of identifiers

I have a file with below content, where few lines are of length 15 while rest are less than 15. AAAAAAAAAAAAA BBBBBBBB cccc I need an output in below way. Delete Subscriber: AAAAAAAAAAAAA in IMSI=...
4 votes
4 answers
151 views

sed - Print lines matched by a pattern range if one line matches a condition with ^

I found this question and it solved 99% of my issue but I can't figure out to match the end pattern with ^. sed - Print lines matched by a pattern range if one line matches a condition Here are my ...
5 votes
3 answers
229 views

Processing a continuous single line of data with stream processing in bash pipeline?

I am debugging an embedded server that outputs a continuous single line of text to a specified network port. There is no newline anywhere in the stream but it is text data and I would like to format ...
  • 193
4 votes
5 answers
577 views

How can I transform dates expressed with month names to numerical month designations?

I use MacOS and I have variable date in format 3.Jan.2023, 12.Nov.2017, 9.Apr.2022, ... I need to change in 03.01.2023, 12.11.2017, 09.04.2022, ...
  • 43
1 vote
2 answers
76 views

Match date pattern that precedes SED range match and include the matching range in output

Having issues with trying to set variable (example: str_pattern) with multiple values and then using those in sed command to match range in reverse to previous blank line or back to ExecuteThread. ...
0 votes
2 answers
47 views

how to use grep to only extract a certain column from a linux command output

I want to use grep to only see a certain column from the output of a Linux command. When I use this command, I get the following output: $ ps ax | grep sshd | grep 'priv' | sort -k 3 4886 ? ...
  • 3
0 votes
5 answers
82 views

How to bring the next line to the end of the first line using awk, separated by a comma?

I have downloaded some sequences from a publicly available database in .fa format. I want to generate a .csv file that contains the name of the sequence, and its length, separated by a comma. This is ...
-5 votes
1 answer
33 views

Read line from 2 files and print values of both files in one line shell script

Need help for unix shell script. I have 2 files: file1: abc def ghi file2: 123 456 789 I need output as below: abc 123 def 456 ghi 789 Request you to please help asap
-1 votes
6 answers
107 views

Output everything before the first slash '/' in a line [duplicate]

I have a set of URLs and I'm really only interested in anything up to the first /. How can I capture this info to a text file? Input (foo.txt): apple.com/nothing.js t1.msn.com/cookie=22 happy.net/...
  • 1
0 votes
2 answers
107 views

Replace multiple lines and words using sed in a for loop

I have 2 files (mac and uid) with data on a Linux machine, that I want to carry to a "template", with a list, using sed to replace the data with variables. list.txt: CP00 CP01 mac.txt: FF-...
0 votes
0 answers
49 views

Surrounding line with quotes using sed in WSL2 Ubuntu not working as expected

The command: sed -e 's/.*/\"&\",/' I tested the regex at https://sed.js.org/ and it worked correctly, so if I had input like this: George Washington, 1789-1797 John Adams, 1797-1801 ...
0 votes
1 answer
74 views

Ensuring begin and end sections

I have files with sections defined by the starting section composed oy TITLE, SUBTITLE, followed by comma separated KEYWORD. ## TITLE [SUBTITLE] KEYWORD,KEYWORD The ending is done using ## END OF ...
  • 13
4 votes
2 answers
140 views

How to select rows according to the frequency of elements in columns

I have a file, as you can see, which is tab-separated data, about 4,000 lines, ten columns. The second column of the file records the different tissues. samples tissue_s tissue_e tissue_d ...
  • 53
0 votes
3 answers
81 views

how to change a word in a specific group using sed?

I need to change a word in a specific define service section where service_description is NRPE - NTP_TIME and where contact_groups is opsgenie-sla1, I need to change to opsgenie-sla2 and I would like ...
1 vote
3 answers
56 views

How do I pick only one record per ID based on the earliest date in another column?

I have a file containing multiple lines, with fields separated by tab: ID Code Date 1 XX 23/1/2018 1 XX 11/3/2021 2 XX 14/5/2011 2 XX 20/9/2013 3 XX 08/7/2014 3 XX 11/9/2016 3 XX ...
  • 15
-2 votes
3 answers
141 views

How to combine cat, grep, awk and sed commands?

When I do enter those commands individualy, I get the results I want: cat /var/log/apt/history.log | grep 'apt install' > installation.txt grep -v "nvidia" installation.txt > tmpfile &...
user avatar
0 votes
1 answer
52 views

Text processing - Modify based off two different files

I have two files named one.txt containing "297","237","14",NULL,"51.195.14.201","02:00:00:1a:1c:06","255.0.0.0","10.0.10.1","...
  • 2,793
0 votes
3 answers
178 views

How can I test if a string has any ASCII white space characters in it?

How can I check if a string has any whitespace characters in it? I do not have to worry about things outside of ASCII for example unicode zero width characters etc. You can assume that the string is ...

1
2 3 4 5
165