Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
7,920
questions
1
vote
1
answer
27
views
How to retrieve data from a logfile between timestamps in a bash script
I have a logfile that I am trying to 'grep' data out of via a bash script. The data I am specifically after is all lines between two timestamps (including the top timestamp) that have the pattern &...
- 11
2
votes
1
answer
65
views
bash how to concat multiple json files to one
I have thousands of json files that look like:
file1 (key1:value_list1)
{"2mac:acg":["1-248","3-245","3-246","4-245","4-246","5-245&...
- 155
2
votes
3
answers
130
views
How to get value after match with or without space?
I have a bash script which has an array of database names, connect to each one of them and run an stored procedure on them. I need to get values from stored procedure's output and use them at rest of ...
- 726
0
votes
1
answer
27
views
checked the matched content from file one with another and find % of matched
I want to match the content from two files, both the files contain some IDs and calculate the percentage of matched and unmatched.
The first file look like:
rs10000012
rs1000005
rs10000202
rs1000022
...
1
vote
1
answer
86
views
Extract identifiers and corresponding brackets among overlapped brackets
I have some files contained identifiers as below:
B#205918
A#273075
E#554065
Eg. sample of file1:
((((A#273075,A#273116),((A#224325,A#192952),A#243232)),(((E#7955,E#7165),E#6239),E#4530)),(((((E#3075,...
- 177
0
votes
2
answers
87
views
Finding values of one file within range of another file and selecting the top value
I have two files A and B. File-A has 4 columns and 600,000 rows. File-B has 4 columns and 5000 rows. Example:
File-A:
ENSB1 1 12245 0.53 0.002
ENSB2 1 13400 0.27 0.0003
ENSB3 1 14780 0.13 0.00001
...
-1
votes
3
answers
93
views
Appending a character to field records based on a repeated value from the second field
I have this input
array='2 0.00000 -1.45191
6 0.81778 -0.63413
10 0.85020 -0.60170
8 1.40260 -0.04931
22 3.25781 1.80590
20 4....
- 13
0
votes
2
answers
41
views
Get values from pipe delimited file to the normal text file in Linux
I have a pipe delimited file
Lookup.txt:
Column1|yellow|value|628291
Column9|red|value|346801
File.txt:
Yellow
YellowDiamond
Redgold
I am trying to get the value of Yellow present In File.txt to the ...
- 103
0
votes
3
answers
84
views
How to replace a field by a shell variable being another field?
Consider the following code:
renamed_column='2
6
10
8
22
20
6-
18
8-
12
16'
array1='2 0.00000 -1.45191
6 0.81778 -0.63413
10 0.85020 -0.60170
8 1.40260 -0.04931
22 3.25781 1.80590
20 4.32051 2.86860
...
- 13
2
votes
1
answer
63
views
Global search + replace, but only in cases where a test passes
I want to do a global search and replace on a directory of files. There are various ways to do this (many editors, sed+find), but I have an extra requirement. I want, for each possible replacement to ...
- 123
1
vote
2
answers
75
views
How to match all lines sharing same words
I'm using integrated search feature with in Visual Studio Code. In search criteria there is regEx way of searching things.
I'm trying to search complete CSV (using Visual Studio Code IDE), exact match ...
- 25
-1
votes
1
answer
41
views
return a list using grep (or alternative) [closed]
I am trying to return a list of "Name", "Exec" and "Icon" fields in /usr/share/applications/*.desktop
I run:
grep Icon= /usr/share/applications/*.desktop | cut -d '=' -f ...
- 1
2
votes
4
answers
78
views
How to extract specific words out of a file
If I have a file that looks something like this eg called myfile.log
1 entry1 # Blah blah
2 entry2 # Blah blah
3 entry3 # Blah blah
4 entry4 # Blah blah
5 myentry5 # ...
- 23
0
votes
1
answer
43
views
How can I batch rename files by combining three variables and ignore file extensions?
I'm trying to create a script that renames a group of files. Normally they will be TIFs but sometimes an occasional oddball could be thrown in so I would like to rename the file and ignore the file ...
0
votes
2
answers
62
views
Change text using bash
I have a program called gamemode, and to check if it's running, you can use gamemode -s, which outputs either "gamemode is active" or "gamemode is inactive".
I want to change "...
- 1
2
votes
1
answer
45
views
Change the printed memory usage from dot to comma and then calculate the memory usage in %
So far, in a previously asked question: Calculate the real memory usage (memtotal - memfree - cached - bufferd) with /proc/meminfo one-liner
With many thanks to @Philippos this command prints the ...
- 31
4
votes
2
answers
415
views
cut /etc/passwd but without colons (:) in the resulting set of fields
I want to list the username, id and group on /etc/passwd using the following format: username uid gid
I have used the following:
cut -d: -f1,3,4 /etc/passwd
But it returns username:uid:gid. How can I ...
2
votes
2
answers
153
views
Add new lines based on the columns of a tab delimited file
I have a tab-separated file like this:
211845 032
215979 002 071
217783 143 156 169
219750 111
For the lines that have multiple tab separated entries, I want to add new lines based on the ...
- 21
1
vote
2
answers
46
views
awk script replace value with increment with if
I have a pipe delimted file
d1000|1000
d1001|100
d1002|10
d1003|1
d1004|
d1005|
I want to modify $2 if length is less than 4 digits, and also keep empty spaces as is
so trying to do it via awk script
...
- 1,631
2
votes
1
answer
65
views
Why does paste command truncate one of the input files?
I have two files which contain only one column of numeric data each, and the same number of rows. When using paste, it does combine the rows from the two files into one row, but the text of the first ...
- 353
0
votes
3
answers
85
views
awk print only modified lines
I have file like this
d1000 1000
d1001 100
d1002 10
d1003 1
I want to modify second column where length is not equal 4.
But I want to print only lines that are modified, so original text in column 2 ...
- 1,631
2
votes
2
answers
86
views
Replace matched regex in file content with variables
I would like to do the following
Search file content with pattern (^[ \t]*services:[ \t]*)
If found, replace the pattern by a String (in variable)
Else, append the String (in variable) at end of file
...
- 123
2
votes
3
answers
115
views
awk/sed split a cluster file in to multiple files
I have a cluster fasta file (called file) which looks like:
>1AB2
>1AB2 AA
NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI
NCIDHFR8EHGBVPIWOBGIGRI
>1AB3 AA
WNIOREHUEBRGOUERGHBERGIORBGREUGEGO
...
- 155
0
votes
3
answers
34
views
change value of one column with condition from another column
I have a series of .txt files with 6 columns like this :
-44.2584 0.2603 42.7879 6 0.1 Precentral_L
49.3816 5.3947 40.4102 6 0.1 Precentral_R
-22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L
26.0365 ...
2
votes
2
answers
101
views
replacement for tr with utf-8 capabilities
in order to isolate the last word in any line of a poem (to have a list of all rhymes), I put together several snippets of code obtaining this
awk '{print $NF}' input.txt | tr 'A-Z' 'a-z' | tr -sc 'a-...
- 297
2
votes
2
answers
91
views
Replace digits and then characters pattern with sed
I have a file like below:
some
arbitrary number of
leading
lines
a prefix followed by wmd v0.0.0-20220406135915-ce5e3ee6c6bf
some
trailing
lines
This is only an example of what the file could look ...
- 413
3
votes
6
answers
327
views
Swap lines in a file
I've a file like this:
DelegatedAdmin: cn=fb6190f7-4a33-4502-a2d5-dce8dc434570,ou=Named,ou=Identities,ou=Active,ou=Vault<br />
changetype: modify<br />
delete: DelegatedAdmin<br />
...
- 31
-2
votes
3
answers
46
views
all possible matched pairs and associated information
I have a txt file as input in which there are 3 columns divided by "," (ID, info1 and info2). I'd like to obtained an output in which there are all possible unique matched pairs based on the ...
- 161
-1
votes
2
answers
41
views
Extracting data from a file with a regular expression in sed
How can I use sed to extract f607b9ab-1452-4758-b047-6b620591211b by using cmrepo-222-02 as my reference?
["+--------------------------------------+---------------+----------+------+-------------+...
1
vote
4
answers
54
views
Split the line into next line after a matching pattern
In the input file shown below, I am trying to find the string job_type and move the it to the next line if it matches job_type.
I tried this, but it is not working:
sed "s/[A-Z][a-z]*job_type:/\...
- 183
-3
votes
3
answers
96
views
how to ignore negative numbers
let's consider the following example
1 4.0 3.5 7.2
2 3.2 2.5 5.2
3 1.7 -1.8 2.9
4 4.1 2.5 5.7
How can I extract the 3rd column of the table by ignoring the negative number?
If I use awk '{print $3}' ...
- 19
1
vote
1
answer
37
views
Check info in each line on a specific set of columns in one variable against all lines in three specific columns in another variable using awk
I've asked a very similar question today before, however I have since realised that I need to increase parameters for the command. I edited the command for a another parameter okay, but the next ...
- 801
1
vote
1
answer
31
views
Extract field and number of occurrences per line
I have this file:
John Green', 'Age: 32', 'State: New York', 'Total cars: 2', 'Manufacter: General Motor', 'Model: Pontiac', 'Year: 2000', 'Manufacter: Ford Motor', 'Model: Endeavour', 'Year: 2010
...
- 1,116
1
vote
1
answer
81
views
Substitution in html file using awk
My awk script reads records from file1, finds the same records in file2 and substitutes alternate positions (of the record) with a defined symbol in that. But few of the values are not getting ...
- 11
0
votes
1
answer
29
views
How to map data based on substring?
I have data in the following manner:
staging_uw_pc_account_contact_role_hive_tb
staging_uw_pc_account_hive_tb
staging_uw_pc_account_location_hive_tb
uw_pc_account_contact_hive_tb
...
- 103
2
votes
2
answers
81
views
Check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk
I need to check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk, keeping lines in the first variable that meet ...
- 801
0
votes
1
answer
37
views
Trying to do a perfect sed replacement
I am trying to do a sed replacement which I am currently doing with two commands. I can't figure out how to combine the two commands together so that it replaces the words correctly in one go.
My role ...
-2
votes
3
answers
61
views
How to remove forward slashes from file in unix shell using sed? [duplicate]
I have the following file :
/
/FUSE
//
Bank S.A / N.V
/F
I want to remove all forward slashes (/), so that the result looks like this:
FUSE
Bank S.A N.V
F
How to do so using sed? Ive tried using ...
- 11
2
votes
2
answers
57
views
awk replace column value with increment values if length equals
I have non-standard data, which I'd like to standardise
file:
d101 11001
e101 9665
f101 9663
d102 11002
e102 11003
f102 11004
g102 11005
desired output:
d101 11001
e101 12001
f101 12002
d102 11002
...
- 1,631
0
votes
1
answer
44
views
Calculate the real memory usage (memtotal - memfree - cached - bufferd) with /proc/meminfo one-liner
I'm trying to calculate the real memory usage for my MOTD and also front OLED display in bash.
Emberassingly I didn't convert the numbers from kB properly to MB. But know that the formula from /proc/...
- 31
0
votes
2
answers
66
views
How to replace all occurrences of a sequence starting with a given string to the next appearance of a symbol?
I would like to edit a huge single line file with thousands of occurrences of some string 'string_string': until the next appearance of a comma , (including this comma) and removing those occurrences ...
- 171
1
vote
1
answer
58
views
How can I use `find` to find the files and awk to search within the files?
In a folder and its subfolders, I want to search a pattern. For this I am using: grep+cut+awk+tr pipeline.
printf "I am so wide alper world zorro the end\t\t\talper\n" > alper.txt
...
- 230
2
votes
1
answer
44
views
How to untag, in an XML file, a chain of tagged characters that are embedded within specified tags
What is the xmlstarlet command to untag, in an XML file, a chain of tagged characters that are embedded within specified tags?
Example: searching every occurrence of the tag <b>, that needs to ...
- 77
0
votes
2
answers
78
views
sed command to fix concatenating two string literals
I'm looking for a sed (or other Unix utility) command to find all instances of two string literals being concatenated, such that the command will replace these concatenations with the corresponding ...
- 413
0
votes
4
answers
94
views
Switch the last two fields with awk
I'm looking for a way to switch the last two fields of a tab-separated file. The number of fields vary among rows, so I can't specify the field number.
Most of what I could find online was field-...
- 559
3
votes
1
answer
101
views
How to parse a string containing multiple hyphens and/or whitespaces for line-by-line processing with grep etc?
I'm working with auditd rules on RHEL 7 and 8. Considering these example files...
file2.txt:
-a always,exit -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete
-...
- 33
3
votes
1
answer
103
views
How to rename, in an XML file, specific tags that are embedded in some other specific tags
What is the xmlstarlet command to replace a tag value by another one, only for specified tags embedded in other specified tags?
Example: searching every occurance of the tag b, that needs to be ...
- 77
12
votes
8
answers
2k
views
How can I replace a character within a specific context in each line of the whole file?
I have a large file which contains hundreds of English phrases in the following form:
\phrase
{. . . * * }
{I shoul-d've stayed home.}
{aɪ ʃʊd‿əv ˈsteɪd ˈhoʊm.} <- only replace on ...
- 251
3
votes
6
answers
484
views
Convert the first letter of all "."-separated parts of a filename to uppercase except for the last one in Mac Bash
I want to change the filename in the Mac Bash to initial capital after every dot/period, except for the filename extension.
Input string example:
one.two.three.four.txt
Desired output:
One.Two.Three....
0
votes
2
answers
68
views
Nushell: Convert list to table
Is there an idiomatic way to convert a list of similar records into a table in Nushell?
I'm working with the Stack Exchange API and getting results back something like:
let questions = ('[
{
&...
- 2,300