2022 Developer Survey is open! Take survey.

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
1 vote
1 answer
27 views

How to retrieve data from a logfile between timestamps in a bash script

I have a logfile that I am trying to 'grep' data out of via a bash script. The data I am specifically after is all lines between two timestamps (including the top timestamp) that have the pattern &...
user avatar
2 votes
1 answer
65 views

bash how to concat multiple json files to one

I have thousands of json files that look like: file1 (key1:value_list1) {"2mac:acg":["1-248","3-245","3-246","4-245","4-246","5-245&...
user avatar
  • 155
2 votes
3 answers
130 views

How to get value after match with or without space?

I have a bash script which has an array of database names, connect to each one of them and run an stored procedure on them. I need to get values from stored procedure's output and use them at rest of ...
user avatar
0 votes
1 answer
27 views

checked the matched content from file one with another and find % of matched

I want to match the content from two files, both the files contain some IDs and calculate the percentage of matched and unmatched. The first file look like: rs10000012 rs1000005 rs10000202 rs1000022 ...
user avatar
1 vote
1 answer
86 views

Extract identifiers and corresponding brackets among overlapped brackets

I have some files contained identifiers as below: B#205918 A#273075 E#554065 Eg. sample of file1: ((((A#273075,A#273116),((A#224325,A#192952),A#243232)),(((E#7955,E#7165),E#6239),E#4530)),(((((E#3075,...
user avatar
  • 177
0 votes
2 answers
87 views

Finding values of one file within range of another file and selecting the top value

I have two files A and B. File-A has 4 columns and 600,000 rows. File-B has 4 columns and 5000 rows. Example: File-A: ENSB1 1 12245 0.53 0.002 ENSB2 1 13400 0.27 0.0003 ENSB3 1 14780 0.13 0.00001 ...
user avatar
-1 votes
3 answers
93 views

Appending a character to field records based on a repeated value from the second field

I have this input array='2 0.00000 -1.45191 6 0.81778 -0.63413 10 0.85020 -0.60170 8 1.40260 -0.04931 22 3.25781 1.80590 20 4....
user avatar
0 votes
2 answers
41 views

Get values from pipe delimited file to the normal text file in Linux

I have a pipe delimited file Lookup.txt: Column1|yellow|value|628291 Column9|red|value|346801 File.txt: Yellow YellowDiamond Redgold I am trying to get the value of Yellow present In File.txt to the ...
user avatar
0 votes
3 answers
84 views

How to replace a field by a shell variable being another field?

Consider the following code: renamed_column='2 6 10 8 22 20 6- 18 8- 12 16' array1='2 0.00000 -1.45191 6 0.81778 -0.63413 10 0.85020 -0.60170 8 1.40260 -0.04931 22 3.25781 1.80590 20 4.32051 2.86860 ...
user avatar
2 votes
1 answer
63 views

Global search + replace, but only in cases where a test passes

I want to do a global search and replace on a directory of files. There are various ways to do this (many editors, sed+find), but I have an extra requirement. I want, for each possible replacement to ...
user avatar
1 vote
2 answers
75 views

How to match all lines sharing same words

I'm using integrated search feature with in Visual Studio Code. In search criteria there is regEx way of searching things. I'm trying to search complete CSV (using Visual Studio Code IDE), exact match ...
user avatar
-1 votes
1 answer
41 views

return a list using grep (or alternative) [closed]

I am trying to return a list of "Name", "Exec" and "Icon" fields in /usr/share/applications/*.desktop I run: grep Icon= /usr/share/applications/*.desktop | cut -d '=' -f ...
user avatar
2 votes
4 answers
78 views

How to extract specific words out of a file

If I have a file that looks something like this eg called myfile.log 1 entry1 # Blah blah 2 entry2 # Blah blah 3 entry3 # Blah blah 4 entry4 # Blah blah 5 myentry5 # ...
user avatar
0 votes
1 answer
43 views

How can I batch rename files by combining three variables and ignore file extensions?

I'm trying to create a script that renames a group of files. Normally they will be TIFs but sometimes an occasional oddball could be thrown in so I would like to rename the file and ignore the file ...
user avatar
0 votes
2 answers
62 views

Change text using bash

I have a program called gamemode, and to check if it's running, you can use gamemode -s, which outputs either "gamemode is active" or "gamemode is inactive". I want to change "...
user avatar
  • 1
2 votes
1 answer
45 views

Change the printed memory usage from dot to comma and then calculate the memory usage in %

So far, in a previously asked question: Calculate the real memory usage (memtotal - memfree - cached - bufferd) with /proc/meminfo one-liner With many thanks to @Philippos this command prints the ...
user avatar
  • 31
4 votes
2 answers
415 views

cut /etc/passwd but without colons (:) in the resulting set of fields

I want to list the username, id and group on /etc/passwd using the following format: username uid gid I have used the following: cut -d: -f1,3,4 /etc/passwd But it returns username:uid:gid. How can I ...
user avatar
2 votes
2 answers
153 views

Add new lines based on the columns of a tab delimited file

I have a tab-separated file like this: 211845 032 215979 002 071 217783 143 156 169 219750 111 For the lines that have multiple tab separated entries, I want to add new lines based on the ...
user avatar
  • 21
1 vote
2 answers
46 views

awk script replace value with increment with if

I have a pipe delimted file d1000|1000 d1001|100 d1002|10 d1003|1 d1004| d1005| I want to modify $2 if length is less than 4 digits, and also keep empty spaces as is so trying to do it via awk script ...
user avatar
  • 1,631
2 votes
1 answer
65 views

Why does paste command truncate one of the input files?

I have two files which contain only one column of numeric data each, and the same number of rows. When using paste, it does combine the rows from the two files into one row, but the text of the first ...
user avatar
  • 353
0 votes
3 answers
85 views

awk print only modified lines

I have file like this d1000 1000 d1001 100 d1002 10 d1003 1 I want to modify second column where length is not equal 4. But I want to print only lines that are modified, so original text in column 2 ...
user avatar
  • 1,631
2 votes
2 answers
86 views

Replace matched regex in file content with variables

I would like to do the following Search file content with pattern (^[ \t]*services:[ \t]*) If found, replace the pattern by a String (in variable) Else, append the String (in variable) at end of file ...
user avatar
  • 123
2 votes
3 answers
115 views

awk/sed split a cluster file in to multiple files

I have a cluster fasta file (called file) which looks like: >1AB2 >1AB2 AA NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI NCIDHFR8EHGBVPIWOBGIGRI >1AB3 AA WNIOREHUEBRGOUERGHBERGIORBGREUGEGO ...
user avatar
  • 155
0 votes
3 answers
34 views

change value of one column with condition from another column

I have a series of .txt files with 6 columns like this : -44.2584 0.2603 42.7879 6 0.1 Precentral_L 49.3816 5.3947 40.4102 6 0.1 Precentral_R -22.5897 9.5277 54.8691 6 0.1 Frontal-Sup_L 26.0365 ...
user avatar
2 votes
2 answers
101 views

replacement for tr with utf-8 capabilities

in order to isolate the last word in any line of a poem (to have a list of all rhymes), I put together several snippets of code obtaining this awk '{print $NF}' input.txt | tr 'A-Z' 'a-z' | tr -sc 'a-...
user avatar
  • 297
2 votes
2 answers
91 views

Replace digits and then characters pattern with sed

I have a file like below: some arbitrary number of leading lines a prefix followed by wmd v0.0.0-20220406135915-ce5e3ee6c6bf some trailing lines This is only an example of what the file could look ...
user avatar
3 votes
6 answers
327 views

Swap lines in a file

I've a file like this: DelegatedAdmin: cn=fb6190f7-4a33-4502-a2d5-dce8dc434570,ou=Named,ou=Identities,ou=Active,ou=Vault<br /> changetype: modify<br /> delete: DelegatedAdmin<br /> ...
user avatar
  • 31
-2 votes
3 answers
46 views

all possible matched pairs and associated information

I have a txt file as input in which there are 3 columns divided by "," (ID, info1 and info2). I'd like to obtained an output in which there are all possible unique matched pairs based on the ...
user avatar
  • 161
-1 votes
2 answers
41 views

Extracting data from a file with a regular expression in sed

How can I use sed to extract f607b9ab-1452-4758-b047-6b620591211b by using cmrepo-222-02 as my reference? ["+--------------------------------------+---------------+----------+------+-------------+...
user avatar
1 vote
4 answers
54 views

Split the line into next line after a matching pattern

In the input file shown below, I am trying to find the string job_type and move the it to the next line if it matches job_type. I tried this, but it is not working: sed "s/[A-Z][a-z]*job_type:/\...
user avatar
  • 183
-3 votes
3 answers
96 views

how to ignore negative numbers

let's consider the following example 1 4.0 3.5 7.2 2 3.2 2.5 5.2 3 1.7 -1.8 2.9 4 4.1 2.5 5.7 How can I extract the 3rd column of the table by ignoring the negative number? If I use awk '{print $3}' ...
user avatar
1 vote
1 answer
37 views

Check info in each line on a specific set of columns in one variable against all lines in three specific columns in another variable using awk

I've asked a very similar question today before, however I have since realised that I need to increase parameters for the command. I edited the command for a another parameter okay, but the next ...
user avatar
  • 801
1 vote
1 answer
31 views

Extract field and number of occurrences per line

I have this file: John Green', 'Age: 32', 'State: New York', 'Total cars: 2', 'Manufacter: General Motor', 'Model: Pontiac', 'Year: 2000', 'Manufacter: Ford Motor', 'Model: Endeavour', 'Year: 2010 ...
user avatar
1 vote
1 answer
81 views

Substitution in html file using awk

My awk script reads records from file1, finds the same records in file2 and substitutes alternate positions (of the record) with a defined symbol in that. But few of the values are not getting ...
user avatar
0 votes
1 answer
29 views

How to map data based on substring?

I have data in the following manner: staging_uw_pc_account_contact_role_hive_tb staging_uw_pc_account_hive_tb staging_uw_pc_account_location_hive_tb uw_pc_account_contact_hive_tb ...
user avatar
2 votes
2 answers
81 views

Check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk

I need to check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk, keeping lines in the first variable that meet ...
user avatar
  • 801
0 votes
1 answer
37 views

Trying to do a perfect sed replacement

I am trying to do a sed replacement which I am currently doing with two commands. I can't figure out how to combine the two commands together so that it replaces the words correctly in one go. My role ...
user avatar
-2 votes
3 answers
61 views

How to remove forward slashes from file in unix shell using sed? [duplicate]

I have the following file : / /FUSE // Bank S.A / N.V /F I want to remove all forward slashes (/), so that the result looks like this: FUSE Bank S.A N.V F How to do so using sed? Ive tried using ...
user avatar
2 votes
2 answers
57 views

awk replace column value with increment values if length equals

I have non-standard data, which I'd like to standardise file: d101 11001 e101 9665 f101 9663 d102 11002 e102 11003 f102 11004 g102 11005 desired output: d101 11001 e101 12001 f101 12002 d102 11002 ...
user avatar
  • 1,631
0 votes
1 answer
44 views

Calculate the real memory usage (memtotal - memfree - cached - bufferd) with /proc/meminfo one-liner

I'm trying to calculate the real memory usage for my MOTD and also front OLED display in bash. Emberassingly I didn't convert the numbers from kB properly to MB. But know that the formula from /proc/...
user avatar
  • 31
0 votes
2 answers
66 views

How to replace all occurrences of a sequence starting with a given string to the next appearance of a symbol?

I would like to edit a huge single line file with thousands of occurrences of some string 'string_string': until the next appearance of a comma , (including this comma) and removing those occurrences ...
user avatar
  • 171
1 vote
1 answer
58 views

How can I use `find` to find the files and awk to search within the files?

In a folder and its subfolders, I want to search a pattern. For this I am using: grep+cut+awk+tr pipeline. printf "I am so wide alper world zorro the end\t\t\talper\n" > alper.txt ...
user avatar
  • 230
2 votes
1 answer
44 views

How to untag, in an XML file, a chain of tagged characters that are embedded within specified tags

What is the xmlstarlet command to untag, in an XML file, a chain of tagged characters that are embedded within specified tags? Example: searching every occurrence of the tag <b>, that needs to ...
user avatar
  • 77
0 votes
2 answers
78 views

sed command to fix concatenating two string literals

I'm looking for a sed (or other Unix utility) command to find all instances of two string literals being concatenated, such that the command will replace these concatenations with the corresponding ...
user avatar
0 votes
4 answers
94 views

Switch the last two fields with awk

I'm looking for a way to switch the last two fields of a tab-separated file. The number of fields vary among rows, so I can't specify the field number. Most of what I could find online was field-...
user avatar
  • 559
3 votes
1 answer
101 views

How to parse a string containing multiple hyphens and/or whitespaces for line-by-line processing with grep etc?

I'm working with auditd rules on RHEL 7 and 8. Considering these example files... file2.txt: -a always,exit -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete -...
user avatar
3 votes
1 answer
103 views

How to rename, in an XML file, specific tags that are embedded in some other specific tags

What is the xmlstarlet command to replace a tag value by another one, only for specified tags embedded in other specified tags? Example: searching every occurance of the tag b, that needs to be ...
user avatar
  • 77
12 votes
8 answers
2k views

How can I replace a character within a specific context in each line of the whole file?

I have a large file which contains hundreds of English phrases in the following form: \phrase {. . . * * } {I shoul-d've stayed home.} {aɪ ʃʊd‿əv ˈsteɪd ˈhoʊm.} <- only replace on ...
user avatar
3 votes
6 answers
484 views

Convert the first letter of all "."-separated parts of a filename to uppercase except for the last one in Mac Bash

I want to change the filename in the Mac Bash to initial capital after every dot/period, except for the filename extension. Input string example: one.two.three.four.txt Desired output: One.Two.Three....
user avatar
0 votes
2 answers
68 views

Nushell: Convert list to table

Is there an idiomatic way to convert a list of similar records into a table in Nushell? I'm working with the Stack Exchange API and getting results back something like: let questions = ('[ { &...
user avatar
  • 2,300

1
2 3 4 5
159