Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

1
vote
3answers
48 views

find the difference between two files and print it in the proper format

I have two text file "account.txt" and "customer.txt" **account.txt** 876251251 716126181 888281211 666615211 787878787 111212134 **customer.txt** 876251251 716126181 792342108 792332668 666615211 ...
0
votes
0answers
22 views

Rearrange data, transpose (rotate) data from columns to rows [on hold]

Need a way to transpose (rotate) data from columns to rows as follow: I have an output .vcf file with data in the following disposition: Sample1 Sample2 Sample3 n...0|1 0|1 0|1 ...
4
votes
1answer
298 views

How can I add 10 lines from a file (file2) to another one after 2 lines (file1)?

I have two different files separated by Tab. File 1 looks like this: transcr_15824 3.95253441295071 3.99992738843234 3.93880798313547 YML042W 10.3143219248979 10.6898819949325 11....
3
votes
2answers
89 views

Sed replace word except when word is preceded by a specific string

I want to replace all instance of abc with 123 provided it isn't preceded by https://. Code: file=Data.txt Initial="# Start"; orig="abc"; new="123"; Final="$" # Line Number sed -r -e "/${Initial}/...
-1
votes
1answer
25 views

Finding and replacing text when there are custom elements within [duplicate]

I have about 450 html files I need to edit. I have the directory they all are in, so that narrows things down from the perspective of searching. What needs to be changed is a bit of known text, some ...
2
votes
3answers
31 views

Split content of specific columns when specific characters are present in those columns

I have this data tab delimited: ed00011 89 0.12 NA NA No ed0002s 28 0.11 c3(3.1e-1) NA No ed0001 22 0.37 NA ...
0
votes
1answer
58 views

Sed Replace a pattern between a pattern and the end of file

From SED: insert text after the last line? - Unix & Linux Stack Exchange, I learnt that $ if used outside the substitution in sed represents the end of file. This I have tried to use in the ...
1
vote
2answers
62 views

convert tab delimited data to , delimited records with double quoted fields “ ” [on hold]

I want to convert tab delimited csv into " ", add empty space if value is empty FirstName LastName Address1 Address2 City State ZIP John1 ...
4
votes
3answers
332 views

Manipulating text in first column only? [duplicate]

Example Input: foobar@example.com foo@example.com,bar@example.com Example Output: foobar@example.org foo@example.com,bar@example.com So, in "normal" circumstances, it would obviously be easy to do ...
0
votes
2answers
38 views

Unix script to join data from different rows into one single row

We have a large tsv file where the data for a single row are splitted into different rows with new line delimiter. We need to join them together based on the tab counts. For eg: If suppose the total ...
0
votes
2answers
42 views

Gnu Sed: Space Character class not matching for end of line character

Pattern: abc followed by space or is the last character in a line. orig="abc[[:space:]]+"; new=Hello sed -i -r -e "s|${orig}|${new}|g" ./file ; File: abc abcd abc abc d After executing: abc abcd ...
0
votes
1answer
62 views

Grep is printing “grep:” - How to ignore it [on hold]

I am looking to get port number in below filesunder directory Variable. When the file's port line is empty, I am getting grep: printed out. Is there a way to ignore it, and Print nothing? if ls $...
1
vote
2answers
95 views

How to replace a string when another string is found in a line?

I want to replace a string 'xy' with another string 'ab' when string 'ny' is found in a line of a unix file. Sample text: If we have xy today we can go to ny. If we have xy tomorrow we can go to ny ...
2
votes
2answers
113 views

How to break line after matching case but needs to go back a couple of characters

I have the following sample: 1. TNT 00:00 2. Swing From The Gutters 7:34 3. Ten-Day Interval 13:29 4. I Set My Face To The Hillside 18:16 5. The Equator 24:26 6. A Simple Way To Go Faster Than Light ...
0
votes
2answers
27 views

Bash: compare target date to series of dates

Given a target date and a set of two columns (the first being a random number and the second a date in order from most recent to oldest), I want to find the number for the row with the date closest to ...
0
votes
2answers
42 views

Pick off 'first' dates using cal function in bash

I'm trying to pick off the first Monday, second Tuesday, second Saturday, etc. out of the cal function and piping awk commands. For this month, I can get the first Tuesday correctly with the command ...
0
votes
1answer
23 views

shell script /unix commands for text file content editing [duplicate]

I want to edit the file contents in the 2nd line and 2nd column ,which is a file name from that I want to remove the part after the last underscore and before the file extension . The contents of the ...
1
vote
5answers
73 views

Remove text between consecutive occurences of strings

I need to remove all text between two given strings in a text file. The strings may be located on different lines. For instance, in the following text file @article{ginsberg_lifespan_2018, title =...
0
votes
2answers
49 views

Want to edit a file content for Removing the string after the last underscore and before the file extension using Unix commands

I want to edit the file contents in the 2nd row and 2nd column ,which is a file name from that I want to remove the part after the last underscore and before the file extension . The contents of the ...
-1
votes
0answers
19 views

Linux multi-file multi-line text search and replace regex-supported GUI tool

Can you recommend Linux tool (prefer with GUI) with possibility multi-file (recursively with sub-folders) and multi-line search and replace text in files? Something like regexxer but with ...
0
votes
1answer
23 views

awk or bash to read IDs from CSV, match rows in second CSV, change value in 2nd column

I have two .csv items-to-change.txt is a single column list of unique ids: 218294 222084 197931 198161 197432 201021 214009 items-by-location.txt has unique ids and location number (tab delimited) ...
0
votes
4answers
42 views

How to extract word which has only numbers separated by 'dot'

From below lines abcd efgh ijhk lmn opqrs 9.0.8.2c tuv wxyz abcd efgh ijhk lmn opqrs 8.1.3.9b How to extract only 9.0.8.2c 8.1.3.9b
0
votes
0answers
25 views

How to remove all comments from a javascript file using common Linux command line tools?

I have a javascript file with deeply needed comments. Javascript uses C style comments: //this is a single line comment And /* this is a multi-line comment */ However, the actual code looks ...
0
votes
0answers
15 views

Text processing - Regex search for string and output unique values [duplicate]

I have a use case where I would like to search the following text for any Windows AD objects in backslash format. Here is a use case to use as an example. My goal is to output the unique results to a ...
-2
votes
3answers
41 views

sed/awk string replacement with newline and tabs

I have an XML file that contains the following: <ipaddr>192.168.1.1</ipaddr> <subnet>24</subnet> The goal is to modify <ipaddr>192.168.1.1</ipaddr> to <...
0
votes
3answers
30 views

Get the last element of a row, separated with ;

I have a file (test.txt), where the separator is ;. How can I write out the last element from each row? (the length of each element is different)
0
votes
3answers
33 views

How can I parse the output of `ps` to different fields?

I would like to parse the output of ps, to get any field using awk. It seems that ps separate the fields by whitespaces, but the values in COMMAND field usually have whitespaces. Is there a way to ...
0
votes
1answer
27 views

sed command not giving proper output for regular expression used

I am trying to fetch data between a range of time intervals from the log file. However, the complete data is not fetched for the end date. Below is the command used: sed -n '/26/Nov/2018/,/30/Nov/...
4
votes
3answers
156 views

Add lines to files to make them equal length

I have a bunch of .csv files with N columns and different number of rows (lines). I would like to add as many empty lines ;...; (N semicolons) to make them the same length. I can get the length of the ...
0
votes
1answer
19 views

How to replace an entire paragraph in a file?

I want to change this: ... %%%23begin aaaaaaa \\ bbbbbbb \\ ccccccc \\ %%%23end ... to this ... %%%23begin xxxxxxxx \\ yyyyyyyy \\ zzzzzzzzz \\ tttttttt \\ %%%23end ... the number ...
1
vote
3answers
65 views

How to switch first name with last name in this situation?

I'm given a file with many lines like so: Vinh Tranh:438-910-7449:8235 Maple Street, Wilmington, VM 29085:9/23/63:68900 I need to change the order from Firstname Lastname to Lastname, Firstname. Ive ...
0
votes
2answers
12 views

Git: keep color of git stash list --pretty=<..> through a pipe

I have this command: git stash list --pretty="%C(yellow)%h %>(14)%Cgreen%cr %C(blue)%gs" gives output colored, but if I pipe it git stash list <..> | less -R, color disappear in output. ...
-1
votes
1answer
39 views

how to add words before line on all the scripts in current folder

under /home/scrript , we have 187 bash scripts all the scripts have the following line: kill -PIPE $PID we want to add the following words before this line on all the scripts as the following [[ $...
-1
votes
1answer
46 views

Remove lines in file

Trying to find lines that are greater than 59 but do not include any spaces. E.g., a1e4fa338aae3e87250f4dbff518b8d72a004201395996a8474a252d25: found a1e4fa f38aae3e8725 0f7dbff518b ...
0
votes
1answer
44 views

Compare 2 files and print matching values to a new file

I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";". File1 Example: AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH Each line has a unique ID in $7. I ...
0
votes
1answer
22 views

Slicing JSON with JQ to Create Arrays for New Relic Ingestion

sorry for the long title. I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards. I am using a bash ...
1
vote
2answers
50 views

How can I get less to display \n as a newline?

I have some files containing \n newlines that I am viewing with less. I would like to be able to have less display those newlines as actual newlines. For example an array: Array\n(\n [Color] =&...
-1
votes
2answers
40 views

jq: colorize selected field of json file [closed]

Extend of this question: jq: parse json file with constraint from other field After select "name" field I'd like to colorize it: { "checksum": "9e44bb7b76d8c39c45420dd2158a4521", "roots": { ...
0
votes
1answer
20 views

jq: parse json file with constraint from other field [duplicate]

Extend of this question: How to combine strings from JSON values, keeping only part of the string? Above link's output also include the "name" of folder type, need to exclude these "type": ...
-1
votes
1answer
27 views

Truncate long line and append indicating symbol [closed]

I have sample: "this is a long title must in a doule quotes" this_is_a_solid_long_line_without_space_or_tabs "this is a long title must in a doule quotes" is a long line in double quotes that ...
2
votes
3answers
60 views

Splitting each row of a correlation matrix into individual files

I have a correlation matrix of 22000 genes and for some analysis, I need to split each row of the matrix into a new file. Which means I need to create 22000 individual files. I don't want to use the ...
2
votes
2answers
62 views

Remove trailing commas from invalid json (to make it valid)

Let's say I have a file as below { "fruit": "Apple", } I want to remove the comma at the end of the line, if and only if the next line contains "}". So, the output will be : { "fruit": "...
1
vote
6answers
70 views

How to find word and change only next second line word

The file name staff.txt and sample contents are: JHON MANAGER 10000 I want to find JHON in a file and I want to change whatever's in the 2nd line after that one with another word/number. How can I ...
2
votes
1answer
229 views

How to combine strings from JSON values, keeping only part of the string?

I have sample: "name": "The title of website", "sync_transaction_version": "1", "type": "url", "url": "https://url_of_website" I want to get the following ...
2
votes
1answer
23 views

Layout tab/spaces [closed]

I've read this thread https://unix.stackexchange.com/a/7718/256195, that only if var doesn't contain any tab/spaces but in my case it does contain spaces, like below example: "this is a test" ...
0
votes
3answers
76 views

How do i swap the first word of a line with the last word in a line

I was trying to replace the first word in a line with the last word in a line using a sed command i=but i can't seem to figure it out.
0
votes
1answer
24 views

Taking two integer arguments (the first smaller than the second) print out lines between the two integer arguments

I have a list of the most common passwords and their frequencies . It is a long list and it is formatted as such: 59462 password 49992 iloveyou 33919 princess ... 30 yahoo123 To be clear, the ...
0
votes
2answers
28 views

Replace @@@ and NotApplicable string and redirect output to other file? [duplicate]

I'm using IBM AIX which doesn't have much support like sed -i and sed with \t is not working in my case. I would like to replace replace NotApplicable string with a single space ' ' then replace @@@ ...
0
votes
1answer
47 views

Insert new text in the middle of a file replacing old text without re-inserting the same things over and over?

I have a file that I want to update relatively frequently. What needs to be done is that a specific part of the file would be changed to include some more lines. I am doing it with sed and \a option: ...
5
votes
2answers
151 views

cutting a field to length with awk/sed [duplicate]

I have a rather large bunch of files that contains several fields pipe-delimited. 5595340959340|1|MXPYAQWE|870569689456954654|0|0|20181018224126|1212121212121212121212121212|2|0|1000|70|33107||1|...