Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
1
vote
3answers
48 views
find the difference between two files and print it in the proper format
I have two text file "account.txt" and "customer.txt"
**account.txt**
876251251
716126181
888281211
666615211
787878787
111212134
**customer.txt**
876251251
716126181
792342108
792332668
666615211
...
0
votes
0answers
22 views
Rearrange data, transpose (rotate) data from columns to rows [on hold]
Need a way to transpose (rotate) data from columns to rows as follow:
I have an output .vcf file with data in the following disposition:
Sample1 Sample2 Sample3 n...0|1 0|1 0|1 ...
4
votes
1answer
298 views
How can I add 10 lines from a file (file2) to another one after 2 lines (file1)?
I have two different files separated by Tab. File 1 looks like this:
transcr_15824 3.95253441295071 3.99992738843234 3.93880798313547
YML042W 10.3143219248979 10.6898819949325 11....
3
votes
2answers
89 views
Sed replace word except when word is preceded by a specific string
I want to replace all instance of abc with 123 provided it isn't preceded by https://.
Code:
file=Data.txt
Initial="# Start";
orig="abc";
new="123";
Final="$" # Line Number
sed -r -e "/${Initial}/...
-1
votes
1answer
25 views
Finding and replacing text when there are custom elements within [duplicate]
I have about 450 html files I need to edit. I have the directory they all are in, so that narrows things down from the perspective of searching.
What needs to be changed is a bit of known text, some ...
2
votes
3answers
31 views
Split content of specific columns when specific characters are present in those columns
I have this data tab delimited:
ed00011 89 0.12 NA NA No
ed0002s 28 0.11 c3(3.1e-1) NA No
ed0001 22 0.37 NA ...
0
votes
1answer
58 views
Sed Replace a pattern between a pattern and the end of file
From SED: insert text after the last line? - Unix & Linux Stack Exchange, I learnt that $ if used outside the substitution in sed represents the end of file. This I have tried to use in the ...
1
vote
2answers
62 views
convert tab delimited data to , delimited records with double quoted fields “ ” [on hold]
I want to convert tab delimited csv into " ", add empty space if value is empty
FirstName LastName Address1 Address2 City State ZIP
John1 ...
4
votes
3answers
332 views
Manipulating text in first column only? [duplicate]
Example Input:
foobar@example.com foo@example.com,bar@example.com
Example Output:
foobar@example.org foo@example.com,bar@example.com
So, in "normal" circumstances, it would obviously be easy to do ...
0
votes
2answers
38 views
Unix script to join data from different rows into one single row
We have a large tsv file where the data for a single row are splitted into different rows with new line delimiter.
We need to join them together based on the tab counts.
For eg: If suppose the total ...
0
votes
2answers
42 views
Gnu Sed: Space Character class not matching for end of line character
Pattern: abc followed by space or is the last character in a line.
orig="abc[[:space:]]+";
new=Hello
sed -i -r -e "s|${orig}|${new}|g" ./file ;
File:
abc
abcd
abc
abc d
After executing:
abc
abcd
...
0
votes
1answer
62 views
Grep is printing “grep:” - How to ignore it [on hold]
I am looking to get port number in below filesunder directory Variable. When the file's port line is empty, I am getting grep: printed out. Is there a way to ignore it, and Print nothing?
if ls $...
1
vote
2answers
95 views
How to replace a string when another string is found in a line?
I want to replace a string 'xy' with another string 'ab' when string 'ny' is found in a line of a unix file.
Sample text:
If we have xy today we can go to ny.
If we have xy tomorrow we can go to ny ...
2
votes
2answers
113 views
How to break line after matching case but needs to go back a couple of characters
I have the following sample:
1. TNT 00:00 2. Swing From The Gutters 7:34 3. Ten-Day Interval 13:29 4. I Set My Face To The Hillside 18:16 5. The Equator 24:26 6. A Simple Way To Go Faster Than Light ...
0
votes
2answers
27 views
Bash: compare target date to series of dates
Given a target date and a set of two columns (the first being a random number and the second a date in order from most recent to oldest), I want to find the number for the row with the date closest to ...
0
votes
2answers
42 views
Pick off 'first' dates using cal function in bash
I'm trying to pick off the first Monday, second Tuesday, second Saturday, etc. out of the cal function and piping awk commands. For this month, I can get the first Tuesday correctly with the command ...
0
votes
1answer
23 views
shell script /unix commands for text file content editing [duplicate]
I want to edit the file contents in the 2nd line and 2nd column ,which is a file name from that I want to remove the part after the last underscore and before the file extension .
The contents of the ...
1
vote
5answers
73 views
Remove text between consecutive occurences of strings
I need to remove all text between two given strings in a text file. The strings may be located on different lines. For instance, in the following text file
@article{ginsberg_lifespan_2018,
title =...
0
votes
2answers
49 views
Want to edit a file content for Removing the string after the last underscore and before the file extension using Unix commands
I want to edit the file contents in the 2nd row and 2nd column ,which is a file name from that I want to remove the part after the last underscore and before the file extension .
The contents of the ...
-1
votes
0answers
19 views
Linux multi-file multi-line text search and replace regex-supported GUI tool
Can you recommend Linux tool (prefer with GUI) with possibility multi-file (recursively with sub-folders) and multi-line search and replace text in files?
Something like regexxer but with ...
0
votes
1answer
23 views
awk or bash to read IDs from CSV, match rows in second CSV, change value in 2nd column
I have two .csv
items-to-change.txt is a single column list of unique ids:
218294
222084
197931
198161
197432
201021
214009
items-by-location.txt has unique ids and location number (tab delimited)
...
0
votes
4answers
42 views
How to extract word which has only numbers separated by 'dot'
From below lines
abcd efgh ijhk lmn opqrs 9.0.8.2c tuv wxyz
abcd efgh ijhk lmn opqrs 8.1.3.9b
How to extract only
9.0.8.2c
8.1.3.9b
0
votes
0answers
25 views
How to remove all comments from a javascript file using common Linux command line tools?
I have a javascript file with deeply needed comments. Javascript uses C style comments:
//this is a single line comment
And
/* this is
a multi-line
comment */
However, the actual code looks ...
0
votes
0answers
15 views
Text processing - Regex search for string and output unique values [duplicate]
I have a use case where I would like to search the following text for any Windows AD objects in backslash format. Here is a use case to use as an example.
My goal is to output the unique results to a ...
-2
votes
3answers
41 views
sed/awk string replacement with newline and tabs
I have an XML file that contains the following:
<ipaddr>192.168.1.1</ipaddr>
<subnet>24</subnet>
The goal is to modify <ipaddr>192.168.1.1</ipaddr> to <...
0
votes
3answers
30 views
Get the last element of a row, separated with ;
I have a file (test.txt), where the separator is ;. How can I write out the last element from each row? (the length of each element is different)
0
votes
3answers
33 views
How can I parse the output of `ps` to different fields?
I would like to parse the output of ps, to get any field using awk. It seems that ps separate the fields by whitespaces, but the values in COMMAND field usually have whitespaces.
Is there a way to ...
0
votes
1answer
27 views
sed command not giving proper output for regular expression used
I am trying to fetch data between a range of time intervals from the log file. However, the complete data is not fetched for the end date.
Below is the command used:
sed -n '/26/Nov/2018/,/30/Nov/...
4
votes
3answers
156 views
Add lines to files to make them equal length
I have a bunch of .csv files with N columns and different number of rows (lines). I would like to add as many empty lines ;...; (N semicolons) to make them the same length. I can get the length of the ...
0
votes
1answer
19 views
How to replace an entire paragraph in a file?
I want to change this:
...
%%%23begin
aaaaaaa \\
bbbbbbb \\
ccccccc \\
%%%23end
...
to this
...
%%%23begin
xxxxxxxx \\
yyyyyyyy \\
zzzzzzzzz \\
tttttttt \\
%%%23end
...
the number ...
1
vote
3answers
65 views
How to switch first name with last name in this situation?
I'm given a file with many lines like so:
Vinh Tranh:438-910-7449:8235 Maple Street, Wilmington, VM 29085:9/23/63:68900
I need to change the order from Firstname Lastname to Lastname, Firstname. Ive ...
0
votes
2answers
12 views
Git: keep color of git stash list --pretty=<..> through a pipe
I have this command:
git stash list --pretty="%C(yellow)%h %>(14)%Cgreen%cr %C(blue)%gs"
gives output colored, but if I pipe it git stash list <..> | less -R, color disappear in output.
...
-1
votes
1answer
39 views
how to add words before line on all the scripts in current folder
under /home/scrript , we have 187 bash scripts
all the scripts have the following line:
kill -PIPE $PID
we want to add the following words before this line on all the scripts as the following
[[ $...
-1
votes
1answer
46 views
Remove lines in file
Trying to find lines that are greater than 59 but do not include any spaces.
E.g.,
a1e4fa338aae3e87250f4dbff518b8d72a004201395996a8474a252d25: found
a1e4fa f38aae3e8725 0f7dbff518b ...
0
votes
1answer
44 views
Compare 2 files and print matching values to a new file
I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";".
File1 Example:
AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH
Each line has a unique ID in $7.
I ...
0
votes
1answer
22 views
Slicing JSON with JQ to Create Arrays for New Relic Ingestion
sorry for the long title.
I need to take some JSON, slice it using JQ so New Relic can parse it properly as it only takes a specific format when pushing JSON to their dashboards.
I am using a bash ...
1
vote
2answers
50 views
How can I get less to display \n as a newline?
I have some files containing \n newlines that I am viewing with less. I would like to be able to have less display those newlines as actual newlines.
For example an array:
Array\n(\n [Color] =&...
-1
votes
2answers
40 views
jq: colorize selected field of json file [closed]
Extend of this question: jq: parse json file with constraint from other field
After select "name" field I'd like to colorize it:
{
"checksum": "9e44bb7b76d8c39c45420dd2158a4521",
"roots": {
...
0
votes
1answer
20 views
jq: parse json file with constraint from other field [duplicate]
Extend of this question: How to combine strings from JSON values, keeping only part of the string?
Above link's output also include the "name" of folder type, need to exclude these "type":
...
-1
votes
1answer
27 views
Truncate long line and append indicating symbol [closed]
I have sample:
"this is a long title must in a doule quotes" this_is_a_solid_long_line_without_space_or_tabs
"this is a long title must in a doule quotes" is a long line in double quotes that ...
2
votes
3answers
60 views
Splitting each row of a correlation matrix into individual files
I have a correlation matrix of 22000 genes and for some analysis, I need to split each row of the matrix into a new file. Which means I need to create 22000 individual files.
I don't want to use the ...
2
votes
2answers
62 views
Remove trailing commas from invalid json (to make it valid)
Let's say I have a file as below
{
"fruit": "Apple",
}
I want to remove the comma at the end of the line, if and only if the next line contains "}". So, the output will be :
{
"fruit": "...
1
vote
6answers
70 views
How to find word and change only next second line word
The file name staff.txt and sample contents are:
JHON
MANAGER
10000
I want to find JHON in a file and I want to change whatever's in the 2nd line after that one with another word/number.
How can I ...
2
votes
1answer
229 views
How to combine strings from JSON values, keeping only part of the string?
I have sample:
"name": "The title of website",
"sync_transaction_version": "1",
"type": "url",
"url": "https://url_of_website"
I want to get the following ...
2
votes
1answer
23 views
Layout tab/spaces [closed]
I've read this thread https://unix.stackexchange.com/a/7718/256195, that only if var doesn't contain any tab/spaces but in my case it does contain spaces, like below example:
"this is a test" ...
0
votes
3answers
76 views
How do i swap the first word of a line with the last word in a line
I was trying to replace the first word in a line with the last word in a line using a sed command i=but i can't seem to figure it out.
0
votes
1answer
24 views
Taking two integer arguments (the first smaller than the second) print out lines between the two integer arguments
I have a list of the most common passwords and their frequencies . It is a long list and it is formatted as such:
59462 password
49992 iloveyou
33919 princess
...
30 yahoo123
To be clear, the ...
0
votes
2answers
28 views
Replace @@@ and NotApplicable string and redirect output to other file? [duplicate]
I'm using IBM AIX which doesn't have much support like sed -i and sed with \t is not working in my case.
I would like to replace replace NotApplicable string with a single space ' ' then replace @@@ ...
0
votes
1answer
47 views
Insert new text in the middle of a file replacing old text without re-inserting the same things over and over?
I have a file that I want to update relatively frequently.
What needs to be done is that a specific part of the file would be changed to include some more lines. I am doing it with sed and \a option:
...
5
votes
2answers
151 views
cutting a field to length with awk/sed [duplicate]
I have a rather large bunch of files that contains several fields pipe-delimited.
5595340959340|1|MXPYAQWE|870569689456954654|0|0|20181018224126|1212121212121212121212121212|2|0|1000|70|33107||1|...
