Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
7,693
questions
0
votes
1answer
40 views
When storing text on a USB drive, how do I make cat not hang?
Warning: I used these commands on a drive that had nothing on it (/dev/sdb). Do not attempt this on a drive with anything important on it.
I was experimenting some, and I discovered that the ...
0
votes
0answers
41 views
sed script that replaces a line containing @ [closed]
I want to change an autostart file via script, i tried to use sed but anything is okay what can be found on Raspberry Pi OS.
The autostart file:
@lxpanel --profile LXDE-pi
@pcmanfm --desktop --profile ...
-2
votes
1answer
53 views
Parse fixed width file and lookup those parsed values against Oracle Db
Files arrive within a specific time window (midnight-0600hrs EST) and get saved to the orders directory on the Linux host. these files are stored in a zip format. usually around 30 files are expected ...
3
votes
1answer
110 views
convert and replace only EPOCH value in a file
I have a input a like below which having epoch and a number in some 2nd, 3rd columns in random.I am trying to the convert the epoch dates by using awk built-in function strftime() and it replace the ...
0
votes
3answers
59 views
Selective extraction from patterned occurence (distant repeated lines) using awk or sed
A file has a line pattern where a line that does not contains numbers occurs twice in the file.
Energy initial, next-to-last, final =
-87057.8200168321 -87208.7223900015 -87208.7197287879
Energy ...
2
votes
3answers
101 views
Split file based on second column value
Can you help me to split a file that looks like the following into two files depending on the second column (groups starting with value 1 plus the following 2 lines should be in one file and groups ...
0
votes
6answers
75 views
Finding two patterns in a line and remove the patterns and sequence between them
I have a file with lines like this:
ABCDABCBCBBBCBCDDBBBBBBBBBBBBBBBBBBBBBBBBBXYZ
ABCDCCCBCCBBBBBBBBBBBBBBBBBBBBBBXYZ
ABCDACDCDCCCCBBBBBBBBBBBBBBBBBBBBBBBBXYZ
And I want
BBBBBBBBBBBBBBBBBBBBBXYZ
...
3
votes
5answers
327 views
How to fetch maximum value from logs
I have log file which shows ten values and last column value is memory in bytes. If I wanted to display highest memory used in last four days, what command or script should I use? With the help of ...
-1
votes
3answers
58 views
I have a fix length file how to extract the data to looks like csv file
So I have the following file:
HDR2013100120131001000000000000 ...
0
votes
1answer
25 views
Read first line split based on delimiter and store it in array (in ksh)
I am reading the first line of the file, header row, split by delimiter | and assigning it to array.
IFS='|' read -r -a header < "/file_uploads/file_headers.txt"
This is working fine in ...
0
votes
1answer
37 views
how to to use if-statement to do a command when a condition is met and another command when the the condition is not met
I have a list of samples with defined min and max rho values,
cat my.table
SAMPLE min_rho max_rho
RR26400a 0.1 0.3
RR26400c 0.2 0.4
RR26401a 0.1 0.7
RR26401c 0.5 0.8
RR26402a 0.4 0....
3
votes
2answers
102 views
How to search for a string in a very large file with very long lines?
It turns out I was accidentally using grep wrong yesterday. I just checked my bash history and saw what I was executing:
grep search-string-here -f large-file-with-long-lines.txt
And that was what ...
1
vote
2answers
42 views
Add number of repetitions in a separate column based on values in multiple columns
I am trying to add the number of occurance in the last column of a pipe delimited file based on column 1, 2 and 3.
I have tried awk '{ print $0 "|" ++count[$1,$2,$3] }'
But it is producing ...
2
votes
2answers
95 views
Find lines in one "key - value-list" file that contain data fields not mapped to that key in a second "key-value" file
I have 2 files. One file contains a minimum of two fields (a username, an age, and a variable number of "fruits" each in their own field) per line, the other file always contains two fields ...
0
votes
0answers
65 views
Compare the second column in file 1 and the second column in file 2 in linux
I have two files, file 1 and file 2. I want to suppose when the chromosome number are the same, if the second column in file 1 is greater than (the second column - 1,000) in file 2 and smaller than (...
1
vote
0answers
23 views
OSS Software for Quran Corpus(text annotations)
Is there any software that can produce annotations like this:
Link: https://corpus.quran.com/treebank.jsp?chapter=112&verse=1
I found a few options here: https://corpus-analysis.com/tag/...
1
vote
6answers
288 views
print lines between 1st occurence of 1st pattern to last occurence of 2nd pattern with AWK
I have a datafile, with dates in 2nd column
# cat datafile
-;20210106;-;-;-;
-;20210112;-;-;-;
-;20210112;-;-;-;
-;20210112;-;-;-;
...
-;20210217;-;-;-;
-;20210217;-;-;-;
-;20210217;-;-;-;
-;20210217;-...
0
votes
3answers
37 views
How can I remove characters before and after a text in a chain of characters with Grep, knowing that text is produced through a variable?
I'm using
grep 'name=>' log.txt | cut -d\ -f3
I get this kind of result:
name=>"Faya-Largeau",
name=>"Paris",
name=>"Madrid",
name is the output of a ...
-2
votes
2answers
47 views
extract everything after "[0-9]. " until line end in unix command
Suppose I have file which has entries like
5. variable1:
10. variable7:
230. variable21803:
I would like to get all the variable as an output without the "5. " etc. What I've tried is
grep -...
0
votes
1answer
46 views
Remove and add a character at the beginning of the specified line in a dynamic file
I need to change a specific line of a file; first I will add a char and after some time I will remove that char.
Adding a char to a specific line has been asked and answered in this question and works ...
2
votes
5answers
141 views
How can I extract some values that fall within a range
I have about 4500 lines of aminoacid variations that look like this:
S1437T
H1266Y
T2662A
E1397A
E626K
S1538T
E3021K
Briefly, the letters flanking the numbers are the amino acid residues and the ...
1
vote
2answers
64 views
regex lookbehind ignore quotes
I have the below string in my file and I want to extract the word after -s, i.e yesterday without quotes :
-t -w -g london -s 'yesterday' -e 'yesterday' -f 100000"
Required output
yesterday
...
0
votes
4answers
82 views
Is it possible to move a string which starts with # after another string to the preceding line?
I have a script file that looks like this:
auditctl -a always,exit -F arch=b64 -S openat -F auid=1000 <TAB> #description of command....
auditctl -a .... ...
-2
votes
3answers
65 views
I need to grep a code from file1 and file2 and write it to file3 [closed]
I have a file with my product names which contain special characters and spaces in between the name. At the end there's a code of that product.
In file2 I would have only names of the product without ...
4
votes
6answers
329 views
Count and return how many replacements were done with sed in a file
I have a large JSON file in which I want to replace one string with another. It should not, but it might happen that this string would be used in a context where I do not want to replace it.
I know ...
-4
votes
1answer
59 views
Want bash script to get txt file all possible combinations
I have following characters, numbers and special characters:
012345789
!@#$%
ABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrstuvwxyz
I want a bash script that create a txt file that have all the ...
0
votes
0answers
15 views
How to make LRF (Load Ready FIle) from this text file that contains XML Tags
everyone i was given TST.DATA_INPUT that was encoded in EBCDIC after transforming it in ASCII using:
dd if=~/KSH_Advanced/data/TST.DATA_INPUT of=~/KSH_Advanced/data/target.txt ibs=800 cbs=80 conv=...
-2
votes
2answers
85 views
fixed length file issue
few records in fixed length file are split into two records
The file looks some thing similar to below. The third record is split into two records. How can I remove '\r' from the end and move the ...
0
votes
3answers
66 views
find/detect file name goups
On fs i have files like:
PREFIX_GROUPNAME_OTHERNAMES[.txt|.*]
eg:
A_ABC_A.txt
A_ABC_B.txt
A_ABC_C.txt
A_XYZ_A.txt
A_XYZ_B.txt
A_XYZ_C.txt
For some futher tasks i want to get the group names.
$# ...
0
votes
7answers
223 views
Matching multiple ranges with sed range expressions [closed]
Update: As many commentators have pointed out, my question was ill-formed. Sorry and thank you! I am updating the my question but leaving the old question at the end for reference.
Updated question
I ...
2
votes
3answers
107 views
Replace a character only between the double-quotes with sed or awk [duplicate]
I have such a scenario inside a CSV file:
ID,PRICE,QUANTITY,ARRIVAL
01299,"41,5",1,0
26528,"412,03",0,0
38080,"2,35",0,0
38081,"2,35",0,0
..
..
The question I ...
4
votes
5answers
476 views
Replace a hexadecimal value by a (modified) decimal value in a text file
Inside a file test.txt I have a hexadecimal value
0x0000000000000000000000000000000000000000047546124890225541102135415377465907
Only one line. There are no other lines or characters. It is also ...
2
votes
2answers
278 views
Shift of dates not working as expected
I have a file with records in this format :
D20220327,S2927,977,1
D20220328,S2927,977,1
D20220329,S2927,977,1
D20220330,S2927,977,1
D20220331,S2927,977,1
D20220401,S2927,977,1
D20220402,S2927,...
3
votes
7answers
610 views
remove trailing spaces from text files ONLY when necessary
I'm removing the trailing spaces using
sed -i 's/[ \t]*$//' *.txt
However, this command will rewrite all the files.
Is there a convenient way to judge that if there are trailing spaces in a text file ...
3
votes
4answers
459 views
How can I duplicate a line and search and replace on the duplicate with sed?
I'm trying to duplicate lines in a text file that contain certain special characters, but in the duplicate, substitute the special character with "regular" ASCII characters. The concrete use-...
2
votes
3answers
52 views
How to read an input file of strings, match and change the matches in-place?
I have a text file with strings/filenames in separate lines, eg. filename.txt. There are hundreds of filenames
ABC123_S386_R1_001
JKL345_S441_R1_001
filename9000_S587_R1_001
and another text file ...
0
votes
2answers
55 views
How to remove a specific duplicate string from a field in a text file?
I have a file with three columns, and I need to remove lines that contain specific duplicated field.
1 V(Cl8) 2.121
2 V(C1,H3) 2.067
3 V(Cl7) ...
1
vote
1answer
30 views
How to print if match present and leave if not with awk
I have a result file A.tsv (separator = \t):
BC01 2551 99
BC03 2547 95
BC04 2578 99
BC05 2547 97
And a file with sample information B.tsv (separator = \t):
BC01
BC04
BC02
BC03
...
2
votes
3answers
122 views
How can I join two lines in a file if they both fit a pattern?
I have a file with multiple lines and I want to join lines if they both fit a specific pattern.
I know that I can find lines that fit the pattern and get the next line with:
grep -E -A1 'Pattern' ...
-1
votes
1answer
33 views
How to search and replace text in files but keep/reuse a certain part of text?
I want to refactor many JavaScript files in many directories which contain access to an object in this format: myObj.something.somethingElse and I want it to be myObj.getSomething(somethingElse) how ...
0
votes
1answer
65 views
How to parse a file and write to separate columns
I am trying to parse the output from antismash to count the number of BGC. I found the scripts that people have with python which I know nothing about, so I am trying to figure this out using bash ...
0
votes
2answers
49 views
count the number of rows with specific numbers of colums
I have several files with the following content:
GGHTERR_01218 GGHTERR_02418 GGHTERR_01991
GGHTERR_02211 GGHTERR_02297 GGHTERR_02379
GGHTERR_02294 GGHTERR_02455 GGHTERR_02374
GGHTERR_00532 ...
0
votes
2answers
41 views
Can I post-format column tab sizes so that spacing is at lowest common denominator in Linux?
I can use stat to create an ls output that shows both formats of permission information which can be handy:
stat --printf="%A\t%a\t%h\t%U\t%G\t%s\t%.19y\t%n\n" . .*
drwxr-xr-x 755 ...
0
votes
1answer
13 views
xinput keycodes do not align with xmodmap table
Is there a table that maps keycodes from the command xinput test-xi2 --root 3 to keysyms?
I've tried xmodmap -pke which I found in the ArchWiki but they don't correspond. How does X even interact with ...
0
votes
1answer
72 views
Merging two text files using AWK
I don't have much experience in awk, so I struggle to overcome one problem with it.
I have two files named file1.txt and file2.txt.
File1.txt:
20 101 1 2 3 4
20 102 5 6 7 8
20 108 3 3 3 3
File2.txt:
...
-1
votes
1answer
36 views
Recode column values based on previous values
1859115 2258379 24636 Yes 06S14028968 13 1 1 2
1859115 2258379 24636 Yes 06S14028968 13 1 1 2
1859116 2255037 21608 Yes 06S14028969 11 0 2 3
1859117 2268746 34027 Yes 06S14028970 10 0 2 1
Above is ...
3
votes
10answers
790 views
How to take only uniq rows based on a column using linux cmd?
Here is my dataset:
col1,col2,col3
a,b,c
a,d,f
d,u,v
f,g,h
d,u,g
x,t,k
Expected output:
f,g,h
x,t,k
Selection criteria:
If anything occurs in col1 multiple times, then all of the associated rows ...
-2
votes
2answers
50 views
How to convert Fixed width Unix file to pipe delimited using column Ordinal Position and field length
I want to convert a fixed width file to pipe delimited file in UNIX by using Column Ordinal Position and field length.
The input looks like this,
00100000003779U20210203
00200000004238U20210203
...
0
votes
3answers
93 views
Search for pattern and create file with the same name
Here's a simple line of my usage:
grep -i '"location_country":"country name"' file.txt >> sample.txt
I'm searching a large file with multiple countries and what I would like ...
0
votes
3answers
48 views
Find all lines containing string - summarize and count them
I have a lot of files in several subfolders that include a string type="abc_ - I'm not too sure how many variants of abc_ there is - so I want to know how many occurrences of the unknown type=&...
