Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
0
votes
1answer
40 views

When storing text on a USB drive, how do I make cat not hang?

Warning: I used these commands on a drive that had nothing on it (/dev/sdb). Do not attempt this on a drive with anything important on it. I was experimenting some, and I discovered that the ...
0
votes
0answers
41 views

sed script that replaces a line containing @ [closed]

I want to change an autostart file via script, i tried to use sed but anything is okay what can be found on Raspberry Pi OS. The autostart file: @lxpanel --profile LXDE-pi @pcmanfm --desktop --profile ...
-2
votes
1answer
53 views

Parse fixed width file and lookup those parsed values against Oracle Db

Files arrive within a specific time window (midnight-0600hrs EST) and get saved to the orders directory on the Linux host. these files are stored in a zip format. usually around 30 files are expected ...
3
votes
1answer
110 views

convert and replace only EPOCH value in a file

I have a input a like below which having epoch and a number in some 2nd, 3rd columns in random.I am trying to the convert the epoch dates by using awk built-in function strftime() and it replace the ...
0
votes
3answers
59 views

Selective extraction from patterned occurence (distant repeated lines) using awk or sed

A file has a line pattern where a line that does not contains numbers occurs twice in the file. Energy initial, next-to-last, final = -87057.8200168321 -87208.7223900015 -87208.7197287879 Energy ...
2
votes
3answers
101 views

Split file based on second column value

Can you help me to split a file that looks like the following into two files depending on the second column (groups starting with value 1 plus the following 2 lines should be in one file and groups ...
0
votes
6answers
75 views

Finding two patterns in a line and remove the patterns and sequence between them

I have a file with lines like this: ABCDABCBCBBBCBCDDBBBBBBBBBBBBBBBBBBBBBBBBBXYZ ABCDCCCBCCBBBBBBBBBBBBBBBBBBBBBBXYZ ABCDACDCDCCCCBBBBBBBBBBBBBBBBBBBBBBBBXYZ And I want BBBBBBBBBBBBBBBBBBBBBXYZ ...
3
votes
5answers
327 views

How to fetch maximum value from logs

I have log file which shows ten values and last column value is memory in bytes. If I wanted to display highest memory used in last four days, what command or script should I use? With the help of ...
-1
votes
3answers
58 views

I have a fix length file how to extract the data to looks like csv file

So I have the following file: HDR2013100120131001000000000000 ...
0
votes
1answer
25 views

Read first line split based on delimiter and store it in array (in ksh)

I am reading the first line of the file, header row, split by delimiter | and assigning it to array. IFS='|' read -r -a header < "/file_uploads/file_headers.txt" This is working fine in ...
0
votes
1answer
37 views

how to to use if-statement to do a command when a condition is met and another command when the the condition is not met

I have a list of samples with defined min and max rho values, cat my.table SAMPLE min_rho max_rho RR26400a 0.1 0.3 RR26400c 0.2 0.4 RR26401a 0.1 0.7 RR26401c 0.5 0.8 RR26402a 0.4 0....
3
votes
2answers
102 views

How to search for a string in a very large file with very long lines?

It turns out I was accidentally using grep wrong yesterday. I just checked my bash history and saw what I was executing: grep search-string-here -f large-file-with-long-lines.txt And that was what ...
1
vote
2answers
42 views

Add number of repetitions in a separate column based on values in multiple columns

I am trying to add the number of occurance in the last column of a pipe delimited file based on column 1, 2 and 3. I have tried awk '{ print $0 "|" ++count[$1,$2,$3] }' But it is producing ...
2
votes
2answers
95 views

Find lines in one "key - value-list" file that contain data fields not mapped to that key in a second "key-value" file

I have 2 files. One file contains a minimum of two fields (a username, an age, and a variable number of "fruits" each in their own field) per line, the other file always contains two fields ...
0
votes
0answers
65 views

Compare the second column in file 1 and the second column in file 2 in linux

I have two files, file 1 and file 2. I want to suppose when the chromosome number are the same, if the second column in file 1 is greater than (the second column - 1,000) in file 2 and smaller than (...
1
vote
0answers
23 views

OSS Software for Quran Corpus(text annotations)

Is there any software that can produce annotations like this: Link: https://corpus.quran.com/treebank.jsp?chapter=112&verse=1 I found a few options here: https://corpus-analysis.com/tag/...
1
vote
6answers
288 views

print lines between 1st occurence of 1st pattern to last occurence of 2nd pattern with AWK

I have a datafile, with dates in 2nd column # cat datafile -;20210106;-;-;-; -;20210112;-;-;-; -;20210112;-;-;-; -;20210112;-;-;-; ... -;20210217;-;-;-; -;20210217;-;-;-; -;20210217;-;-;-; -;20210217;-...
0
votes
3answers
37 views

How can I remove characters before and after a text in a chain of characters with Grep, knowing that text is produced through a variable?

I'm using grep 'name=>' log.txt | cut -d\ -f3 I get this kind of result: name=>"Faya-Largeau", name=>"Paris", name=>"Madrid", name is the output of a ...
-2
votes
2answers
47 views

extract everything after "[0-9]. " until line end in unix command

Suppose I have file which has entries like 5. variable1: 10. variable7: 230. variable21803: I would like to get all the variable as an output without the "5. " etc. What I've tried is grep -...
0
votes
1answer
46 views

Remove and add a character at the beginning of the specified line in a dynamic file

I need to change a specific line of a file; first I will add a char and after some time I will remove that char. Adding a char to a specific line has been asked and answered in this question and works ...
2
votes
5answers
141 views

How can I extract some values that fall within a range

I have about 4500 lines of aminoacid variations that look like this: S1437T H1266Y T2662A E1397A E626K S1538T E3021K Briefly, the letters flanking the numbers are the amino acid residues and the ...
1
vote
2answers
64 views

regex lookbehind ignore quotes

I have the below string in my file and I want to extract the word after -s, i.e yesterday without quotes : -t -w -g london -s 'yesterday' -e 'yesterday' -f 100000" Required output yesterday ...
0
votes
4answers
82 views

Is it possible to move a string which starts with # after another string to the preceding line?

I have a script file that looks like this: auditctl -a always,exit -F arch=b64 -S openat -F auid=1000 <TAB> #description of command.... auditctl -a .... ...
-2
votes
3answers
65 views

I need to grep a code from file1 and file2 and write it to file3 [closed]

I have a file with my product names which contain special characters and spaces in between the name. At the end there's a code of that product. In file2 I would have only names of the product without ...
4
votes
6answers
329 views

Count and return how many replacements were done with sed in a file

I have a large JSON file in which I want to replace one string with another. It should not, but it might happen that this string would be used in a context where I do not want to replace it. I know ...
-4
votes
1answer
59 views

Want bash script to get txt file all possible combinations

I have following characters, numbers and special characters: 012345789 !@#$% ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz I want a bash script that create a txt file that have all the ...
0
votes
0answers
15 views

How to make LRF (Load Ready FIle) from this text file that contains XML Tags

everyone i was given TST.DATA_INPUT that was encoded in EBCDIC after transforming it in ASCII using: dd if=~/KSH_Advanced/data/TST.DATA_INPUT of=~/KSH_Advanced/data/target.txt ibs=800 cbs=80 conv=...
-2
votes
2answers
85 views

fixed length file issue

few records in fixed length file are split into two records The file looks some thing similar to below. The third record is split into two records. How can I remove '\r' from the end and move the ...
0
votes
3answers
66 views

find/detect file name goups

On fs i have files like: PREFIX_GROUPNAME_OTHERNAMES[.txt|.*] eg: A_ABC_A.txt A_ABC_B.txt A_ABC_C.txt A_XYZ_A.txt A_XYZ_B.txt A_XYZ_C.txt For some futher tasks i want to get the group names. $# ...
0
votes
7answers
223 views

Matching multiple ranges with sed range expressions [closed]

Update: As many commentators have pointed out, my question was ill-formed. Sorry and thank you! I am updating the my question but leaving the old question at the end for reference. Updated question I ...
2
votes
3answers
107 views

Replace a character only between the double-quotes with sed or awk [duplicate]

I have such a scenario inside a CSV file: ID,PRICE,QUANTITY,ARRIVAL 01299,"41,5",1,0 26528,"412,03",0,0 38080,"2,35",0,0 38081,"2,35",0,0 .. .. The question I ...
4
votes
5answers
476 views

Replace a hexadecimal value by a (modified) decimal value in a text file

Inside a file test.txt I have a hexadecimal value 0x0000000000000000000000000000000000000000047546124890225541102135415377465907 Only one line. There are no other lines or characters. It is also ...
2
votes
2answers
278 views

Shift of dates not working as expected

I have a file with records in this format : D20220327,S2927,977,1 D20220328,S2927,977,1 D20220329,S2927,977,1 D20220330,S2927,977,1 D20220331,S2927,977,1 D20220401,S2927,977,1 D20220402,S2927,...
3
votes
7answers
610 views

remove trailing spaces from text files ONLY when necessary

I'm removing the trailing spaces using sed -i 's/[ \t]*$//' *.txt However, this command will rewrite all the files. Is there a convenient way to judge that if there are trailing spaces in a text file ...
3
votes
4answers
459 views

How can I duplicate a line and search and replace on the duplicate with sed?

I'm trying to duplicate lines in a text file that contain certain special characters, but in the duplicate, substitute the special character with "regular" ASCII characters. The concrete use-...
2
votes
3answers
52 views

How to read an input file of strings, match and change the matches in-place?

I have a text file with strings/filenames in separate lines, eg. filename.txt. There are hundreds of filenames ABC123_S386_R1_001 JKL345_S441_R1_001 filename9000_S587_R1_001 and another text file ...
0
votes
2answers
55 views

How to remove a specific duplicate string from a field in a text file?

I have a file with three columns, and I need to remove lines that contain specific duplicated field. 1 V(Cl8) 2.121 2 V(C1,H3) 2.067 3 V(Cl7) ...
1
vote
1answer
30 views

How to print if match present and leave if not with awk

I have a result file A.tsv (separator = \t): BC01 2551 99 BC03 2547 95 BC04 2578 99 BC05 2547 97 And a file with sample information B.tsv (separator = \t): BC01 BC04 BC02 BC03 ...
2
votes
3answers
122 views

How can I join two lines in a file if they both fit a pattern?

I have a file with multiple lines and I want to join lines if they both fit a specific pattern. I know that I can find lines that fit the pattern and get the next line with: grep -E -A1 'Pattern' ...
-1
votes
1answer
33 views

How to search and replace text in files but keep/reuse a certain part of text?

I want to refactor many JavaScript files in many directories which contain access to an object in this format: myObj.something.somethingElse and I want it to be myObj.getSomething(somethingElse) how ...
0
votes
1answer
65 views

How to parse a file and write to separate columns

I am trying to parse the output from antismash to count the number of BGC. I found the scripts that people have with python which I know nothing about, so I am trying to figure this out using bash ...
0
votes
2answers
49 views

count the number of rows with specific numbers of colums

I have several files with the following content: GGHTERR_01218 GGHTERR_02418 GGHTERR_01991 GGHTERR_02211 GGHTERR_02297 GGHTERR_02379 GGHTERR_02294 GGHTERR_02455 GGHTERR_02374 GGHTERR_00532 ...
0
votes
2answers
41 views

Can I post-format column tab sizes so that spacing is at lowest common denominator in Linux?

I can use stat to create an ls output that shows both formats of permission information which can be handy: stat --printf="%A\t%a\t%h\t%U\t%G\t%s\t%.19y\t%n\n" . .* drwxr-xr-x 755 ...
0
votes
1answer
13 views

xinput keycodes do not align with xmodmap table

Is there a table that maps keycodes from the command xinput test-xi2 --root 3 to keysyms? I've tried xmodmap -pke which I found in the ArchWiki but they don't correspond. How does X even interact with ...
0
votes
1answer
72 views

Merging two text files using AWK

I don't have much experience in awk, so I struggle to overcome one problem with it. I have two files named file1.txt and file2.txt. File1.txt: 20 101 1 2 3 4 20 102 5 6 7 8 20 108 3 3 3 3 File2.txt: ...
-1
votes
1answer
36 views

Recode column values based on previous values

1859115 2258379 24636 Yes 06S14028968 13 1 1 2 1859115 2258379 24636 Yes 06S14028968 13 1 1 2 1859116 2255037 21608 Yes 06S14028969 11 0 2 3 1859117 2268746 34027 Yes 06S14028970 10 0 2 1 Above is ...
3
votes
10answers
790 views

How to take only uniq rows based on a column using linux cmd?

Here is my dataset: col1,col2,col3 a,b,c a,d,f d,u,v f,g,h d,u,g x,t,k Expected output: f,g,h x,t,k Selection criteria: If anything occurs in col1 multiple times, then all of the associated rows ...
-2
votes
2answers
50 views

How to convert Fixed width Unix file to pipe delimited using column Ordinal Position and field length

I want to convert a fixed width file to pipe delimited file in UNIX by using Column Ordinal Position and field length. The input looks like this, 00100000003779U20210203 00200000004238U20210203 ...
0
votes
3answers
93 views

Search for pattern and create file with the same name

Here's a simple line of my usage: grep -i '"location_country":"country name"' file.txt >> sample.txt I'm searching a large file with multiple countries and what I would like ...
0
votes
3answers
48 views

Find all lines containing string - summarize and count them

I have a lot of files in several subfolders that include a string type="abc_ - I'm not too sure how many variants of abc_ there is - so I want to know how many occurrences of the unknown type=&...

1
2 3 4 5
154