Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

How to backup Thunderbird to mbox for use with the Pine or Alpine e-mail client?

How should the mbox backup from Thunderbird show? Perhaps there's a validation tool? First screen: ALPINE 2.25 FOLDER LIST ...
Nicholas Saunders's user avatar
0 votes
1 answer
47 views

Replacing a user entered string with another

So I am writing a program in bash involving a madlibs and I have to create a function that replaces the first occurence of a word for example. If the first word inputted was apple I can use this ...
Lemoniyy's user avatar
0 votes
3 answers
486 views

If line starts with a character remove text until another character in same line

Some lines start with a #, then any number (one or more digits), then =. Those lines starting with hash, remove text from beginning of line up to equal sign (including it). Example in file: some text #...
Smeterlink's user avatar
3 votes
4 answers
268 views

Isolate one specific instance of a word in a string

I have a grep command to process the string test-test test I want the command to only show me the "free-standing" word test highlighted, but not test-test. However, when I run echo "...
xUr's user avatar
  • 245
0 votes
5 answers
102 views

Delete lines below, excluding the line with match pattern

One can use sed '/pattern/Q' or sed '/pattern/,$d' to delete lines below a match pattern, but it also deletes the line containing the match pattern. How to exclude the line containing the match ...
Sadi's user avatar
  • 475
0 votes
6 answers
168 views

Delete all lines before n lines above the line containing the first match

If a file contains the following: Line 1 Line 2 Line 3 Line 4 Line 5 Line 6 Line 7 Line 8 Line 9 How to delete the lines above two lines before "Line 5", so that the file will now contain ...
Sadi's user avatar
  • 475
-3 votes
1 answer
33 views

Divide values in all columns of one file by a value in the corrosponding column if a second file

I have two files. The first one looks like this: The second file looks like this: I need to normalize the first table by the second, meaning for all values in column "sample" in file 1, I ...
Dvik's user avatar
  • 1
1 vote
4 answers
313 views

Retrieve the 1st and 5th column of a tab-separated file, convert the spaces in the 5th to tabs

I have a tsv file with tab-separated columns. I want to obtain the 5th column, which has space-separated values. Convert the space-separation to tab-separation and save as a new file. Attempt: cut -d&...
Anon's user avatar
  • 133
2 votes
4 answers
149 views

Parse text file add semicolons before and after numbers; join all other words that are before/ between numbers with hyphens

I have a text file with accounts like: Input Sample Paid 100 15/02/2022 3000 recd 50 15/02/2022 nelur trip 3050 PAID 80 25/03/2022 Adjusted towards trip 3130 14 PAID 50 26/03/2022 Given to Nate Cash (...
tgkprog's user avatar
  • 249
1 vote
4 answers
153 views

edit text/srt file

I have a .srt file like this Input File 1 00:00:17,920 --> 00:00:21,159 The essential is invisible to the eye. 3 2 00:00:21,160 --> 00:00:22,559 This phrase comes from 4 3 00:00:22,560 --> ...
dsoum's user avatar
  • 13
3 votes
1 answer
64 views

How can I pipe text that contains ANSI movement characters ESC[xC and ESC[xD and output the text formatted accodingly

I have text files that include the ANSI movement codes ESC[xC and ESC[xD. I want to filter these out but with each line output with those codes taken into account. Consider the line: this cat is ...
Scott's user avatar
  • 33
-4 votes
4 answers
147 views

Bash Script to covert rows in to columns between matching patterns

My Input file looks like this.. START line1 line2 line3 END START line11 line22 line33 END I want my script output to be like this: START line1 line2 line3 END START line11 line22 line33 END Please ...
Mouli's user avatar
  • 11
5 votes
6 answers
661 views

Keep unique values (comma separated) from each column

I have a .tsv (tab-separated columns) file on a Linux system with the following columns that contain different types of values (strings, numbers) separated by a comma: col1 col2 . NS,NS,...
df_v's user avatar
  • 51
3 votes
8 answers
680 views

How to enclose each line in single/double quote and end with comma except for the last line should not have a comma?

I want to enclose these in single/double quote and ending with a comma. But I also need it so that the last line does not include the comma as it is the last record. I've found several post of the ...
benbart's user avatar
  • 31
1 vote
4 answers
189 views

Using sed to replace the 1st ";" of a line with a string and then the 2nd ";" with another string

I have a semicolon separated playlist in the following format: Thåström ; Stjärna som är din ; Skebokvarnsv. 209 I want to replace the first ; with a dash (-) and the 2nd ; with the string "- ...
mariano-daniel's user avatar
1 vote
5 answers
127 views

Add two delimiters after every sixth comma

I'm trying to expand on this question but can't figure out this issue: Let's say I've got a file roll.txt: echo "'123456789','987651234','129873645','213456789','987612345','543216789','432156789'...
jophuh's user avatar
  • 13
0 votes
3 answers
177 views

How can I sort the lines in this text file?

I have a text file with book name and chapter in the bible. I would like to sort the lines so that lines are first sorted according to the order between their books, the lexical order between the ...
Tim's user avatar
  • 100k
1 vote
5 answers
556 views

Copy contents of one file to another using sed

I want to copy all text from one file into another file using sed on Alma Linux 8. For example, the first file, old.txt, is: 192.168.0.1 192.168.0.2 192.168.0.3 192.168.0.4 192.168.0.5 ... And the ...
Abdullah's user avatar
2 votes
4 answers
230 views

Replacing any <> closed string with a $ sign at the beginning of the same string in a text file

I have some files resides on a Linux system, containing some placeholders like this file one below: test.txt: This is a line with <VARIABLE1>@<VARIABLE2>. This is a line with <VARIABLE3&...
Rafiq's user avatar
  • 168
-4 votes
3 answers
326 views

Counting matches with grep, when multiple matches per line are possible

The standard usage of grep is to return lines that match a pattern. If a line can contain several matches of the pattern, how can I count each match individually, not the total number of matches?
Arunabh Bhattacharya's user avatar
3 votes
8 answers
361 views

Replace last word occurrence in file

The scenario is I want to comment out the last occurrence of a word in shell script using sed. Assume I have temp.sh with the following content: Abc 123 Abc Sdf 2 Abc Abc Utyr Qww I want to replace ...
Curiouscat's user avatar
0 votes
2 answers
79 views

Fitering rows of one CSV file based on two values from another file in Bash

I have two CSV files. One is long with columns: Chr_Name, h, j, start_pos, end_pos. Another is short file including columns Chr_Name and position. I need to filter the rows from the first file based ...
nishat's user avatar
  • 1
1 vote
5 answers
184 views

Duplicate part of a line to another part

I would like to copy the first part (IxoscaEVm****t1_, without the '.p[number]') of the line starting with ">" and paste a before the last ":" of the same line. Input: >...
alex kiarie's user avatar
0 votes
0 answers
50 views

How to convert accented characters in an email

I want to send an email with the body ($BODY) containing accented characters like é, à, ô... echo "$TO" | mail -s $SUBJECT $BODY But the result shows unreadable characters. I tried to ...
Romeo Dejent's user avatar
0 votes
2 answers
101 views

perl: replace a string with variable containing new lines (line breaks)

I want to make edits containing meta characters. Here https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed one of solutions is via perl with s\Q ...
Martian2020's user avatar
  • 1,137
0 votes
2 answers
125 views

How to check the headers between 2 CSV files in unix?

File1 - filename.csv File2 - filename.csv I wanted to compare the headers between these 2 files in linux using a ksh script. case 1:- Say, the header of file1 is INPUT --> NAME,UNIT CODE, VAR ...
Jay's user avatar
  • 1
0 votes
2 answers
57 views

How can I associate special logic with file extensions?

Is there an easy way to have arbitrary text editors call special logic based on a file extension? For example, let's say I had a zip file that had a collection of files. text.txt image.jpg othercrap....
DeepDeadpool's user avatar
  • 1,253
0 votes
2 answers
99 views

sed "invalid command code W"

I am trying to prepend text to the begging/header of a set of .vtt text files with sed in a loop. I'm not replacing any text, only adding text. I need some line breaks in the text. The text should ...
Bleakley's user avatar
  • 185
4 votes
4 answers
696 views

Can I extract complete dates from file with grep command?

I need help using grep to extract a zoned date time from a file on a Linux system. Source file is a XML with the data below: <item start="20231010073000 +0100" stop="20231010100000 +...
aris's user avatar
  • 43
1 vote
3 answers
219 views

Convert lower-case to uppercase with the output to a new column using miller

I want to copy column NAME to column NAME-LOWERCASE. NAME-LOWERCASE should only contain lowercase letters. The uppercase should be untouched in all column expect NAME-LOWERCASE input NAME,test PTC,N ...
pwrsheller's user avatar
2 votes
4 answers
115 views

Delete a whole word from a CSV file that is not part of another word using SED

I've searched for an answer to this and have come close, but not close enough. We receive a CSV file that contains the text "NULL" which needs to be replaced by nothing. Example: Input ...
aRBee's user avatar
  • 23
0 votes
1 answer
59 views

How to list process with their depth?

I have a slightly different process tree question. I need to list the process and their depth in scalar value. pstree gives the result in a "graphic" manner. root 100930 0.0 0.3 ...
piovisqui's user avatar
  • 875
0 votes
1 answer
103 views

Need Help with Date expression - AWK in a csv file

Situation: csv file has date column with different date formats, I want to convert them into a unambiguous date format (like +"%m-%d-%Y"). Example File data : File name = Date_Test_new.csv 3/...
ArielUnix2023's user avatar
0 votes
1 answer
52 views

What is the best practice to change variables in automations

I am trying to create auto install Ubuntu OS with packer on Hyper-V and I have project like this: packer-main/ | |--->http/ | |--->user-data | |--->templates/ | |--->build.pkr....
robotiaga's user avatar
  • 111
4 votes
5 answers
635 views

Replace a block of spaces with a comma

I have a file in txt format that I want to convert to csv. Between each field there are blocks of spaces. The amount of spaces is not the same between every field. Is there a command with sed or awk ...
pwrsheller's user avatar
1 vote
4 answers
86 views

Remove duplicates by adding numerical suffix

How do I append a numerical suffix to lines to remove duplicates? Pseudo code: if currLine.startsWith("tag:") x = numFutureLinesMatching(currLine) if (x > 0) currLine = currLine + ...
Steven's user avatar
  • 738
6 votes
6 answers
1k views

Making single digit numbers between 1 and 9 into double digits, inside CSV file

I have a CSV file with thousands of lines like these 1664;4;5;35;37;43;5;6 1663;21;23;32;40;49;8;11 1662;16;17;34;35;44;5;10 1661;2;9;23;32;40;6;7 1660;23;25;30;44;47;9;12 1659;3;5;9;32;43;6;10 1658;4;...
Duck's user avatar
  • 4,586
-1 votes
2 answers
204 views

Use find, cat, etc... to generate an M3U playlist from album playlists

I have a music folder with selected playlists in each folder titled "playlist.m3u" (should also match *.m3u8). What I would like to do is have a shell script generate a playlist concatenated ...
mrjpaxton's user avatar
5 votes
1 answer
311 views

Why does 'tr' with '-c' option and set 2 extension add an unwarranted character to the end?

I wanted to use tr to substitute "illegal" characters in a string with a replacement character, where "illegal" characters are all outside of a set of "allowed" ...
AdminBee's user avatar
  • 22.4k
0 votes
5 answers
247 views

Uniquify first column and add numbers in second column in a file in linux

I am collecting data from multiple files and dumping into a single summary file. cat *files* | sort -u > final.rpt a 1 a 5 a 6 b 2 b 3 b 0 c 1 c 7 How can I uniquify based on string in first ...
Nirav's user avatar
  • 17
4 votes
9 answers
1k views

Convert hundred of numbers in a column to row separated by a comma

How can I convert the following from a column: 1 2 3 4 5 6 . . . . 98 99 100 To a row separated by a comma: 1,2,3,4,5,6,....,98,99,100 I am using Linux.
golf4life's user avatar
0 votes
5 answers
122 views

How to create a list of words from custom elements?

I want to make a list of words in a text file from custom elements: element1: word1 word2 word3 element2: word4 word5 element3: word6 word7 I want each to be made of a combination of the words in ...
thebunnyrules's user avatar
0 votes
2 answers
97 views

AWK: If two key columns match between files, add the 16th column of one file to the matching row of the other while keeping non-matching rows

I have two tab separated files (FileA.tsv and FileB.tsv). FileA.tsv id graph circle several columns... length 196-0 196 0 ---- 12874 195-1 195 1 ---- 12874 56-0 56 0 ---- 3349 115-1 115 1 ---- ...
JAL's user avatar
  • 3
0 votes
1 answer
35 views

Parse results of a "dig" bulk query

We use a security rating service which among other things, searches the world for imposter domains that resemble our domains. So it does find some of those but also finds ones that don't exist. It ...
JetJaguar's user avatar
25 votes
3 answers
5k views

Is it now safe to parse the output of GNU ls?

The accepted wisdom for the past few decades has been that it is never a good idea to parse the output of ls ([1],[2]). For example, if I want to save a file's modification date along with its name ...
terdon's user avatar
  • 238k
0 votes
6 answers
171 views

Merge successive lines into one line

In a text file, if the first 25 characters in a line are a space, how can I append that line to the previous line until another line comes along that starts with an ASCII character in column one. ...
Walter Reis's user avatar
1 vote
2 answers
207 views

Need help with a sed script for specific word replacements in a text file

I'm trying to automate a specific text replacement task for Romanian text using sed or any other suitable Unix tool. Here are my requirements: Replace all instances of the letter â with î inside ...
Mafsi's user avatar
  • 141
4 votes
4 answers
296 views

Using the command line, multiply all entries within a column to a fixed number, but only if the line is not a comment

I am trying to: Always add 0 0 0 to the first row of the file. Multiply 2*pi or 6.2832 to the first column of a three-column file formatted similar below, but only if the line begins with a numeral. ...
Carla H.'s user avatar
  • 161
0 votes
3 answers
76 views

How can i show just "total36" in this case?

For my Homework I tried in linux to just get the output total 36 from the output of ls -s as produced here: vboxuser@Ubuntu:~$ ls -s insgesamt 36 4 Bilder 4 Dokumente 4 Downloads 4 Musik 4 ...
AweBrawl's user avatar
0 votes
1 answer
161 views

Process CSV file through command line: Remove only middle rows between consecutive row entries if consecutive entries have the same 2nd column value

I have a CSV file with only 2 columns (but lots of rows) and the occasional irregular row which always starts with an asterisk (*) character and may span more than two columns. Using just the Linux ...
Carla H.'s user avatar
  • 161

1
2 3 4 5
168