Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
8,359
questions
0
votes
0
answers
12
views
How to backup Thunderbird to mbox for use with the Pine or Alpine e-mail client?
How should the mbox backup from Thunderbird show? Perhaps there's a validation tool?
First screen:
ALPINE 2.25 FOLDER LIST ...
0
votes
1
answer
47
views
Replacing a user entered string with another
So I am writing a program in bash involving a madlibs and I have to create a function that replaces the first occurence of a word for example. If the first word inputted was apple I can use this ...
- 1
0
votes
3
answers
486
views
If line starts with a character remove text until another character in same line
Some lines start with a #, then any number (one or more digits), then =. Those lines starting with hash, remove text from beginning of line up to equal sign (including it).
Example in file:
some text
#...
- 209
3
votes
4
answers
268
views
Isolate one specific instance of a word in a string
I have a grep command to process the string
test-test test
I want the command to only show me the "free-standing" word test highlighted, but not test-test.
However, when I run
echo "...
- 245
0
votes
5
answers
102
views
Delete lines below, excluding the line with match pattern
One can use sed '/pattern/Q' or sed '/pattern/,$d' to delete lines below a match pattern, but it also deletes the line containing the match pattern.
How to exclude the line containing the match ...
- 475
0
votes
6
answers
168
views
Delete all lines before n lines above the line containing the first match
If a file contains the following:
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
How to delete the lines above two lines before "Line 5", so that the file will now contain ...
- 475
-3
votes
1
answer
33
views
Divide values in all columns of one file by a value in the corrosponding column if a second file
I have two files. The first one looks like this:
The second file looks like this:
I need to normalize the first table by the second, meaning for all values in column "sample" in file 1, I ...
- 1
1
vote
4
answers
313
views
Retrieve the 1st and 5th column of a tab-separated file, convert the spaces in the 5th to tabs
I have a tsv file with tab-separated columns. I want to obtain the 5th column, which has space-separated values. Convert the space-separation to tab-separation and save as a new file.
Attempt:
cut -d&...
- 133
2
votes
4
answers
149
views
Parse text file add semicolons before and after numbers; join all other words that are before/ between numbers with hyphens
I have a text file with accounts like:
Input Sample
Paid 100 15/02/2022 3000
recd 50 15/02/2022 nelur trip 3050
PAID 80 25/03/2022 Adjusted towards trip 3130
14 PAID 50 26/03/2022 Given to Nate Cash (...
- 249
1
vote
4
answers
153
views
edit text/srt file
I have a .srt file like this
Input File
1
00:00:17,920 --> 00:00:21,159
The essential is invisible to the eye. 3
2
00:00:21,160 --> 00:00:22,559
This phrase comes from 4
3
00:00:22,560 --> ...
- 13
3
votes
1
answer
64
views
How can I pipe text that contains ANSI movement characters ESC[xC and ESC[xD and output the text formatted accodingly
I have text files that include the ANSI movement codes ESC[xC and ESC[xD. I want to filter these out but with each line output with those codes taken into account.
Consider the line:
this cat is ...
- 33
-4
votes
4
answers
147
views
Bash Script to covert rows in to columns between matching patterns
My Input file looks like this..
START
line1
line2
line3
END
START
line11
line22
line33
END
I want my script output to be like this:
START line1 line2 line3 END
START line11 line22 line33 END
Please ...
- 11
5
votes
6
answers
661
views
Keep unique values (comma separated) from each column
I have a .tsv (tab-separated columns) file on a Linux system with the following columns that contain different types of values (strings, numbers) separated by a comma:
col1 col2
. NS,NS,...
- 51
3
votes
8
answers
680
views
How to enclose each line in single/double quote and end with comma except for the last line should not have a comma?
I want to enclose these in single/double quote and ending with a comma. But I also need it so that the last line does not include the comma as it is the last record.
I've found several post of the ...
- 31
1
vote
4
answers
189
views
Using sed to replace the 1st ";" of a line with a string and then the 2nd ";" with another string
I have a semicolon separated playlist in the following format:
Thåström ; Stjärna som är din ; Skebokvarnsv. 209
I want to replace the first ; with a dash (-) and the 2nd ; with the string "- ...
- 177
1
vote
5
answers
127
views
Add two delimiters after every sixth comma
I'm trying to expand on this question but can't figure out this issue:
Let's say I've got a file roll.txt:
echo "'123456789','987651234','129873645','213456789','987612345','543216789','432156789'...
- 13
0
votes
3
answers
177
views
How can I sort the lines in this text file?
I have a text file with book name and chapter in the bible. I would like to sort the lines so that
lines are first sorted according to the order between their books, the lexical order between the ...
- 100k
1
vote
5
answers
556
views
Copy contents of one file to another using sed
I want to copy all text from one file into another file using sed on Alma Linux 8. For example, the first file, old.txt, is:
192.168.0.1
192.168.0.2
192.168.0.3
192.168.0.4
192.168.0.5
...
And the ...
- 19
2
votes
4
answers
230
views
Replacing any <> closed string with a $ sign at the beginning of the same string in a text file
I have some files resides on a Linux system, containing some placeholders like this file one below:
test.txt:
This is a line with <VARIABLE1>@<VARIABLE2>.
This is a line with <VARIABLE3&...
- 168
-4
votes
3
answers
326
views
Counting matches with grep, when multiple matches per line are possible
The standard usage of grep is to return lines that match a pattern.
If a line can contain several matches of the pattern, how can I count each match individually, not the total number of matches?
3
votes
8
answers
361
views
Replace last word occurrence in file
The scenario is I want to comment out the last occurrence of a word in shell script using sed.
Assume I have temp.sh with the following content:
Abc 123 Abc
Sdf 2
Abc
Abc
Utyr
Qww
I want to replace ...
- 33
0
votes
2
answers
79
views
Fitering rows of one CSV file based on two values from another file in Bash
I have two CSV files.
One is long with columns: Chr_Name, h, j, start_pos, end_pos.
Another is short file including columns Chr_Name and position.
I need to filter the rows from the first file based ...
- 1
1
vote
5
answers
184
views
Duplicate part of a line to another part
I would like to copy the first part (IxoscaEVm****t1_, without the '.p[number]') of the line starting with ">" and paste a before the last ":" of the same line.
Input:
>...
- 139
0
votes
0
answers
50
views
How to convert accented characters in an email
I want to send an email with the body ($BODY) containing accented characters like é, à, ô...
echo "$TO" | mail -s $SUBJECT $BODY
But the result shows unreadable characters.
I tried to ...
0
votes
2
answers
101
views
perl: replace a string with variable containing new lines (line breaks)
I want to make edits containing meta characters. Here https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed one of solutions is via perl with s\Q ...
- 1,137
0
votes
2
answers
125
views
How to check the headers between 2 CSV files in unix?
File1 - filename.csv
File2 - filename.csv
I wanted to compare the headers between these 2 files in linux using a ksh script.
case 1:-
Say,
the header of file1 is INPUT -->
NAME,UNIT CODE, VAR
...
- 1
0
votes
2
answers
57
views
How can I associate special logic with file extensions?
Is there an easy way to have arbitrary text editors call special logic based on a file extension?
For example, let's say I had a zip file that had a collection of files.
text.txt
image.jpg
othercrap....
- 1,253
0
votes
2
answers
99
views
sed "invalid command code W"
I am trying to prepend text to the begging/header of a set of .vtt text files with sed in a loop. I'm not replacing any text, only adding text.
I need some line breaks in the text.
The text should ...
- 185
4
votes
4
answers
696
views
Can I extract complete dates from file with grep command?
I need help using grep to extract a zoned date time from a file on a Linux system.
Source file is a XML with the data below:
<item start="20231010073000 +0100" stop="20231010100000 +...
- 43
1
vote
3
answers
219
views
Convert lower-case to uppercase with the output to a new column using miller
I want to copy column NAME to column NAME-LOWERCASE. NAME-LOWERCASE should only contain lowercase letters. The uppercase should be untouched in all column expect NAME-LOWERCASE
input
NAME,test
PTC,N
...
- 185
2
votes
4
answers
115
views
Delete a whole word from a CSV file that is not part of another word using SED
I've searched for an answer to this and have come close, but not close enough.
We receive a CSV file that contains the text "NULL" which needs to be replaced by nothing. Example:
Input
...
- 23
0
votes
1
answer
59
views
How to list process with their depth?
I have a slightly different process tree question. I need to list the process and their depth in scalar value. pstree gives the result in a "graphic" manner.
root 100930 0.0 0.3 ...
- 875
0
votes
1
answer
103
views
Need Help with Date expression - AWK in a csv file
Situation: csv file has date column with different date formats, I want to convert them into a unambiguous date format (like +"%m-%d-%Y").
Example File data : File name = Date_Test_new.csv
3/...
0
votes
1
answer
52
views
What is the best practice to change variables in automations
I am trying to create auto install Ubuntu OS with packer on Hyper-V and I have project like this:
packer-main/
|
|--->http/
| |--->user-data
|
|--->templates/
| |--->build.pkr....
- 111
4
votes
5
answers
635
views
Replace a block of spaces with a comma
I have a file in txt format that I want to convert to csv. Between each field there are blocks of spaces. The amount of spaces is not the same between every field.
Is there a command with sed or awk ...
- 185
1
vote
4
answers
86
views
Remove duplicates by adding numerical suffix
How do I append a numerical suffix to lines to remove duplicates?
Pseudo code:
if currLine.startsWith("tag:")
x = numFutureLinesMatching(currLine)
if (x > 0)
currLine = currLine + ...
- 738
6
votes
6
answers
1k
views
Making single digit numbers between 1 and 9 into double digits, inside CSV file
I have a CSV file with thousands of lines like these
1664;4;5;35;37;43;5;6
1663;21;23;32;40;49;8;11
1662;16;17;34;35;44;5;10
1661;2;9;23;32;40;6;7
1660;23;25;30;44;47;9;12
1659;3;5;9;32;43;6;10
1658;4;...
- 4,586
-1
votes
2
answers
204
views
Use find, cat, etc... to generate an M3U playlist from album playlists
I have a music folder with selected playlists in each folder titled "playlist.m3u" (should also match *.m3u8).
What I would like to do is have a shell script generate a playlist concatenated ...
- 25
5
votes
1
answer
311
views
Why does 'tr' with '-c' option and set 2 extension add an unwarranted character to the end?
I wanted to use tr to substitute "illegal" characters in a string with a replacement character, where "illegal" characters are all outside of a set of "allowed" ...
- 22.4k
0
votes
5
answers
247
views
Uniquify first column and add numbers in second column in a file in linux
I am collecting data from multiple files and dumping into a single summary file.
cat *files* | sort -u > final.rpt
a 1
a 5
a 6
b 2
b 3
b 0
c 1
c 7
How can I uniquify based on string in first ...
- 17
4
votes
9
answers
1k
views
Convert hundred of numbers in a column to row separated by a comma
How can I convert the following from a column:
1
2
3
4
5
6
.
.
.
.
98
99
100
To a row separated by a comma:
1,2,3,4,5,6,....,98,99,100
I am using Linux.
- 57
0
votes
5
answers
122
views
How to create a list of words from custom elements?
I want to make a list of words in a text file from custom elements:
element1:
word1
word2
word3
element2:
word4
word5
element3:
word6
word7
I want each to be made of a combination of the words in ...
- 1,077
0
votes
2
answers
97
views
AWK: If two key columns match between files, add the 16th column of one file to the matching row of the other while keeping non-matching rows
I have two tab separated files (FileA.tsv and FileB.tsv).
FileA.tsv
id
graph
circle
several columns...
length
196-0
196
0
----
12874
195-1
195
1
----
12874
56-0
56
0
----
3349
115-1
115
1
----
...
- 3
0
votes
1
answer
35
views
Parse results of a "dig" bulk query
We use a security rating service which among other things, searches the world for imposter domains that resemble our domains. So it does find some of those but also finds ones that don't exist. It ...
- 13
25
votes
3
answers
5k
views
Is it now safe to parse the output of GNU ls?
The accepted wisdom for the past few decades has been that it is never a good idea to parse the output of ls ([1],[2]). For example, if I want to save a file's modification date along with its name ...
- 238k
0
votes
6
answers
171
views
Merge successive lines into one line
In a text file, if the first 25 characters in a line are a space, how can I append that line to the previous line until another line comes along that starts with an ASCII character in column one. ...
- 13
1
vote
2
answers
207
views
Need help with a sed script for specific word replacements in a text file
I'm trying to automate a specific text replacement task for Romanian text using sed or any other suitable Unix tool. Here are my requirements:
Replace all instances of the letter â with î inside ...
- 141
4
votes
4
answers
296
views
Using the command line, multiply all entries within a column to a fixed number, but only if the line is not a comment
I am trying to:
Always add 0 0 0 to the first row of the file.
Multiply 2*pi or 6.2832 to the first column of a three-column file formatted similar below, but only if the line begins with a numeral. ...
- 161
0
votes
3
answers
76
views
How can i show just "total36" in this case?
For my Homework I tried in linux to just get the output total 36 from the output of ls -s as produced here:
vboxuser@Ubuntu:~$ ls -s
insgesamt 36
4 Bilder 4 Dokumente 4 Downloads 4 Musik 4 ...
- 9
0
votes
1
answer
161
views
Process CSV file through command line: Remove only middle rows between consecutive row entries if consecutive entries have the same 2nd column value
I have a CSV file with only 2 columns (but lots of rows) and the occasional irregular row which always starts with an asterisk (*) character and may span more than two columns. Using just the Linux ...
- 161