Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
0
votes
3answers
24 views

Extract one string between double quotations

I'm trying to make a custom polybar module that shows the current active window class, since the default module outputs the window name. I'm using this command to get the window class: xprop -id $(...
1
vote
3answers
71 views

Simple shell script unable to go through thousands of files; starts fine, but throws “unexpected EOF while looking for matching `”`" after some time

Shell Script in Question Let me explain what I am trying to do by e.g. so you can understand better. Let's say I have 100 .torrent files in a directory. 2 of them will download xxx.epub and yyy.epub ...
0
votes
1answer
50 views

Bash script to comment and uncomment line

hope you are doing well during this time. The thing is I d like to comment some lines in my txt file. I have a code to comment and uncomment multiple lines, however I dont know how to get company name....
1
vote
2answers
49 views

Remove duplicates csv based on first value keeping the longest line between duplicates

I have many different csv files in a folder (megadrive.txt, snes.txt) like this: Aerial Assault (USA);Aerial Assault (USA);Sega Master System;;;;;;;;;0;;;;; Aerial Assault (USA);Aerial Assault (USA);...
0
votes
1answer
28 views

How to replace a matching string in one file with matching string in another file?

I have a file like this: head cleandata.map 1 1:775852:T:C 0 775852 1 1:1120590:A:C 0 1120590 1 1:1145994:T:C 0 1145994 1 1:1148494:A:G 0 1148494 ... And another file: head ...
2
votes
1answer
33 views

Replace specific character matches in all rows, using the first line as a reference

H, I want to replace specific character matches (.) in all rows, using the first line as a reference My attempt at re-hashing an answer I got to a different question: awk -F'|' 'BEGIN{OFS=FS} NR==1 ...
0
votes
0answers
48 views

How to output only certain lines from a file having the line numbers in another file? [duplicate]

I have a file with desired line numbers: 4988 5008 5028 5050 5072 5094 5116 5138 5160 5182 5204 5224 . etc . These are line numbers indexes into a log file that I have: URL : https://dc4-...
6
votes
7answers
1k views

How do I replace ALL text in a file after a certain line with the content of another text file, using sed/awk?

I am currently trying to automate updating the text of a file, titled original_file.txt. Imagine the file looks like the following: common_text ### REPLACE EVERYTHING AFTER THIS LINE ### ...
1
vote
1answer
26 views

egrep, sed or awk to get the number of active dhcp leases?

Let's say I have the following dhcpd.leases file: # The format of this file is documented in the dhcpd.leases(5) manual page. # This lease file was written by isc-dhcp-4.3.6-P1 # authoring-byte-...
0
votes
0answers
13 views

Is there a way to pipe to a visual editor and then continue the pipeline? [duplicate]

I know that some command-line programs can open visual editors like Sublime in a "wait mode" which opens the editor, lets you edit, and on-close it resumes the program with the file contents (like git ...
-2
votes
4answers
62 views

Counting commutative columns in a file

I have pairwise strings like this in my file A B A C B A B C C A I'm looking for a way to count how many commutative pairs I got. I.e A B and B A makes one such pair but B C doesn't (because we miss ...
-2
votes
1answer
44 views

I want to extract information from my text file and save it in a separate file

I want to extract information from my file and save it in a separate file. The input file looks like: >Feature NODE_18_1000_cov_1.366138 1 888 CDS db_xref COG:COG3385 inference ab initio ...
-1
votes
4answers
80 views

I want to replace texts in my file

My file looks like: NODE_13_length_50291_cov_8.067223 GTTGCCTAATATCGCCAATCCAGCTCCT NODE_25_length_22118_cov_8.258986 TATCGCCAATCCAGCTCCT NODE_36_length_15273_cov_5.283743 TGCCTAATATCGCCAATCCAG I ...
6
votes
6answers
314 views

Replace each character of white space at the end of each line with '_'

I'm wanting to replace each character of white space at the end of each line with '_'. I found a similar question and answer for leading whitespace. But have failed to reconstruct it for trailing ...
0
votes
4answers
108 views

find most common consecutive two word sequence awk

I have a text file named 'file'. I need to find most common two word sequence using awk, while "this is" and "is this" are considered equal in our count. sample text : name my name is this is our ...
1
vote
2answers
69 views

How to add add '\n' to the end of each line in a log file

I'm using msmtp to create e-mail alerts on some RPi kiosks we have set up at my company, and unfortunately if I try to include the last 5 lines of a log file in the e-mail alert, msmtp doesn't handle ...
-1
votes
0answers
33 views

Merging two files in awk

The fields in both files are delimited by * and in both files, the first field is the same. file 1: D00106863615 .. D00626054332 (4k records) file 2 D00106863615 *01*1*20161002*3**20100921*...
0
votes
1answer
35 views

Need program - “editor” that lets you check-mark lines to keep

I was wondering if anybody could recommend a program or plug-in (for an editor) to solve the following problem: I have a file with a large list (one line per item). I need to manually go through ...
2
votes
2answers
107 views

awk: replace a character on a given line and column if it doesn't match a character on the same given column in the first line

I've tried for ages and I haven't even gotten close. Using awk, how do I: replace every '*' with '-' on every line > 1 in every column, but only if the corresponding column in line 1 is not an '*'? ...
0
votes
1answer
58 views

Can one use backreference as replacement value in sed?

Source file: Linux 2.6.32-754.18.2.el6.x86_64 (myhostname) 03/24/2020 _x86_64_ (64 CPU) 07:32:01 PM 57 25.79 0.00 4.99 0.00 0.00 0.00 0.02 0....
0
votes
1answer
25 views

I would like to analyze a TXT file, but the data is a 1 column array, and it needs to be a in a multi-column array

Is there a way for me to manipulate the data (i.e.; change some feature of, so that it looks like it should, a multi-column array) without having to manually go through it? There are thousands of rows ...
-1
votes
0answers
17 views

salt.state.file.replace: replacing entries in /etc/fstab

I would like to achieve the same as under sed: replacing entries in the /etc/fstab, but as salt.state. How would you implement the following awk-command as salt.state.file.replace? awk '$2 ~ "/tmp"{ ...
-6
votes
3answers
41 views

Modify text file with if-else statement

I have an input file like this: 3 1 10 1 6 9 4 2 4 1 9 2 2 2 1 2 4 2 2 2 9 4 9 2 9 1 2 The conditions that need to be followed: If NF==5 and if $2 and $4 are the same then ...
1
vote
1answer
34 views

Group following fields by matched header line using Awk

I have an output of a cli tool that gives me a list like this one: Listing accountBrand [2020-03-24 18:03:42] 20200211204415-create-accountBrand.js : Initial creation of accountBrand [2020-03-24 ...
0
votes
0answers
28 views

Read data from two CSV files dates different format and divide

I have two csv files one called data.csv and another called GBPUSD.csv In the data.csv there are three columns Name , Date and Amount and in the GBPUSD.csv file there is Date & Price what I need ...
1
vote
5answers
58 views

Removing strings from beginning and end of .txt files in directory?

I have a set of 1-line .txt files in a directory. They all have the same basic format, which I have pasted below labelled 'input': For every file in the directory, I would like to remove the string ...
0
votes
3answers
41 views

Removing text after a delimiter up to a certain point in Bash/Linux

target_id tpm tpm tpm tpm tpm tpm tpm tpm tpm tpm tpm PB.1.1|1:6643-11779(-)|transcript/55100 9207.33 13266.4 15244.3 2844.27 2708.52 2731.91 3193.83 2675....
1
vote
1answer
71 views

Piping errors to my function causes command to be ignored after 1 failure

Context: I have a Bash Script that copies files function log () { read IN if [ "$IN" == "" ]; then : else echo "$datetime"$'\t'"$IN" | tee -a logfile fi } function ...
0
votes
2answers
35 views

Is it possible to merge rows based on the composite unique values of two columns

Sorry I am very new to Linux and I am uncertain the capability of bash for what I want to achieve. I want to merge column-2 values if column-1 and column-3 values are identical. In this case, if ...
0
votes
3answers
63 views

List the output in a table form

I have a output of Student Name: abc Roll Num: 123 Student Name: xyz Roll Num: 124 and I need to print in below format Student Name Roll Num abc 123 xyz 124 can ...
0
votes
2answers
42 views

Merge multiple rows based on a single column value

I have an input file like this: 3 1 10 1 6 9 4 2 4 1 9 2 2 2 1 I want to merge two rows based on the 5th column value, the output would look like this: 3 1 10 1 6 9 4 2 4 9 2 2 ...
8
votes
2answers
2k views

What does $0=$1 mean in awk?

Let's say, I have an input text file (sample_simple.txt) like this: 3 1 10 1 6 9 4 2 4 1 9 2 2 2 1 By using the command: awk '$0=$1" "$1*$2" "$3*$4' sample_simple.txt, I get the following output: ...
-2
votes
0answers
33 views

Remove a field when numeric

Delete field 3 and field 4 only when field 3 is numeric before 172.16.230.160 wsk 172.15.230.160 wsk read 0 written 4442728 after 172.16.230.160 wsk read 0 written 4442728
0
votes
3answers
44 views

Linux pattern searching

If I have a file with lots of directory info like: /home/svdev/src/lib /home/dev/project/doc /home/server/etc I want to get the directory name next to "/home" directory. So for this set of data, my ...
0
votes
1answer
48 views

How to add the content of file A into file B between the pattern match

I have two files: This is the content of file A: etc... this is a test file having \@ and \# and \$ as well looking for awk or sed solution to print this content in another file between the matching ...
0
votes
2answers
79 views

How to regroup string with same length using awk

I have a long list of strings like this: _ah_ _asn_ _ai_ _errr_ _an_ And I would like to transform it like to this "_ah_ai_an_", "_asn_", "_errr_" I have tried : cat file | awk '{ print length, $...
0
votes
1answer
21 views

Split file based on beginning and end XML tag

Can split be used to split a file into multiple segments where the size of each segment is based on the number of matches of an XML element ? For example for below XML split when encounters of "<...
0
votes
1answer
43 views

Find consecutive lines with specific strings and modify file based on table

I have a text manipulation problem that I have been unable to solve. Say I have a text file like the one below (text.txt). There will be cases in which a line with /locus_tag is followed by a line ...
-1
votes
3answers
39 views

Insert lines after character

I have data in a long lined file example: <currency>USD</currency><amount>111</amount><amountf>111/amountf><recv_confirms>2</recv_confirms><fee>22&...
-1
votes
2answers
57 views

awk interpolation of values in column

i'm facing a problem with looong txt files i have this exemple : 542 701.00 789 971 703.00 123 708.00 879 785 974 101 716.00 and i need to replace the missed values in column ...
0
votes
3answers
35 views

How can I display the first n different lines in two files from a shell?

How can I display the first n different lines in two files from a shell ? I've tried with grep -vf but it does not really work. Suppose n = 5 This input: file1 a b c d e f g h i j k l m n o file2 ...
0
votes
3answers
78 views

Regular relational operator '<' not working as expected awk

I have a file like this: 0.2 0.2 0.2 0.2 0.2 0.2 0.2024 0.2025 0.2027 0.2027 0.2029 0.2059 0.2059 0.2059 0.2059 0.2099 0.2099 0.2099 0.2105 0.2113 0.2113 0.2195 0.2198 0.2206 0.2206 0.2206 0.2989 0....
-1
votes
3answers
54 views

How can I check if same text appears in two different files

I want to check (trick or any shortcut) whether text-string exists in two files. The content of file one is a.txt b.txt c.txt d.txt The content of file two is c.txt a.txt d.txt How I can check if ...
0
votes
0answers
55 views

How to ensure that list order stays the same, when lines are missing?

I run into a problem and can't wrap my head around it. Any help is welcome. Example Correct order, given or in external file: red blue green brown black Input: red blue green black Output: red ...
-1
votes
2answers
51 views

Capture only the file name from the directory Path

Requirement Col1 , Col2 , Col3 , Col4 1 , a:/b/c/abc.txt , f:/b/c/abc1.txt , a:/b/c/abc6.txt 2 , b:/bd/a/s/3/d/mon.dat , g:/b/c/abc 2.txt, b:/b/c/ab7....
1
vote
1answer
42 views

Remove newlines using shell command (must work in a KDE global shortcut)

I need a shortcut to strip newlines from the text in the current selection. I want to put my command into a KDE global shortcut, which I already know how to do. The command I came up with is: sh -c '...
0
votes
1answer
16 views

XML read column and add new data based off date format

I have the following data in a XML document NAME 19/02/2019 23/02/2019 23/02/2019 13:21 12.99 0 12.99 Paid PayPal NAME 19/02/2019 23/02/2019 23/02/2019 13:22 12.99 0 12....
0
votes
3answers
76 views

one shell command to find every n-gram works in text

I have a text stream or a file that contains words separated by whitespace.  Like: I have a toy. you may not like it. Every whitespace-separated word may be composed by two or more small words, ...
0
votes
1answer
67 views

Copy the records between header and trailer into a new file including header and trailer [closed]

I have a file which has multiple headers,detail and trailer records.(fixed length) For instance my file looks like Habcd20200317 R123 abc def R234 dfg thg T2 2018907 Hagfd20200316 R346 adf thg R456 ...
0
votes
7answers
93 views

Struggling with GREP commands, removing decimal point correctly

I've written the shell command below to output the CPU's clock speed as an integer: grep -m 1 'cpu MHz' /proc/cpuinfo | grep -o -E '[0-9]+' OUTPUT: 900 063 This is because the exact clock speed is ...

1
2 3 4 5
130