Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
2answers
119 views

Find the line number which contains the pattern with custom delimiter

As in example I'm trying to get line numbers which contains the pattern. My pattern contains slashes so I wanted to add custom delimiter. This simple one works: sed -n '/file/=' temp.txt Using ...
1
vote
3answers
78 views

Writing a program for editing .txt data - Python or Unix?

I only have little experience in programming, and I'm currently working on improving my skills. Basically, I need to write a program, that can do some specific processes to some data in a .txt file. ...
2
votes
3answers
55 views

Unix: Add unique names after signs in text

I have a file looking like the following: abcdefghijklmnopqrst // abcdefghijklmnopqrst // abcdefghijklmnopqrst I use the following code to insert a line with // to the beginning of the document (as ...
0
votes
1answer
26 views

Compare two files [on hold]

I want to compare two files in which the data is delimited by a pipe |. Both the files have the same results but only few fields might vary. File 1: A|B|C|D File 2: A|B|F|D I want to compare ...
6
votes
3answers
179 views

current spotify track in dwm status bar

I'm trying to get the current track playing in spotify into my dwm status bar. I got to the point where I get the information with this command. enter code heredbus-send --print-reply --session ...
3
votes
5answers
163 views

How to replace only the third occurrence of the file in unix using only sed command?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
32
votes
8answers
6k views

Linux tools to treat files as sets and perform set operations on them

Does anyone know of any linux tool specifically designed to treat files as sets and perform set operations on them? Like difference, intersection, etc?
1
vote
3answers
46 views

Display distinct values of a list and number of occurences

I have a program that writes on the standard output a list of string values, one per line, and I would like to display in real time the list of distinct values along with the number of occurences for ...
1
vote
2answers
50 views

Copy the first 2 lines from a file and then remove the lines

I need to process a file using a shell script. The file arrives with the data as well as some embedded information releavant to how it is processed. The file is in the following format; ...
3
votes
4answers
2k views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
16
votes
3answers
12k views

How do I remove certain lines (using line numbers) in a file?

There are specific lines that I want to remove from a file. Let's say it's line 20-37 and then line 45. How would I do that without specifying the content of those lines?
1
vote
4answers
2k views

Count tabs per line in text file with utils

I have a tab-tabulated file. I would like to check if every line has the same number of tabs. For first step, I'd like to print the number of tabs for each individual line. I've tried grep -o '\t' ...
-2
votes
0answers
35 views

Pattern matching and replacing with a blank [on hold]

In file 'moreover-raw.txt _2013_02_01' this code given may be repeating.I want to replace 'http://d.mo','?xml tag','!DOCTYPE tag' and '---FIELD_DELIMITER---' with a blank space every time they appear. ...
3
votes
7answers
221 views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
1
vote
3answers
68 views

Replace date with current date

I have a file abc.fil whose content is in the format yyyymmdd.xyz.doc. I want to cat that file abc.fil such that I get the output as 20150127.xyz.doc. Can any one please help me to resolve this ...
10
votes
4answers
5k views

How to insert a new line after every second line?

sed/bash/wharever :D INPUT: sadf asdf yxcv cxv eqrt asdf OUTPUT: sadf asdf yxcv cxv eqrt asdf
17
votes
5answers
19k views

Shell: How to read the bytes of a binary file and print as hexadecimal?

In shell, how can I read the bytes of a binary file I have, and print the output as hexadecimal numbers?
2
votes
3answers
62 views

Replace multiline string in files

I have a number of files I want to update by replacing one multi-line string with another multi-line string. Something along the lines of: * Some text, * something else * another thing And I want ...
2
votes
4answers
55 views

Removing an empty line and the line above it

I have a large .fna file containing millions of DNA sequences. Due to some write errors after truncating my primers, I'm left with some sequence headers with no sequence following it. An example is ...
21
votes
7answers
6k views

A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1

Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ? I have my ...
0
votes
1answer
45 views

Extracting info from blocks inside text file

I have a text file which has blocks like dir1/dir2/dir3/name_run_number1: line1_run_number1_part1 line2_run_number1_part2 line3_run_number1_part3... Each block is separated with a blank line and ...
0
votes
3answers
52 views

how to replace a character within in a multiple double quoted words within a line using sed?

I have text file including text format like this. 012345,"[ThinkPadT2/3Gband,Mac]",Lenovo,"iPhone3G,A1241" How do i replace comma inside words only within double quote by pipe(|) operator using ...
19
votes
6answers
10k views

Removing control chars (including console codes / colours) from script output

I can use the "script" command to record an interactive session at the command line. However, this includes all control characters and colour codes. I can remove control characters (like backspace) ...
0
votes
2answers
30 views

Multiply a column using awk

How do I multiply a column by an integer (say 3) and replace the older value by new one, using awk? Input: Data 9390.900391 10573.089844 80.000000 200.000000 2.700000 Data ...
0
votes
2answers
30 views

How do I download a PDF, then output a chunk of text?

My city water bill is emailed to me every month as a link to the current PDF hosted on their server. So I can easily download this PDF any time I want. How do I take that PDF and extract my account ...
2
votes
5answers
169 views

Edit Nth occurence interactively in a search in vim

Given a text with multiple occurrences of the word foo, I would like to do an interactive search through the text, and upon the Nth occurrence of foo, edit it. I can only find the search and replace ...
1
vote
2answers
47 views

Formatting ifconfig using sed/awk

I'm using ifconfig on OpenSUSE. When I run ifconfig eth0 I get eth0 Link encap:Ethernet HWaddr CE:FD:75:DF:A5:6D inet addr:172.16.4.177 Bcast:172.16.5.255 Mask:255.255.254.0 inet6 ...
1
vote
4answers
42 views

Two operations with sed on the same pattern

I use this following sed command for display names of files with a specific form: ls -1 *|sed 's/^\(.*\).png/\1\/\1,/g' If I have two files named BOH_Contour.png and BOV_Web.png, I obtain ...
1
vote
1answer
69 views

How to remove junk characters from the file generated by script command in linux [duplicate]

I am using script command to record everything from terminal. But when I am opening the generated file its having lots of junk character. Can anyone help me to remove these junk characters from file ...
-4
votes
1answer
63 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
3
votes
1answer
45 views

Show exact number of CRLF occurrences in file

grep -c $'\r\n' file returns the exact same number as grep -c $'\n' file, it shows the number of LF in the file instead of the number of CRLF. (The file definitely does not contain that number of ...
2
votes
2answers
187 views

How to remove all CRLF in file (not replace with LF)

I'd like to remove all carriage returns followed by line feeds (CRLF), such as \r\n in a file. How can I do that? I can't use dos2unix because that replaces CRLF with LF. And I can't use tr because ...
37
votes
2answers
51k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
1
vote
2answers
32 views

Help with Script to Process txt file to CSV

I am trying/attempting to write a script to turn this: Mikrotik01#show unique routing entry | 192.168.255.232 | inc ID Thu June 10 15:21:32.808 CST Unique Entry: 192.168.255.21 Unique Entry: ...
4
votes
3answers
12k views

Shell script to send an alert mail if disk usage exceeds 90%

I am new to shell scripting and want to implement a script on my server which will automatically send email alerts if the disk usage goes over 90% and second email alert when it goes over 95%. My ...
0
votes
4answers
812 views

How to get data count between two strings with multiple occurences in the same file

I have the data in my file as START-OF-FIELDS ID NAME DEPT END-OF-FIELDS START-OF-DATA 1|joy|cs 2|sam|ec END-OF-DATA START-OF-FIELDS ID NAME DOB DEPT ADDRESS END-OF-FIELDS START-OF-DATA ...
2
votes
1answer
227 views

How do I print hyphens if a variable is empty?

I am collecting windows system information using wmic on Linux system. For that purpose, I make various wmic calls and their output is stored in some variables. After required data is collected, I ...
1
vote
5answers
61 views

Using sed to cut a string starting with n spaces and ending with n spaces

How to use sed to cut text starting with x spaces and ending with y spaces? For instance This is my string: kkk 111 fff aaabbb 5d98 ccc mmmppp 9369d and I want to get this output: ...
3
votes
3answers
133 views

Strip off word pattern at the end of string using sed

I have following string: ksh -x SCRIPT2.ksh $BUSSINESSDATE & PID=$!; ksh -x SCRIPT1.ksh $BUSSINESSDATE & PID=$!; I want to strip off the last word & PID=$!;. Desired result is: ksh -x ...
27
votes
14answers
56k views

How to count the number of a specific character in each line?

I was wondering how to count the number of a specific character in each line by some text processing utilities? For example, to count " in each line of the following text "hello!" Thank you! The ...
3
votes
5answers
236 views

Awk - output the second line of a number of .dat files to one file

I have searched through previously asked questions and can't find an answer to what I am trying to do. note, I am new to linux. I have multiple files something like: (in reality i have 80) file1.dat ...
8
votes
3answers
3k views

How to delete commands in history matching a given string?

I need to delete all commands in my history matching a string. I've tried: $ history | grep searchstring | cut -d" " -f2 | history -d -bash: history: -d: option requires an argument $ history | grep ...
2
votes
2answers
67 views

Search and Replace question

I need to find all occurrences of AAsomeArbitraryStringBB and replace it with CCsomeArbitraryStringDD. So AAHelloBB Text AAByeByeBB becomes CCHelloDD Text CCByeByeDD. It's important to note ...
1
vote
1answer
42 views

Computing differences of column entries

I have a textfile with: 1 100 2 80 3 70 4 64 Now I need another file that contains the first column equal to above and the second column should be the difference of to consecutive values of the ...
2
votes
2answers
77 views

Extracting values from a file keyed by multiple keys

Consider a file with key=value pairs, and each key is optionally a concatenation of multiple keys. In other words, many keys can map to one value. The reason behind this is that each key is a ...
2
votes
4answers
92 views

Fastest way to read last line within pattern of multiple files with the same extension

I have in a directory /home/files/ some files that all have the same extension .txt. For example 1.txt 2.txt 3.txt 4.txt 5.txt test.txt These files are all the same in data format but the data are ...
1
vote
2answers
77 views

Oneliner which prints groups and their numbers which start with a '1'?

I have to write an oneliner which print system groups and their identifiers, for all groups whose identifiers start with '1'. For example, the result of cat command on /etc/group is this: I must ...
1
vote
2answers
42 views

splitting a file by column number

I have a file with 10,671 columns. I am trying to split it into 10 smaller files, each with 1000 columns until all columns have been printed, named file_transposed01, file_transposed02, ... ...
0
votes
3answers
3k views

compare two files based on a column and print it

I have two big files of 400,000 lines. I want to compare the column 1 of the second file with column 1 of first file recursively. If they match I would like to print the whole line. It is a sorted ...
1
vote
5answers
231 views

log file: only process entries older than “MMM DD HH:MM:SS”

I need to process my logfile /var/log/mylogs.log. I am only interested in the new entries, lets say, since Jul 20 15:00:00 but the log contains older entries as well. Is there any simple way, how I ...