Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
34 views

Reinterpret terminal output

I have a program that executes some commands in a terminal (for example apt-get update) and stores the output. How can I reinterpret the output, so I can remove all information that is not needed ? ...
4
votes
2answers
85 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
36
votes
3answers
3k views

Why is using a shell loop to process text considered bad practice?

Is using a while loop to process text generally considered bad practice in POSIX shells? As Stéphane Chazelas pointed out, some of the reasons for not using shell loop are conceptual, reliability, ...
-2
votes
0answers
34 views

Longest/shortest Username in linux

I'm trying to configure a script that displays all the users on the computer, the groups they belong to, and the shortest/longest usernames. I have the first part set up with the users and groups but ...
18
votes
9answers
26k views

Command to display first few and last few lines of a file

I have a file with many rows, and each row has a timestamp at the starting, like [Thread-3] (21/09/12 06:17:38:672) logged message from code..... So, I frequently check 2 things from this log file. ...
0
votes
1answer
60 views

need help merging two Unix files

I have 2 pipe delimited files say file1 and file2. Where file1 has say 33 columns and file2 may have 34/35/36 columns. So let's not consider how mant columns we have. What I want to do is compare the ...
0
votes
1answer
39 views

make a table output

if it possible to do a snmpwalk in a perl script and put the output in a table to make a sort of association like for each hostname i have in the same line if index and desc i have the script in bash ...
4
votes
3answers
95 views

Redirect the tail output to a file then stop when match found

there is this log file that data coming in is continuous. what i wanted to happen is to tail -f this log file then redirect it to a file with the ff conditions example of the log files content ...
0
votes
1answer
57 views

Run script on multiple hosts [duplicate]

I have below script which creates user list and also filters ignore.txt file to ignore matching usernames. I would like to run this script on multiple hosts like hostA, hostB, hostC. Like if I run on ...
0
votes
1answer
38 views

Add the number in the column if exists

I need to count Nodes_Util for each Queue. In the example below, queue1 has only one line, but it varies and it could be nothing sometimes and it applies to all queues. I want to dynamically add the ...
5
votes
3answers
2k views

tail -f, but with line numbers

I'm trying to see how many times foo bar appears in /var/log/foo.log within an arbitrary amount of time on a remote server, but nothing that I've tried so far has worked. I've already got a timer ...
0
votes
2answers
61 views

put data in columns

I am trying to put this output in columns can anyone help? Routeur 2 ==> 89.89.156.112 If_Name :vRtrIfName.1.1 :vRtrIfName.1.2 Routeur 2 ==> 89.89.156.112 If_index 546 789 Routeur 2 ==> ...
5
votes
2answers
119 views

Transpose a file, and replace missing values

I have Readings for Names, from Machines and sometimes these readings are replicated. Where a reading was not found, it is left as blank. Name Instrument Rep R1 R2 R3 N1 I1 1 1 2 3 N2 I1 1 1 3 4 ...
60
votes
12answers
29k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
22
votes
7answers
2k views

Is there a tool that combines zcat and cat transparently?

When handling log files, some end up as gzipped files thanks to logrotate and others not. So when you try something like this: $ zcat * you end up with a command line like zcat xyz.log xyz.log.1 ...
1
vote
6answers
67 views

Count the number of 0 and 1 in a file [duplicate]

I have a file (see below), and I would like to count the number of 1's and of 0's: 1 1 1 0 0 1 1 0 0 0 ... I tried to use awk but it doesn't work, for example, to count the number of 1's: awk -F ...
0
votes
4answers
889 views

How to get data count between two strings with multiple occurences in the same file

I have the data in my file as START-OF-FIELDS ID NAME DEPT END-OF-FIELDS START-OF-DATA 1|joy|cs 2|sam|ec END-OF-DATA START-OF-FIELDS ID NAME DOB DEPT ADDRESS END-OF-FIELDS START-OF-DATA ...
2
votes
2answers
100 views

How to compare strings in two files

I have two text files, one file contains entries such as Id Value 1 apple 2 orange 3 mango 4 banana 5 strawberry 6 papaya In other ...
1
vote
1answer
35 views

In-place substitution for paste command [duplicate]

When I run the following code echo '1' > file.txt echo '2' | paste file.txt - > file.txt I would expect the content of file.txt be 1 2, because the paste command as it name suggests should ...
17
votes
4answers
4k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
1
vote
1answer
70 views

How to merge two files in the same row?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
3
votes
1answer
14 views

Find repository names from gitolite info output

How do I extract the names from the gitolite info command output, for further piping into a script? I'm writing a migration script to migrate all my repositories from gitolite to a Gitlab server. ...
2
votes
2answers
58 views

Need to obtain Awk output in specific sequence

Data in file: 1234567-5678907 3456789-1234563 3456789-1234567 . . . . n Now I want the output in the below format: 1234567-5678907,3456789-1234563,3456789-1234563....n I have used below command ...
2
votes
1answer
38 views

ksh/bash Formatting Files through sed by cat file|sed command

I am new to unix platform, (Alternate Approaches can be discussed) I am trying to change the colors of content of a file conditionally. My file is: A B C D Value Value Value Value Value ...
18
votes
4answers
22k views

How can I find out how many lines a text file contains without viewing it?

How can I find how many lines a text file contains without opening the file in an editor or a viewer application? Is there a handy Unix console command to see the number?
1
vote
2answers
27 views

How to merge the lines in Unix (Sun Solaris) in a specific pattern

I have below n number of lines in a flat file: uk,1234560000 uk,6789067000 uk,4567890000 . . . n lines Now I have to make it in below format: ...
1
vote
1answer
38 views

If statement in shell-script only half-working

I've been trying to write a script to rotate my monitor, I'll then assign it to a keyboard key. I've been fairly successful in making it rotate once, however when I run the script again, it doesn't ...
-3
votes
2answers
53 views

match two files and filter out [duplicate]

I have two files file A >TCONS_00000075 gene=XLOC_000030 CCGCCGGCTGCTGCGCGCACCGACTTGTCACCACCCCAGCACGTCCTCCACGTATACAAGCGCTACGGTC ...
1
vote
1answer
41 views

Cut some fields from a text file to another

I have the following command: grep -owf champs.txt t.txt and I want to cut the result of this command from the file t.txt to another file. I know that it can be achieved by a nested command but I ...
8
votes
3answers
21k views

remove particular characters from a variable using bash

I want to parse a variable (in my case it's development kit version) to make it dot(.) free. If version='2.3.3', desired output is 233. I tried as below, but it requires . to be replaced with another ...
24
votes
11answers
7k views

A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1

Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ? I have my ...
3
votes
4answers
235 views

Using sed for brackets

I would like to find out, if, in my file, there is a right number of brackets. I can make my file look like (((()))()(())) ((()))() How can I use sed to compute those brackets and print yes or no ...
14
votes
9answers
485 views

Filter or pipe certain sections of a file

I have an input file with some sections the are demarcated with start and end tags, for example: line A line B @@inline-code-start line X line Y line Z @@inline-code-end line C line D I want to ...
3
votes
2answers
99 views

sed: inserting text before end of <HEAD> section

I need to insert some text into an HTML file before the end of the HEAD section. I'm trying: sed -i "s#</head>#<style> @page { prince-shrink-to-fit: auto } ...
2
votes
3answers
195 views

How to find a specific 'string' and print out the whole line?

I have a file named myfile.csv containing the following: abc:123:myname:1231 def:423324:arbitrary:value:string StackExchange:Unix:Linux From the terminal I run ./myscript.sh def The contents of ...
1
vote
4answers
109 views

matching and merging lines with awk & printing with Solaris

I want to merge the matching pattern with awk on a Solaris system. $ luxadm probe No Network Array enclosures found in /dev/es Found Fibre Channel device(s): Node WWN:2000000c50f1c8da Device ...
2
votes
3answers
228 views

how to concatenate a file with multiple header into one

Can you suggest me how I can combine. a.fasta with 10000 sequences having headers 1, 2, 3, .... 10000 .......................into just one header e.g >1 AAATTTTGGGGCCC >2 ACCCCGGGTTT ...
0
votes
2answers
73 views

How can I filter ping results by time?

How do I filter out any pings that are under a certain ms, ie, only echo ping responses above 500ms to the text file? IP=$1 SECONDS_Between_Pings=$2 ping -i $2 $1 | while read pong; do echo $(date) ...
0
votes
5answers
118 views

Read File word by word

My problem is to read a text file word by word and recovers every word in a variable. I tried to do: while read ligne; do { for ( i=1; i<= length ($ligne); i++); do { var=$(awk '{print $i}' ...
0
votes
2answers
33 views

How to scrape multiple IP's and add in a script as a variable?

I am trying to add in my script an awk statement or a: if; else; then if eth0:4 is a match then put in eth0:4 IP How would you get the IP in your script as a variable if you have multiple IP's ...
0
votes
5answers
175 views

grep to fetch whole line

I have a file like: Chr1 Cufflinks exon 7136 7944 . + . gene_id "XLOC_000001"; transcript_id "TCONS_00000003"; exon_number "5"; gene_name "LOC_Os01g01010"; oId ...
0
votes
1answer
173 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
1
vote
2answers
51 views

How to obtain a plain (raw) binary dump of any file in Unix/Linux?

I have read many similar question on the Stack Exchange community but none of them address a 'clean' binary dump. I have used: xxd and sed combinations to try to obtain a clean binary dump from any ...
4
votes
3answers
123 views

Remove duplicate lines while keeping the order of the lines

[root@server]# awk '!seen[$0]++' out.txt > cleaned awk: (FILENAME=out.txt FNR=8547098) fatal error: internal error Aborted [root@server]# The ""server"" has: 8 GByte RAM + 16 GByte SWAP, x>300 ...
0
votes
3answers
92 views

Remove special characters in a text file

I have a text file which contains in each line a sql query. For each line, I need to remove some special characters. Expl: if I have: select * from Users; insert into Users values ...
2
votes
4answers
82 views

Why isn't this awk command doing a full outer join?

Objective: Merge the contents of two files using common key present in the files file1.txt ========= key1 11 key2 12 key3 13 file2.txt ========= key2 22 key3 23 key4 24 key5 ...
0
votes
0answers
25 views

seeking hints for correct escaping of sed commands WITHIN yaml

I am having a hard time finding the issue why following lines of "code" break my cloud-init config. Basically I am calling a son object as part of the DigitalOcean API which has a yam format - I got ...
2
votes
0answers
63 views

Bash regular expression comparison fail for hex byte \x01

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly. Using GNU bash 4.1.5(1). Is this a bug, or is ...
2
votes
4answers
200 views

How to 'drop' characters from in front of a string?

I have a string that I would like to manipulate. The string is H08W2345678 how would I be able to manipulate it so the output is just W2345678? Similarly if the I wanted to drop the last 4 ...
1
vote
3answers
6k views

compare two columns of different files and print if it matches

I am using solaris 10 and Grep options involving -f doesnt work . I have two pipe separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2 : abc|123| ...