The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
5answers
2k views

Sum the values in a column except the header

I have a file as given below -------------------------------------------------------------- Name_Customer Item_Purchased Item_Amount Credit ...
2
votes
2answers
125 views

“Fill columns before rows” option in unix column command

According to man column: -x Fill columns before filling rows. This option doesn't seem to do anything. Any idea how to use it?
2
votes
2answers
165 views

add fields from different files

I have two files A and B with the same number of lines. Each of them has multiple columns. I want to add field 6 in file A with field 3 in file B for each line. e.g.: file A: 2769 RUT10EKl3fY 0 ...
2
votes
2answers
66 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
2
votes
2answers
732 views

Merging multiple lines based on column 1

I have a file like below.. abc, 12345 def, text and nos ghi, something else jkl, words and numbers abc, 56345 def, text and nos ghi, something else jkl, words and numbers abc, 15475 def, ...
2
votes
4answers
2k views

Extract lines from a text file with one column equal to the maximum

I have a text file (with header line) called file.txt. I am trying to extract the lines equal to the maximum value of a specific column (I do not know what the max value is): ID t1 q1 t2 q2 q3 1 f ...
2
votes
4answers
225 views

Adding values of the column that have the same data ID?

I want to prepare a uniform daily data from a mixed daily and sub-daily data by adding the sub-daily values and producing a uniform daily data. My sample input and desired out put data looks like ...
2
votes
1answer
2k views

i/o format with printf in bash-shell

I am trying to output the results of my shell script using printf. To keep the results aligned in column, I am using tab as: printf "%s\t %s\t %s\t %s\t %s\t %s\t %s\n" $i $est $var_m $jk $mst $var_M ...
2
votes
3answers
976 views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
2
votes
2answers
133 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
2answers
81 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
2
votes
2answers
57 views

How do I echo a line with linebreak at the end at window border? [duplicate]

I would like to echo a list all in one line, TAB separated (like ls does with files in one folder) for i in one two some_are_very_long_stuff b c; do echo $i; done will print one line per word: one ...
2
votes
2answers
566 views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
2
votes
2answers
94 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
2
votes
2answers
266 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
2answers
143 views

How do I use cut to separate by multiple whitespace?

I would like to get the last column of this sample: [ 3] 1.0- 2.0 sec 1.00 MBytes 8.39 Mbits/sec [ 3] 2.0- 3.0 sec 768 KBytes 6.29 Mbits/sec [ 3] 3.0- 4.0 sec 512 KBytes 4.19 Mbits/sec ...
2
votes
2answers
130 views

Extract a column from whitespace-separated fields that are not aligned

How can we extract a column from output in which whitespace is not in a regular format? For example, when we type ps -ef, it shows all the process, process id's.... etc. How can we get only the last ...
2
votes
3answers
216 views

Extracting part from a file column

I want to extract part from a log file column which is like this: xx.xxx.xx.xx#59796: Edit: This is the actual log line: Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view ...
2
votes
2answers
2k views

Data pivot with awk

I'm trying to pivot a file using awk. This is an example of my input File: VarName;TimeString;VarValue;Validity;Time_ms A;23.11.201215:03:53;1;1;41236627696,7593 ...
2
votes
1answer
1k views

intersection of two files according to the first column

I have two files in file A, there are sequence_numbers in the other file B, there are many columns, and the first column is sequnce numbers, I want to get a files with all the lines in the B with the ...
2
votes
2answers
743 views

How do you truncate all columns in a file to a specific length?

Given columned text in a file that is delimited by a certain character, how do you truncate the width of either a specific column(s) or all columns? The use case is that I am reading some log files ...
2
votes
3answers
62 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
2
votes
1answer
136 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
2
votes
1answer
42 views

How do you implement totals from usage output generated by a program in a script so that they carry on sanely through repeated execution?

I have a script with a pipeline where the data is throttled by a program. Every 20 minutes, a throughput status is sent on stderr, and I simply redirect this to a file: command | cstream -t -512k -T ...
2
votes
3answers
86 views

A shell tool to “tablify” input data containing ANSI escape codes

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away ...
2
votes
1answer
237 views

text processing-perl

I am a biologist. I would like help of computer experts to help in text file editing. Here is what the input file looks like: ##dsfsd2 ##sdf-sdf sasg 5.6.3 gi34_ex Gen CDS 161 317 . ...
1
vote
3answers
169 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
1
vote
1answer
103 views

Why doesn't this unexpand seem to work?

It's my understanding that unexpand can replace multiple spaces with tabs. I have a file, x.x with contents 1 2 3 4 5 and I type expand x.x but the output is still 1 2 3 4 5 and ...
1
vote
2answers
59 views

Filter Large Data Sets on Dates with Command Line

I have a huge data sets that consists of tab separated values that are stored in rows. An example row looks like this: Dec 4 14:37:36.381651 algorc1 [27751:l@27932]/error: [] - [T0000A124M5] ...
1
vote
2answers
65 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
1
vote
3answers
372 views

Find matches for the 5th columns in a pipe delim files

File 1: Connect|20130320000023|UTC|PPP|test3@test3.co.uk|test1@test1.co.uk|0BCBE578| File 2: Connect|20130320000023|UTC|PPP|test1@test1.co.uk|test1@test1.co.uk|0BCBE578| ...
1
vote
2answers
1k views

Align columns in ASCII file

I have a text file that looks like his: #c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 4599 995,274 2523,658 ...
1
vote
3answers
190 views

How to sort and join at the same time?

I have 2 files, one having 2 columns, another having 1 column. The second file is sorted using sort -u. Now the task is I need to join this column with the first column of the first file, which is not ...
1
vote
1answer
38 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
1
vote
3answers
363 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
1
vote
1answer
155 views

Transpose several columns (of varying lengths) into rows

A similar question to mine was asked that concerned transposing multiple columns into rows. However, the particular column format that I am dealing with is slightly different in that my file contains ...
1
vote
1answer
92 views

Join multiple files in a directory column wise

I would like to join about 100 files in the same directory column-wise. I have tried paste file1 file2 | column -s $'\t' -t This works fine for 2 files. But when I try to do it for 100 files, I ...
1
vote
1answer
240 views

Transposing multiple columns is a row

I need to transpose a file from this format 1 2 3 4 6 7 8 9 a b c d f g h i j to this format 1 2 6 7 a b f g 3 4 8 9 c d h k two columns in a single row in the way posted above. Any ...
1
vote
2answers
23 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
982 views

Sorting by one column then the

Say i have a output file with following: 010513 08:04 ROUTER1 192.168.1.1 68.130.5.10 HIT(1) 010513 08:04 ROUTER1 172.168.1.1 28.130.5.10 HIT(1) 010513 08:04 ROUTER1 122.168.1.1 ...
1
vote
1answer
223 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
1
vote
2answers
105 views

Split out data from a column

So at the moment I have some input email addresses and would like to separate the alias and the domain, then copy them to the end of the line. For example, from a line that has ...
0
votes
3answers
432 views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
0
votes
4answers
421 views

Search for a Value only in one column w.o using awk, sed, or perl

I have a file that looks like this : Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles Karen Thompson Sales Peter RobertsonPARTGolden ...
0
votes
3answers
159 views

ps output differs when run from command line and from script

I have to create list of PID's of the running processes. Here is the script ps echo "fields are " ps | cut -d' ' -f1 ps is showing PID TTY TIME CMD 2168 pts/4 00:00:00 sh 2177 ...
0
votes
1answer
1k views

shell script to print rows if there is a value in column 2

1 **2** 3 4 3 1 4 2 1 4 3 2 4 **2** 1 3 How can I print all rows, on which there is a 2 in the second column.
0
votes
3answers
45 views

Reduce length of specific column in delimited text file

I need for the 5th column in a delimited text file to be reduced to just the first 5 characters. All other columns must remain unedited. Input: file1.txt column1 column2 column3 ...
0
votes
2answers
901 views

How to get the value from rows and columns from text file in bash?

I am working on a bash project. I need to get all the values (one at a time because I will be using them somewhere else) from a text file. The text file I will be getting is structured like this: ...
0
votes
1answer
277 views

how to assign values of the column for given conditional statements

I need help with a script to assign values of the column with a given conditional argument. To be specific, here is my data and my condition input data.file: 0.4 3.6 -2.4 0.1 -2.0 1.6 2.2 3.6 ...
0
votes
5answers
57 views

re-arrange values in one column without affecting other columns using awk or sed

I have a csv file in following format: abc,124,123,2014-08-13,abc def,124,123,2014-08-13,abc ghi,124,123,2014-08-13,abc Now I need to modify only 5th column (2014-08-13) to produce output as given ...