The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
3answers
209 views

How do I reorder or rearrange the output below

Here in my INPUT: 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" ...
2
votes
3answers
862 views

changing column separators in a file

I have a large file, that I need to process and after writing some scripts that don't seem to work properly, I have discovered that a small subset of the lines in the file are actually space separated ...
2
votes
4answers
158 views

How do I extract the first integer from text string in a column of a tab-delimited file?

I work in Medical Genetics and often have delimited text files where in one column (ex. column 5) there is a text string with a "mutation" in our jargon: c.2458C>T or c.45_46delAA or ...
2
votes
2answers
2k views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
2
votes
5answers
4k views

Sum the values in a column except the header

I have a file as given below -------------------------------------------------------------- Name_Customer Item_Purchased Item_Amount Credit ...
2
votes
3answers
881 views

How does this awk align the spacing?

Can someone please tell me what does this do? cat infile| awk ' { for(i=3;i<=NF;i++){ $2=$2" "$i } printf "%-4s %s\n", $1,$2 ; }' > outfile.txt infile contains: 1 John Smith 2 ...
2
votes
2answers
274 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
2
votes
2answers
158 views

“Fill columns before rows” option in unix column command

According to man column: -x Fill columns before filling rows. This option doesn't seem to do anything. Any idea how to use it?
2
votes
2answers
209 views

add fields from different files

I have two files A and B with the same number of lines. Each of them has multiple columns. I want to add field 6 in file A with field 3 in file B for each line. e.g.: file A: 2769 RUT10EKl3fY 0 ...
2
votes
2answers
116 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
2
votes
4answers
2k views

Extract lines from a text file with one column equal to the maximum

I have a text file (with header line) called file.txt. I am trying to extract the lines equal to the maximum value of a specific column (I do not know what the max value is): ID t1 q1 t2 q2 q3 1 f ...
2
votes
4answers
317 views

Adding values of the column that have the same data ID?

I want to prepare a uniform daily data from a mixed daily and sub-daily data by adding the sub-daily values and producing a uniform daily data. My sample input and desired out put data looks like ...
2
votes
1answer
3k views

i/o format with printf in bash-shell

I am trying to output the results of my shell script using printf. To keep the results aligned in column, I am using tab as: printf "%s\t %s\t %s\t %s\t %s\t %s\t %s\n" $i $est $var_m $jk $mst $var_M ...
2
votes
2answers
2k views

Align columns in ASCII file

I have a text file that looks like his: #c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 4599 995,274 2523,658 ...
2
votes
3answers
1k views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
2
votes
2answers
200 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
2answers
128 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
2
votes
2answers
69 views

How do I echo a line with linebreak at the end at window border? [duplicate]

I would like to echo a list all in one line, TAB separated (like ls does with files in one folder) for i in one two some_are_very_long_stuff b c; do echo $i; done will print one line per word: one ...
2
votes
2answers
973 views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
2
votes
2answers
105 views

How do I insert new columns into a .csv file?

Example Data: "Monday","12345","Friday" In the above file, I want to add a new column in between 12345 and Friday. How can I do that ?
2
votes
1answer
51 views

Produce aligned columns in a shell script

In shell how do I have an even length of spaces by the length of words on my menu? I've tried to get the length of the longest character string and then create each column length by the longest ...
2
votes
2answers
158 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
2
votes
3answers
1k views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
2
votes
2answers
725 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
2answers
154 views

Extract a column from whitespace-separated fields that are not aligned

How can we extract a column from output in which whitespace is not in a regular format? For example, when we type ps -ef, it shows all the process, process id's.... etc. How can we get only the last ...
2
votes
2answers
2k views

Data pivot with awk

I'm trying to pivot a file using awk. This is an example of my input File: VarName;TimeString;VarValue;Validity;Time_ms A;23.11.201215:03:53;1;1;41236627696,7593 ...
2
votes
3answers
306 views

Extracting part from a file column

I want to extract part from a log file column which is like this: xx.xxx.xx.xx#59796: Edit: This is the actual log line: Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view ...
2
votes
2answers
844 views

How do you truncate all columns in a file to a specific length?

Given columned text in a file that is delimited by a certain character, how do you truncate the width of either a specific column(s) or all columns? The use case is that I am reading some log files ...
2
votes
3answers
62 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
2
votes
1answer
162 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
2
votes
1answer
42 views

How do you implement totals from usage output generated by a program in a script so that they carry on sanely through repeated execution?

I have a script with a pipeline where the data is throttled by a program. Every 20 minutes, a throughput status is sent on stderr, and I simply redirect this to a file: command | cstream -t -512k -T ...
2
votes
3answers
115 views

A shell tool to “tablify” input data containing ANSI escape codes

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away ...
2
votes
1answer
244 views

text processing-perl

I am a biologist. I would like help of computer experts to help in text file editing. Here is what the input file looks like: ##dsfsd2 ##sdf-sdf sasg 5.6.3 gi34_ex Gen CDS 161 317 . ...
1
vote
3answers
137 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
1
vote
3answers
258 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
1
vote
1answer
149 views

Why doesn't this unexpand seem to work?

It's my understanding that unexpand can replace multiple spaces with tabs. I have a file, x.x with contents 1 2 3 4 5 and I type expand x.x but the output is still 1 2 3 4 5 and ...
1
vote
2answers
238 views

Filter Large Data Sets on Dates with Command Line

I have a huge data sets that consists of tab separated values that are stored in rows. An example row looks like this: Dec 4 14:37:36.381651 algorc1 [27751:l@27932]/error: [] - [T0000A124M5] ...
1
vote
2answers
74 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
1
vote
3answers
532 views

Find matches for the 5th columns in a pipe delim files

File 1: Connect|20130320000023|UTC|PPP|test3@test3.co.uk|test1@test1.co.uk|0BCBE578| File 2: Connect|20130320000023|UTC|PPP|test1@test1.co.uk|test1@test1.co.uk|0BCBE578| ...
1
vote
3answers
206 views

How to sort and join at the same time?

I have 2 files, one having 2 columns, another having 1 column. The second file is sorted using sort -u. Now the task is I need to join this column with the first column of the first file, which is not ...
1
vote
1answer
45 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
1
vote
1answer
187 views

Transpose several columns (of varying lengths) into rows

A similar question to mine was asked that concerned transposing multiple columns into rows. However, the particular column format that I am dealing with is slightly different in that my file contains ...
1
vote
1answer
50 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
1
vote
1answer
38 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
126 views

Join multiple files in a directory column wise

I would like to join about 100 files in the same directory column-wise. I have tried paste file1 file2 | column -s $'\t' -t This works fine for 2 files. But when I try to do it for 100 files, I ...
1
vote
1answer
310 views

Transposing multiple columns is a row

I need to transpose a file from this format 1 2 3 4 6 7 8 9 a b c d f g h i j to this format 1 2 6 7 a b f g 3 4 8 9 c d h k two columns in a single row in the way posted above. Any ...
1
vote
2answers
50 views

Make `column` use a single space as column separators in output

$ echo 'a b' | column -t a b There are 2 spaces between a and b in the output. How can I make column use only one space for separation when possible? EDIT 2: Changed data to include more cases ...
1
vote
1answer
1k views

Sorting by one column then the

Say i have a output file with following: 010513 08:04 ROUTER1 192.168.1.1 68.130.5.10 HIT(1) 010513 08:04 ROUTER1 172.168.1.1 28.130.5.10 HIT(1) 010513 08:04 ROUTER1 122.168.1.1 ...
1
vote
1answer
247 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
1
vote
2answers
114 views

Split out data from a column

So at the moment I have some input email addresses and would like to separate the alias and the domain, then copy them to the end of the line. For example, from a line that has ...