The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
1answer
33 views

Why should you avoid “stringently columnar” input formats?

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]: (ii) Expect the output of every program to become the input to ...
-1
votes
1answer
27 views

Count of Lines within a Range

I'm trying to do a count of all the values (numeric) in the third column of a file that fall within a specific range of numbers. I've found the min and max but have no idea how to do the count now. ...
-1
votes
1answer
72 views

How to set Row & Column in shell Script

I am a beginner in shell scripting. Just made 1 script with the help of others and it is working like we expected. However i want to organize it more. Current Script { date '+%F %T'; ...
-1
votes
3answers
74 views

Need to Split the output in Row & Column

I am just a beginner in shell scripting and currently working on one script which will capture all the Endpoint Name & Ongoing calls on my switch and will send the output to one file. Output is ...
3
votes
3answers
74 views

Specific column level substitution

If I have a file with columnar entries, is it possible to substitute specific entries from a specific columns only and save changes to the same file?? Example a file abc.txt: SR|FRUITS|COLOR| ...
10
votes
5answers
576 views

Combining large amount of files

I have ±10,000 files (res.1 - res.10000) all consisting of one column, and an equal number of rows. What I want is, in essence, simple; merge all files column-wise in a new file final.res. I have ...
4
votes
3answers
135 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
1
vote
3answers
152 views

Find records present in file 1 also in both file 2 and file 3

I have three files file1.txt, file2.txt, file3.txt and they are of same format. I want to select records from file1.txt which are present in both file2.txt and file3.txt based on column 2 and column ...
0
votes
1answer
81 views

How do I center-align a column in UNIX?

I’m trying to center-align a column using the command column. The output should look like this: temperatures 50 12 89 63 How can I do this?
0
votes
1answer
79 views

make a table output

if it possible to do a snmpwalk in a perl script and put the output in a table to make a sort of association like for each hostname i have in the same line if index and desc i have the script in bash ...
0
votes
2answers
72 views

put data in columns

I am trying to put this output in columns can anyone help? Routeur 2 ==> 89.89.156.112 If_Name :vRtrIfName.1.1 :vRtrIfName.1.2 Routeur 2 ==> 89.89.156.112 If_index 546 789 Routeur 2 ==> ...
0
votes
3answers
41 views

Creating columns from text output but with multiple word columns

I have the following output: Person 49 Persons Boss 3 HR 21 CEO 93 System Administrator 2 Using column -t produces the following output: Person 49 Persons Boss 3 HR 21 CEO ...
3
votes
4answers
206 views

How do I extract the first integer from text string in a column of a tab-delimited file?

I work in Medical Genetics and often have delimited text files where in one column (ex. column 5) there is a text string with a "mutation" in our jargon: c.2458C>T or c.45_46delAA or ...
2
votes
2answers
192 views

How do I insert new columns into a .csv file?

Example Data: "Monday","12345","Friday" In the above file, I want to add a new column in between 12345 and Friday. How can I do that ?
2
votes
1answer
67 views

Produce aligned columns in a shell script

In shell how do I have an even length of spaces by the length of words on my menu? I've tried to get the length of the longest character string and then create each column length by the longest ...
0
votes
2answers
59 views

How to add a column if line contains string?

If I have a tab-delimited file containing the following lines: Sample1-Baboon 1 2 3 4 Sample2-Human 1 2 3 4 Sample3-Human 1 2 3 4 Sample4-Baboon 1 2 3 4 How can I add a new column ...
1
vote
1answer
67 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
1
vote
2answers
53 views

Make `column` use a single space as column separators in output

$ echo 'a b' | column -t a b There are 2 spaces between a and b in the output. How can I make column use only one space for separation when possible? EDIT 2: Changed data to include more cases ...
4
votes
5answers
626 views

What command(s) will feed a tab-delimited text file and cut each line to 80 characters?

I've got multiple-line text files of (sometimes) tab-delimited data. I'd like to output the file so I can glance over it - so I'd like to only see the first 80 characters of each line (I designed the ...
5
votes
2answers
71 views

Merging/Combining 2 text files according to numeric field

I have 2 text files file_1: 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5 file_2: ...
1
vote
2answers
306 views

Filter Large Data Sets on Dates with Command Line

I have a huge data sets that consists of tab separated values that are stored in rows. An example row looks like this: Dec 4 14:37:36.381651 algorc1 [27751:l@27932]/error: [] - [T0000A124M5] ...
1
vote
3answers
280 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
1
vote
1answer
42 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
0
votes
3answers
88 views

Reduce length of specific column in delimited text file

I need for the 5th column in a delimited text file to be reduced to just the first 5 characters. All other columns must remain unedited. Input: file1.txt column1 column2 column3 ...
1
vote
1answer
46 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
2
votes
3answers
224 views

How do I reorder or rearrange the output below

Here in my INPUT: 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" ...
2
votes
2answers
178 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
6
votes
5answers
266 views

Using awk to identify the number identical columns

I am new to all this, and this is probably a rather simple question, but I am stuck: I have a large number of individual files that contain six columns each (number of rows can vary). As a simple ...
3
votes
2answers
782 views

Select certain column of of each file, paste to a new file

I have 20 tab delimited files with the same number of rows. I want to select every 4th column of each file, pasted together to a new file. In the end, the new file will have 20 columns with each ...
3
votes
2answers
350 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
4
votes
3answers
117 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
2
votes
3answers
2k views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
0
votes
5answers
72 views

re-arrange values in one column without affecting other columns using awk or sed

I have a csv file in following format: abc,124,123,2014-08-13,abc def,124,123,2014-08-13,abc ghi,124,123,2014-08-13,abc Now I need to modify only 5th column (2014-08-13) to produce output as given ...
3
votes
2answers
211 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
2answers
921 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
3answers
64 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
3
votes
2answers
143 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
3
votes
1answer
1k views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
2
votes
1answer
180 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
2
votes
2answers
139 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
6
votes
3answers
227 views

Print a multiline variable in a column

I have a script that collects values from a cluster. The values, in some cases, have multiple lines. I have a printf format that specifies how the data should be positioned however, it doesn't take ...
5
votes
2answers
162 views

Replace common values in two files according to first column

How do I replace common values in two files according to first column? I have file 1 (1.txt): CH063_01084 CH063_01083 CH063_13925 CH063_14473 and file 2 (2.txt): CH063_01083 FGSG_16349 4 ...
1
vote
1answer
137 views

Join multiple files in a directory column wise

I would like to join about 100 files in the same directory column-wise. I have tried paste file1 file2 | column -s $'\t' -t This works fine for 2 files. But when I try to do it for 100 files, I ...
0
votes
2answers
2k views

How to get the value from rows and columns from text file in bash?

I am working on a bash project. I need to get all the values (one at a time because I will be using them somewhere else) from a text file. The text file I will be getting is structured like this: ...
1
vote
2answers
78 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
2
votes
1answer
45 views

How do you implement totals from usage output generated by a program in a script so that they carry on sanely through repeated execution?

I have a script with a pipeline where the data is throttled by a program. Every 20 minutes, a throughput status is sent on stderr, and I simply redirect this to a file: command | cstream -t -512k -T ...
3
votes
3answers
580 views

How can I get the first column of a 300GB txt file?

let me first explain my problem in detail. It's actually pretty simple. I have a huge .txt file, 300GB to be more preciese, and I would like to put all the distinct strings from the first column, that ...
0
votes
4answers
709 views

Search for a Value only in one column w.o using awk, sed, or perl

I have a file that looks like this : Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles Karen Thompson Sales Peter RobertsonPARTGolden ...
4
votes
4answers
245 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
11
votes
8answers
2k views

How do I get the character count of words in a particular column?

I have a CSV file like this: abd,123,egypt,78 cde,456,england,45 How can I get the character count of only the 3rd column words? I can't figure out how to get wc to do this.