The tag has no wiki summary.

learn more… | top users | synonyms

5
votes
2answers
47 views

Merging/Combining 2 text files according to numeric field

I have 2 text files file_1: 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5 file_2: ...
1
vote
2answers
57 views

Filter Large Data Sets on Dates with Command Line

I have a huge data sets that consists of tab separated values that are stored in rows. An example row looks like this: Dec 4 14:37:36.381651 algorc1 [27751:l@27932]/error: [] - [T0000A124M5] ...
1
vote
3answers
166 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
1
vote
2answers
23 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
0
votes
3answers
45 views

Reduce length of specific column in delimited text file

I need for the 5th column in a delimited text file to be reduced to just the first 5 characters. All other columns must remain unedited. Input: file1.txt column1 column2 column3 ...
1
vote
1answer
38 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
2
votes
3answers
179 views

How do I reorder or rearrange the output below

Here in my INPUT: 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" ...
2
votes
2answers
94 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
6
votes
5answers
178 views

Using awk to identify the number identical columns

I am new to all this, and this is probably a rather simple question, but I am stuck: I have a large number of individual files that contain six columns each (number of rows can vary). As a simple ...
3
votes
2answers
152 views

Select certain column of of each file, paste to a new file

I have 20 tab delimited files with the same number of rows. I want to select every 4th column of each file, pasted together to a new file. In the end, the new file will have 20 columns with each ...
3
votes
2answers
117 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
3
votes
3answers
58 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
1
vote
3answers
346 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
0
votes
5answers
57 views

re-arrange values in one column without affecting other columns using awk or sed

I have a csv file in following format: abc,124,123,2014-08-13,abc def,124,123,2014-08-13,abc ghi,124,123,2014-08-13,abc Now I need to modify only 5th column (2014-08-13) to produce output as given ...
2
votes
2answers
132 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
2answers
259 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
3answers
62 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
2
votes
2answers
65 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
3
votes
1answer
553 views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
2
votes
1answer
135 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
2
votes
2answers
81 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
6
votes
3answers
115 views

Print a multiline variable in a column

I have a script that collects values from a cluster. The values, in some cases, have multiple lines. I have a printf format that specifies how the data should be positioned however, it doesn't take ...
4
votes
2answers
140 views

Replace common values in two files according to first column

How do I replace common values in two files according to first column? I have file 1 (1.txt): CH063_01084 CH063_01083 CH063_13925 CH063_14473 and file 2 (2.txt): CH063_01083 FGSG_16349 4 ...
1
vote
1answer
92 views

Join multiple files in a directory column wise

I would like to join about 100 files in the same directory column-wise. I have tried paste file1 file2 | column -s $'\t' -t This works fine for 2 files. But when I try to do it for 100 files, I ...
0
votes
2answers
890 views

How to get the value from rows and columns from text file in bash?

I am working on a bash project. I need to get all the values (one at a time because I will be using them somewhere else) from a text file. The text file I will be getting is structured like this: ...
1
vote
2answers
65 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
2
votes
1answer
42 views

How do you implement totals from usage output generated by a program in a script so that they carry on sanely through repeated execution?

I have a script with a pipeline where the data is throttled by a program. Every 20 minutes, a throughput status is sent on stderr, and I simply redirect this to a file: command | cstream -t -512k -T ...
3
votes
3answers
276 views

How can I get the first column of a 300GB txt file?

let me first explain my problem in detail. It's actually pretty simple. I have a huge .txt file, 300GB to be more preciese, and I would like to put all the distinct strings from the first column, that ...
0
votes
4answers
421 views

Search for a Value only in one column w.o using awk, sed, or perl

I have a file that looks like this : Marketing Ranjit Singh FULLEagles Dean Johnson Marketing Ken Whillans FULLEagles Karen Thompson Sales Peter RobertsonPARTGolden ...
4
votes
4answers
200 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
11
votes
8answers
1k views

How do I get the character count of words in a particular column?

I have a CSV file like this: abd,123,egypt,78 cde,456,england,45 How can I get the character count of only the 3rd column words? I can't figure out how to get wc to do this.
3
votes
3answers
192 views

Subsetting a file's contents based on value in a column

I have a text file as (for example): apple V$NFY_Q6_rc V=0.741 apple V$HOXA7_01_rc V=0.454 apple V$ALPHACP1_01_rc V=0.695 orange V$SP4_Q5 V=0.747 grapes V$SP1_Q2_01 V=0.677 grapes ...
3
votes
3answers
408 views

Append columns in a text file to after the final row

A smaller version of many data files I have looks like this: 0 0 0 0.05 9.6877884e-06 0.0024898597 0.1 4.2838688e-05 0.0049595502 0.15 0.00016929444 0.0074092494 0.2 0.00036426881 0.009839138 0.25 ...
2
votes
2answers
727 views

Merging multiple lines based on column 1

I have a file like below.. abc, 12345 def, text and nos ghi, something else jkl, words and numbers abc, 56345 def, text and nos ghi, something else jkl, words and numbers abc, 15475 def, ...
2
votes
3answers
86 views

A shell tool to “tablify” input data containing ANSI escape codes

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away ...
0
votes
1answer
307 views

How to display data in columns in two variables with the same format?

I coded the following to display data formatted in columns: str1='hello i am robert and your ma frnd' str2='thisisaverylongword thisistoohehehe andherefinishthe string hereis a other ...
0
votes
3answers
430 views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
5
votes
2answers
101 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
4
votes
5answers
117 views

Reformat each two-line sequence into two columns

I would like to tab my printout: 1 2 3 4 5 6 7 8 to 1 2 5 6 3 4 7 8 How to adjust it using simple command line (awk or sed)??
-1
votes
1answer
2k views

Display column that has a specific string

Content of a sample file is as follow: AA BB CC DD BB 1 2 3 4 9 5 6 7 8 10 I want to display the whole columns that contain BB and DD. In the real case, I do not know which columns contain ...
2
votes
2answers
218 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
3
votes
6answers
167 views

How to combine contents of two files having the same value in a column?

File1: 00:00 274 00:04 476 00:05 450 00:06 499 00:07 373 00:08 206 00:09 471 00:10 154 File2: 00:00 183 00:01 60 00:02 344 00:03 540 00:04 450 00:07 348 00:09 473 00:10 203 Desired output: ...
2
votes
2answers
142 views

How do I use cut to separate by multiple whitespace?

I would like to get the last column of this sample: [ 3] 1.0- 2.0 sec 1.00 MBytes 8.39 Mbits/sec [ 3] 2.0- 3.0 sec 768 KBytes 6.29 Mbits/sec [ 3] 3.0- 4.0 sec 512 KBytes 4.19 Mbits/sec ...
2
votes
3answers
213 views

Extracting part from a file column

I want to extract part from a log file column which is like this: xx.xxx.xx.xx#59796: Edit: This is the actual log line: Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view ...
0
votes
3answers
4k views

Shell script read separately columns from file

When we have file which is a kind of list like : Mary 34 George 45 John 56 Josh 29 using the awk command $1 refers to the first column and $2 to the second column.But once we write a shell script ...
2
votes
2answers
1k views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
1
vote
1answer
155 views

Transpose several columns (of varying lengths) into rows

A similar question to mine was asked that concerned transposing multiple columns into rows. However, the particular column format that I am dealing with is slightly different in that my file contains ...
2
votes
2answers
57 views

How do I echo a line with linebreak at the end at window border? [duplicate]

I would like to echo a list all in one line, TAB separated (like ls does with files in one folder) for i in one two some_are_very_long_stuff b c; do echo $i; done will print one line per word: one ...
2
votes
2answers
563 views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
4
votes
5answers
1k views

A better paste command

I have the following two files ( I padded the lines with dots so every line in a file is the same width and made file1 all caps to make it more clear). contents of file1: ETIAM...... SED........ ...