The tag has no usage guidance.

learn more… | top users | synonyms

3
votes
2answers
247 views

get the Xth argument in each line

I would like to get the second argument in a line of output. For example: ps aux gives me an output that contains the cpu-usage %CPU in the third column. How can I filter the output so I have only ...
3
votes
3answers
1k views

Merging content of two files with comparing content of column

I have a problem trying to merge content of two files : file 1 : 873 1.1.1.1 1033 99.99.99.10 1108 78.2.204.174 1257 78.2.8.181 file 2: test1 78.2.8.181 p6 test2 99.99.99.10 p9 test7 1.1.1.1 ...
3
votes
2answers
258 views

add fields from different files

I have two files A and B with the same number of lines. Each of them has multiple columns. I want to add field 6 in file A with field 3 in file B for each line. e.g.: file A: 2769 RUT10EKl3fY 0 ...
3
votes
2answers
352 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
3
votes
2answers
2k views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
3
votes
2answers
213 views

“Fill columns before rows” option in unix column command

According to man column: -x Fill columns before filling rows. This option doesn't seem to do anything. Any idea how to use it?
3
votes
2answers
310 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
3
votes
2answers
223 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
3
votes
3answers
86 views

How do I create uniform columns from text containing both tabs and spaces?

In the simplest form, let's say I have an auto-generated file called file.txt. The contents of file.txt appears as follows: Source Destination Maximum To Maximum ...
3
votes
2answers
939 views

How do I insert new columns into a .csv file?

Example Data: "Monday","12345","Friday" In the above file, I want to add a new column in between 12345 and Friday. How can I do that ?
3
votes
2answers
305 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
3
votes
2answers
3k views

Merging multiple lines based on column 1

I have a file like below.. abc, 12345 def, text and nos ghi, something else jkl, words and numbers abc, 56345 def, text and nos ghi, something else jkl, words and numbers abc, 15475 def, ...
3
votes
2answers
2k views

Cut command not extracting fields properly in aligned columns

I have a text file in which I must cut the fields 3,4,5 and 8: 219 432 4567 Harrison Joel M 4540 Accountant 09-12-1985 219 433 4587 Mitchell Barbara C 4541 Admin Asst ...
3
votes
1answer
243 views

Sort only on the second column [duplicate]

I'm trying to sort a file where typically (not always) the lines are whatever:[A-Ba-b0-9_]: values where fields are separated with :. Some lines do not follow this pattern, I'm not interested in ...
3
votes
1answer
66 views

Why should you avoid “stringently columnar” input formats?

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]: (ii) Expect the output of every program to become the input to ...
3
votes
3answers
887 views

How can I get the first column of a 300GB txt file?

let me first explain my problem in detail. It's actually pretty simple. I have a huge .txt file, 300GB to be more preciese, and I would like to put all the distinct strings from the first column, that ...
3
votes
3answers
3k views

How to reorder a large number of columns?

I'm looking for a pipeable one-liner to reorder a large number of columns (where manually entering the column numbers in e.g. an awk command such as awk '{print $3,$2,$1}' is not feasible). The order ...
3
votes
1answer
425 views

zsh and COLUMNS

I cannot understand the behavior of the $COLUMNS variable under zsh. Here's an example of the sort of behavior I'm referring to. The script $HOME/cols.sh is the following: echo "COLUMNS=$COLUMNS" ...
2
votes
3answers
265 views

How do I reorder or rearrange the output below

Here in my INPUT: 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" ...
2
votes
3answers
2k views

changing column separators in a file

I have a large file, that I need to process and after writing some scripts that don't seem to work properly, I have discovered that a small subset of the lines in the file are actually space separated ...
2
votes
2answers
2k views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
2
votes
5answers
8k views

Sum the values in a column except the header

I have a file as given below -------------------------------------------------------------- Name_Customer Item_Purchased Item_Amount Credit ...
2
votes
3answers
2k views

How does this awk align the spacing?

Can someone please tell me what does this do? cat infile| awk ' { for(i=3;i<=NF;i++){ $2=$2" "$i } printf "%-4s %s\n", $1,$2 ; }' > outfile.txt infile contains: 1 John Smith 2 ...
2
votes
3answers
84 views

Replace a column and preserve spacing

This is a follow up to unix: replace one entire column in one file with a single value from another file I am trying to replace one column of a file (file1) with one specific value from another file ...
2
votes
2answers
388 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
2
votes
2answers
3k views

Align columns in ASCII file

I have a text file that looks like his: #c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 4599 995,274 2523,658 ...
2
votes
2answers
54 views

Issue with column command and color escape codes

I'm colorizing the header of a table formatted with column -ts $'\t' Works well without color codes, but when I add color codes to the first line column doesn't properly align the output. Without ...
2
votes
4answers
226 views

Fill empty lines in specific column with values

I have data of the following format: 2342 1 6473 2 7463 2 3647 2734 1 9343 The lines that do not have a value in the 2nd column are empty in that spot (i.e. there is no space, tab etc.). I am ...
2
votes
2answers
111 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
2
votes
4answers
2k views

Extract lines from a text file with one column equal to the maximum

I have a text file (with header line) called file.txt. I am trying to extract the lines equal to the maximum value of a specific column (I do not know what the max value is): ID t1 q1 t2 q2 q3 1 f ...
2
votes
4answers
482 views

Adding values of the column that have the same data ID?

I want to prepare a uniform daily data from a mixed daily and sub-daily data by adding the sub-daily values and producing a uniform daily data. My sample input and desired out put data looks like ...
2
votes
1answer
5k views

i/o format with printf in bash-shell

I am trying to output the results of my shell script using printf. To keep the results aligned in column, I am using tab as: printf "%s\t %s\t %s\t %s\t %s\t %s\t %s\n" $i $est $var_m $jk $mst $var_M ...
2
votes
1answer
108 views

Produce aligned columns in a shell script

In shell how do I have an even length of spaces by the length of words on my menu? I've tried to get the length of the longest character string and then create each column length by the longest ...
2
votes
2answers
94 views

How do I echo a line with linebreak at the end at window border? [duplicate]

I would like to echo a list all in one line, TAB separated (like ls does with files in one folder) for i in one two some_are_very_long_stuff b c; do echo $i; done will print one line per word: one ...
2
votes
3answers
3k views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
2
votes
1answer
263 views

Transpose several columns (of varying lengths) into rows

A similar question to mine was asked that concerned transposing multiple columns into rows. However, the particular column format that I am dealing with is slightly different in that my file contains ...
2
votes
2answers
196 views

Extract a column from whitespace-separated fields that are not aligned

How can we extract a column from output in which whitespace is not in a regular format? For example, when we type ps -ef, it shows all the process, process id's.... etc. How can we get only the last ...
2
votes
2answers
3k views

Data pivot with awk

I'm trying to pivot a file using awk. This is an example of my input File: VarName;TimeString;VarValue;Validity;Time_ms A;23.11.201215:03:53;1;1;41236627696,7593 ...
2
votes
1answer
39 views

Add/Append a column as first column to a tab delimited file

I have a Tab separated file as follows: A B HM 1 BN 2 I would like to add another column to this file such that this new column becomes the first column of the file as shown below: ...
2
votes
2answers
63 views

Output distribution to a maximal number of columns

I have a bash script foo that provides me with many short lines of output. Think ls -1. To get a more compact presentation, I would like to distribute the output over multiple columns, like ls -C. I ...
2
votes
3answers
454 views

Extracting part from a file column

I want to extract part from a log file column which is like this: xx.xxx.xx.xx#59796: Edit: This is the actual log line: Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view ...
2
votes
1answer
273 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
2
votes
2answers
1k views

How do you truncate all columns in a file to a specific length?

Given columned text in a file that is delimited by a certain character, how do you truncate the width of either a specific column(s) or all columns? The use case is that I am reading some log files ...
2
votes
1answer
71 views

Matching two main columns at the same time between files, and paste supplementary columns into the output file when those main columns match

I have 3 files, each with a unique number of columns, all tab-separated, but some columns are shared between the 3 files. It's the shared columns between the 3 files that I want to use to create some ...
2
votes
3answers
69 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
2
votes
1answer
234 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
2
votes
1answer
50 views

How do you implement totals from usage output generated by a program in a script so that they carry on sanely through repeated execution?

I have a script with a pipeline where the data is throttled by a program. Every 20 minutes, a throughput status is sent on stderr, and I simply redirect this to a file: command | cstream -t -512k -T ...
2
votes
1answer
253 views

text processing-perl

I am a biologist. I would like help of computer experts to help in text file editing. Here is what the input file looks like: ##dsfsd2 ##sdf-sdf sasg 5.6.3 gi34_ex Gen CDS 161 317 . ...
1
vote
3answers
2k views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
1
vote
5answers
177 views

lines to columns with awk

I have to following sample output: <HARDWARE> <NAME>WIN1</NAME> <OS>Windows 7</OS> <IP>1.2.3.4</IP> <DOMAIN>contoso.com</DOMAIN> ...