5
votes
5answers
533 views

Parsing a delimited text file in bash as command arguments

I have a text file split up like so: field1,field2,field3 xield1,xield2,xield3 dield1,dield2,dield3 gield1,gield2,gield3 Each of these columns will be a parameter to a program, and I would like ...
2
votes
2answers
246 views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
2
votes
2answers
96 views

Join two files with different number of columns and rows

I have two files. file_1.txt looks like this: R1 C1 C2 C3 C4 C5 R2 C1 C2 C3 C4 C5 R3 C1 C2 C3 C4 C5 R4 C1 C2 C3 C4 C5 ...
2
votes
2answers
116 views

stripping out the last two [end] with sed?

I'm working on a script that improves the fluxbox 'menu', and my script has this line: sed -i 's/\[end]//g' ~/.fluxbox/menu;sed -i '/^\s*$/d' ~/.fluxbox/menu but this has the unfortunate effect of ...
4
votes
3answers
67 views

Fast access to often used document

For several applications I maintain my personal cheatsheets. I use them mostly for vim, latex, vim latex-suite and bash. For example, when I forget a key combination in vim, I browse to the ...
0
votes
2answers
136 views

Formatting quotes from the fortune to use it with conky

I'm using fortune cookie outputs in my conky script. My fortune output is in the following format: Text1 : Some message including( line break tags, e.g. and comma(,),=,semicolon(;) like ...
0
votes
4answers
76 views

Create a duplicate record based on a condition

I need to write a shell script where I have to create a duplicate record based on a condition and then change the value in a column of the new row which will be created. I am trying to create a ...
2
votes
3answers
87 views

Using sed to insert latex commands around headers of a document

I have a documen with headers in ALL CAPS, for example: NAME env - run a program in a modified environment and I want to insert \item[ before the header and ] \hfill \\ after a header ...
1
vote
2answers
53 views

How can I merge multiple lines with spaces (blank line) separating them? [closed]

I'd like to merge the following multiple lines of output so that they form a single line: line 1: ...
1
vote
3answers
148 views

how to calculate the average value of the column that have the row ID?

I want to calculate the average value of my data when they have the same ID (in my specific case, the same day but different time in a day). This is meant to calculate the daily average value of my ...
2
votes
4answers
92 views

Adding values of the column that have the same data ID?

I want to prepare a uniform daily data from a mixed daily and sub-daily data by adding the sub-daily values and producing a uniform daily data. My sample input and desired out put data looks like ...
0
votes
3answers
88 views

selecting rows that have the same value [closed]

I have a problem in selecting rows that have the same value. My data is too huge to go row by row to do that. I want you guys to let me know scripts that can perform this. My data looks like as ...
1
vote
2answers
567 views

how to select, copy columns of different file to form new files

I have four files each contain daily station data. file 1: prec file 2: minT file 3: maxT file 4: wind Here prec, minT, maxT and wind are files that store values of prec, minT, maxT and wind for ...
1
vote
2answers
140 views

What platform to use for programmatically editing text files? [closed]

I just volunteered at work for a project that involves creating a program to edit log files on a server. By edit I mean extract useful information to another location and remove lines of text that is ...
2
votes
1answer
1k views

Copy multiple files into one (append, merge) in single invocation without shell redirection?

I'm looking for some sort of a command that I can use, to copy/append multiple files into one; but without shell redirection (I'd like to try it in call_usermodehelper, see similar issue in ...
2
votes
3answers
279 views

Help searching in files for regex patterns, recursively, with specialized output

My problem is that I need to: Find all lines matching regex_pattern in all files (deep search) in a given root directory For each line that matches, output: File name Line number containing the ...
2
votes
2answers
84 views

Scripting to domain allocation to server

I have two text files: file #1: domain.txt cocacola.com airtel.com pepsi.com file #2: servers.txt 192.0.53.42 , 4 # 4 domains already allocated on server 192.0.53.42 192.53.1.2 , 1 # 1 ...
3
votes
1answer
57 views

How to edit files in-place while performing operation on the values matching the searched pattern?

Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ? Ideally a solution ...
1
vote
1answer
1k views

How can I find and filter a specific column in a .csv file? [duplicate]

I have .csv files with the following structure: cat,dog,mouse,pig,bird,cow,... 21,34,54,566,78,1,... 23,10,12,569,56,4,... 32,20,13,123,56,3,... 34,30,44,322,66,2,... I want to filter the column ...
3
votes
2answers
302 views

Comparing two columns with two columns

I have a file that I need to compare the values in two columns with another two columns. For examples: Item A B C D 1 201 3101 3101 201 2 3101 201 202 3101 3 3101 201 ...
2
votes
5answers
2k views

How can I split a text file into multiple text files using Perl?

I have a file ABC_TabDelim.txt that contains the following: 00:00:00:00 00:00:05:00 01SC_001.jpg 00:00:14:29 00:00:19:29 01SC_002.jpg 00:01:07:20 00:01:12:20 01SC_003.jpg 00:00:00:00 00:00:03:25 ...
1
vote
1answer
581 views

How to match 2 large file and print the difference in shell script

I have 2 files. File1 has 400k numerical records. e.g: 1 2 3 4 5 6 .. and so on File 2 also has 420k numerical records. e.g: 1 2 3 4 6 .. and so on Both these file are in unsorted manner. I want ...
3
votes
2answers
1k views

System calls, AWK, and Bringing in external inputs

awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this? What I'm aiming to ...
20
votes
1answer
705 views

How to implement a horizontal cat? [duplicate]

Standard cat concatenates files line by line (row by row, if you will). I find myself needing a horizontal cat command more and more often recently; i.e. a command that takes a list of files and ...
2
votes
2answers
348 views

How to traverse a file and extract words using their index?

The given file must not be stored in variable and then traversed due to memory size restrictions: Example: var=$(cat FILE) for i in $var do echo $i done How do you traverse all strings in a file ...
2
votes
1answer
598 views

intersection of two files according to the first column

I have two files in file A, there are sequence_numbers in the other file B, there are many columns, and the first column is sequnce numbers, I want to get a files with all the lines in the B with the ...
1
vote
1answer
125 views

Using join with two files fails on larger file sizes

I am having some issues with a script that is using join to join two files. Eaxmple input files contains lines like this: Here are the input file and the output of the join command: ...
3
votes
3answers
246 views

How to make and apply (patch) one side diff?

I want to make a shell script that updates a config file. The file present on the server has information, like various IP addresses. The new file has more code, from new configs added to the system, ...
9
votes
6answers
749 views

Text Manipulation Across multiple lines

I Have a file that has text like this: AAAA BBBB CCCC DDDD 1234 5678 9012 3456 EEEE 7890 etc... And i want to match up the Alphabetic lines with the Numeric lines so they are like this: ...
2
votes
3answers
405 views

dynamic display of a running process?

I am writing a script and acording to that when i run it certain info will be displayed on the output screen. for example say the constant data displayed is: my name is mukesh. i am 27 years old ...
1
vote
3answers
548 views

To list out duplicate filenames in a listing?

bash-3.00$ cat f.txt -rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt -rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt -rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
0
votes
1answer
388 views

Making a simple PHP script process text in a pipe

Last week I wanted to convert some simplified Chinese characters into traditional ones. I found a deceptively nice looking shell one-liner, which today I found out was 20% error prone. So I did some ...
2
votes
2answers
304 views

Comparing files and their properties

I get information of a certain set of files in my mail every day, which looks like this: 58623208 Sep 14 20:08 blbn_blfbe_drv 57904920 Sep 14 19:54 blbn_cycmn 55814208 Sep 14 06:02 clsa_Upd 38912000 ...
3
votes
1answer
160 views

Help improving my AWK skills

This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations) Could some one improve ...
5
votes
2answers
689 views

How to do a text replacement in a big folder hierarchy?

I want to search and replace some text in a large set of files excluding some instances. For each line, I want a prompt asking me if I need to replace that line or not. Something similar to vim's ...
3
votes
4answers
1k views

Replacing text from a list of replacements. Added complication: backslashes

I have a file A that contains pairs of strings, one per line: \old1 \new1 \old2 \new2 ..... I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") ...
4
votes
6answers
3k views

get numeric ASCII value for a character

I'm trying to write a shell script which asks for an ASCII character in the range A-Z or a-z and returns its equivalent numerical value. For example, the output might look like the following: ...
33
votes
7answers
35k views

Looping through files with spaces in the names?

I wrote the following script to diff the outputs of two directores with all the same files in them as such: #!/bin/bash for file in `find . -name "*.csv"` do echo "file = $file"; diff ...