2
votes
3answers
58 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
3
votes
2answers
54 views

The reverse of `tree` - reconstruct file and directory structure from text file contents?

We all know we can use tree to get a nicely formatted text "visualization" of the structure of a directory; say: $ tree -spugD /usr/include/boost/accumulators/numeric/ ...
2
votes
1answer
197 views

Convert xml subtitle file to srt format

I have a xml subtitle that looks like this: <?xml version="1.0" encoding="utf-8" ?><transcript> <Item from="1.16" duration="4.68"><![CDATA[(Dong-hyuk is coming to see you now.) ...
4
votes
2answers
111 views

Replace common values in two files according to first column

How do I replace common values in two files according to first column? I have file 1 (1.txt): CH063_01084 CH063_01083 CH063_13925 CH063_14473 and file 2 (2.txt): CH063_01083 FGSG_16349 4 ...
1
vote
6answers
84 views

Is there a way to find out the number of lines from a command output?

I am trying to compare command outputs between two servers. I want to find out the number of lines from the output of commands such as ls -l. Is there a way to do so? So far I have not found anything. ...
3
votes
5answers
246 views

How do I extract the content of quoted strings from the output of a command?

I have output from VBoxManage list vms which looks like this: "arch" {de1a1db2-86c5-43e7-a8de-a0031835f7a7} "arch2" {92d8513c-f13e-41b5-97e2-2a6b17d47b67} I need to grab the names arch and ...
1
vote
1answer
156 views

Shell script and crontab to capture file size, name and date in a CSV file and then send via email?

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
1
vote
3answers
133 views

Merging multiple tab-delimited tables based on common columns

I am new to the field and I want to compare and merge 500 hundred tab-delimited files, each with two columns ("Position" and "Poly"). Examples of file1, and file2 (and other files) and the final ...
1
vote
2answers
111 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
2
votes
3answers
221 views

Remove spaces and headers from a dumped database table

I have dumped the database table data into a flat file and below is how the data looks like : (Kindly copy from below ;metier_code ; ;-------------------------; (0 rows affected) ...
1
vote
1answer
135 views

bash scripts - remove duplicate rows with smaller value

Here is an example file: 1 5 20 40 60 2 1 20 20 20 2 2 30 30 30 4 5 40 40 40 7 2 50 30 30 7 1 20 20 20 I would like to remove duplicate rows (with the same value in the first column) with a bigger ...
2
votes
2answers
49 views

How to format textutil -excludedelements

I use Geektool on the Mac to display various information on my desktop. One feature of Geektoool is the ability to use shell mode to launch custom scripts or commands. I am not versed in any type of ...
1
vote
1answer
46 views

How to add a dot to particular lines in LaTeX-document?

I have a LaTeX-document where I have lines of the form \section*{3.1} \addcontentsline{toc}{section}{3.1} or in general \section*{x.y} \addcontentsline{toc}{section}{x.y} for x between 1 and 31 ...
5
votes
5answers
4k views

Parsing a delimited text file in bash as command arguments

I have a text file split up like so: field1,field2,field3 xield1,xield2,xield3 dield1,dield2,dield3 gield1,gield2,gield3 Each of these columns will be a parameter to a program, and I would like ...
2
votes
2answers
670 views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
2
votes
2answers
116 views

Join two files with different number of columns and rows

I have two files. file_1.txt looks like this: R1 C1 C2 C3 C4 C5 R2 C1 C2 C3 C4 C5 R3 C1 C2 C3 C4 C5 R4 C1 C2 C3 C4 C5 ...
2
votes
2answers
123 views

stripping out the last two [end] with sed?

I'm working on a script that improves the fluxbox 'menu', and my script has this line: sed -i 's/\[end]//g' ~/.fluxbox/menu;sed -i '/^\s*$/d' ~/.fluxbox/menu but this has the unfortunate effect of ...
4
votes
3answers
74 views

Fast access to often used document

For several applications I maintain my personal cheatsheets. I use them mostly for vim, latex, vim latex-suite and bash. For example, when I forget a key combination in vim, I browse to the ...
0
votes
2answers
205 views

Formatting quotes from the fortune to use it with conky

I'm using fortune cookie outputs in my conky script. My fortune output is in the following format: Text1 : Some message including( line break tags, e.g. and comma(,),=,semicolon(;) like ...
0
votes
4answers
107 views

Create a duplicate record based on a condition

I need to write a shell script where I have to create a duplicate record based on a condition and then change the value in a column of the new row which will be created. I am trying to create a ...
2
votes
3answers
99 views

Using sed to insert latex commands around headers of a document

I have a documen with headers in ALL CAPS, for example: NAME env - run a program in a modified environment and I want to insert \item[ before the header and ] \hfill \\ after a header ...
1
vote
2answers
77 views

How can I merge multiple lines with spaces (blank line) separating them? [closed]

I'd like to merge the following multiple lines of output so that they form a single line: line 1: ...
1
vote
3answers
175 views

how to calculate the average value of the column that have the row ID?

I want to calculate the average value of my data when they have the same ID (in my specific case, the same day but different time in a day). This is meant to calculate the daily average value of my ...
2
votes
4answers
144 views

Adding values of the column that have the same data ID?

I want to prepare a uniform daily data from a mixed daily and sub-daily data by adding the sub-daily values and producing a uniform daily data. My sample input and desired out put data looks like ...
0
votes
3answers
105 views

selecting rows that have the same value [closed]

I have a problem in selecting rows that have the same value. My data is too huge to go row by row to do that. I want you guys to let me know scripts that can perform this. My data looks like as ...
1
vote
2answers
1k views

how to select, copy columns of different file to form new files

I have four files each contain daily station data. file 1: prec file 2: minT file 3: maxT file 4: wind Here prec, minT, maxT and wind are files that store values of prec, minT, maxT and wind for ...
1
vote
2answers
194 views

What platform to use for programmatically editing text files? [closed]

I just volunteered at work for a project that involves creating a program to edit log files on a server. By edit I mean extract useful information to another location and remove lines of text that is ...
2
votes
1answer
3k views

Copy multiple files into one (append, merge) in single invocation without shell redirection?

I'm looking for some sort of a command that I can use, to copy/append multiple files into one; but without shell redirection (I'd like to try it in call_usermodehelper, see similar issue in ...
2
votes
3answers
412 views

Help searching in files for regex patterns, recursively, with specialized output

My problem is that I need to: Find all lines matching regex_pattern in all files (deep search) in a given root directory For each line that matches, output: File name Line number containing the ...
2
votes
2answers
86 views

Scripting to domain allocation to server

I have two text files: file #1: domain.txt cocacola.com airtel.com pepsi.com file #2: servers.txt 192.0.53.42 , 4 # 4 domains already allocated on server 192.0.53.42 192.53.1.2 , 1 # 1 ...
3
votes
1answer
58 views

How to edit files in-place while performing operation on the values matching the searched pattern?

Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ? Ideally a solution ...
1
vote
1answer
2k views

How can I find and filter a specific column in a .csv file? [duplicate]

I have .csv files with the following structure: cat,dog,mouse,pig,bird,cow,... 21,34,54,566,78,1,... 23,10,12,569,56,4,... 32,20,13,123,56,3,... 34,30,44,322,66,2,... I want to filter the column ...
3
votes
2answers
471 views

Comparing two columns with two columns

I have a file that I need to compare the values in two columns with another two columns. For examples: Item A B C D 1 201 3101 3101 201 2 3101 201 202 3101 3 3101 201 ...
2
votes
5answers
3k views

How can I split a text file into multiple text files using Perl?

I have a file ABC_TabDelim.txt that contains the following: 00:00:00:00 00:00:05:00 01SC_001.jpg 00:00:14:29 00:00:19:29 01SC_002.jpg 00:01:07:20 00:01:12:20 01SC_003.jpg 00:00:00:00 00:00:03:25 ...
1
vote
1answer
1k views

How to match 2 large file and print the difference in shell script

I have 2 files. File1 has 400k numerical records. e.g: 1 2 3 4 5 6 .. and so on File 2 also has 420k numerical records. e.g: 1 2 3 4 6 .. and so on Both these file are in unsorted manner. I want ...
4
votes
2answers
2k views

System calls, AWK, and Bringing in external inputs

awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this? What I'm aiming to ...
20
votes
1answer
870 views

How to implement a horizontal cat? [duplicate]

Standard cat concatenates files line by line (row by row, if you will). I find myself needing a horizontal cat command more and more often recently; i.e. a command that takes a list of files and ...
2
votes
2answers
480 views

How to traverse a file and extract words using their index?

The given file must not be stored in variable and then traversed due to memory size restrictions: Example: var=$(cat FILE) for i in $var do echo $i done How do you traverse all strings in a file ...
2
votes
1answer
819 views

intersection of two files according to the first column

I have two files in file A, there are sequence_numbers in the other file B, there are many columns, and the first column is sequnce numbers, I want to get a files with all the lines in the B with the ...
1
vote
1answer
134 views

Using join with two files fails on larger file sizes

I am having some issues with a script that is using join to join two files. Eaxmple input files contains lines like this: Here are the input file and the output of the join command: ...
3
votes
3answers
267 views

How to make and apply (patch) one side diff?

I want to make a shell script that updates a config file. The file present on the server has information, like various IP addresses. The new file has more code, from new configs added to the system, ...
9
votes
6answers
834 views

Text Manipulation Across multiple lines

I Have a file that has text like this: AAAA BBBB CCCC DDDD 1234 5678 9012 3456 EEEE 7890 etc... And i want to match up the Alphabetic lines with the Numeric lines so they are like this: ...
2
votes
3answers
519 views

dynamic display of a running process?

I am writing a script and acording to that when i run it certain info will be displayed on the output screen. for example say the constant data displayed is: my name is mukesh. i am 27 years old ...
1
vote
3answers
649 views

To list out duplicate filenames in a listing?

bash-3.00$ cat f.txt -rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt -rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt -rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
0
votes
1answer
442 views

Making a simple PHP script process text in a pipe

Last week I wanted to convert some simplified Chinese characters into traditional ones. I found a deceptively nice looking shell one-liner, which today I found out was 20% error prone. So I did some ...
2
votes
2answers
460 views

Comparing files and their properties

I get information of a certain set of files in my mail every day, which looks like this: 58623208 Sep 14 20:08 blbn_blfbe_drv 57904920 Sep 14 19:54 blbn_cycmn 55814208 Sep 14 06:02 clsa_Upd 38912000 ...
3
votes
1answer
167 views

Help improving my AWK skills

This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations) Could some one improve ...
5
votes
2answers
770 views

How to do a text replacement in a big folder hierarchy?

I want to search and replace some text in a large set of files excluding some instances. For each line, I want a prompt asking me if I need to replace that line or not. Something similar to vim's ...
3
votes
4answers
1k views

Replacing text from a list of replacements. Added complication: backslashes

I have a file A that contains pairs of strings, one per line: \old1 \new1 \old2 \new2 ..... I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") ...
4
votes
7answers
4k views

get numeric ASCII value for a character

I'm trying to write a shell script which asks for an ASCII character in the range A-Z or a-z and returns its equivalent numerical value. For example, the output might look like the following: ...