4
votes
6answers
108 views

Obtain values of a section of a file

I have a configuration file in the following format. <Title> [part1] A.File = a A.Val = val1 B.File = a B.Val = val1 [part2] A.File = a1 A.Val = val2 B.File = a B.Val = val1 ...
4
votes
6answers
279 views

want the last line when duplicate awk?

I have the file: key value blah blah blah blah blahblah man1 boy1 blah blah man1 boy2 man1 boy1 I do this to remove duplicate lines: awk '/man1/ { print $1,$2} ' file | awk '!x[$0]++' and the ...
4
votes
4answers
234 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
1
vote
3answers
86 views

How do I Copy and Paste lines between a start and end keyword?

I have two text files, and I want to copy a bunch of lines from one to the other. File one has a list of packages, and I want to copy it to list two. This list of packages isn't at the start of file ...
2
votes
1answer
38 views

Extracting lines around keys from a file

I have a large file with around 15 million lines. The odd lines contains the keys (but not only the keys) and the even lines are the data. That is, the file is something like: WRTZ Tyrosine ...
1
vote
6answers
84 views

Is there a way to find out the number of lines from a command output?

I am trying to compare command outputs between two servers. I want to find out the number of lines from the output of commands such as ls -l. Is there a way to do so? So far I have not found anything. ...
6
votes
3answers
176 views

Shell script to search files for identical text entries

Here's a nice teaser for a shell scripting guru: Take a directory with several text files. Can be a few up to ~1000. All files contain an identifier on a given line (always the same line). Identify ...
3
votes
2answers
118 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
13
votes
13answers
2k views

How to read an IP address backwards?

If we have this string (IP address): 192.168.1.1 How can I derive the (DNS reverse record form) from this string, so it will be shown like 1.1.168.192.in-addr.arpa using a shell script?
1
vote
3answers
61 views

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and atrr.txt is a small file. I want to match the first 2 columns of the first file to the second column of atrr.txt and print the ...
1
vote
1answer
63 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
0
votes
2answers
162 views

Compare first column in different files

I want to compare one file with other files in the same directory. file1.txt contains: move 34#123# get 11#278 put 33#29#567#23 file1.txt should compare with files 1.txt and 2.txt. 1.txt ...
2
votes
3answers
67 views

A shell tool to “tablify” input data containing ANSI escape codes

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away ...
4
votes
2answers
432 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
2
votes
2answers
60 views

Process each line from text file and delete related files from disk

I used the find command to find several files and folders and output them to a text file like so: nohup find /oradba -name '*soapr*' 2>/dev/null >find_soapr_db40.txt & So instead of ...
8
votes
1answer
106 views

Filter the clipboard content without using an intermediate file

I'm trying to modify the text that is in my clipboard, remove certain lines of that contains some strings and then get the output for further manipulation (I prefer coping to my clipboard only if I ...
16
votes
9answers
1k views

Grep lines but let the first line through [duplicate]

Consider the following output from df. Filesystem Size Used Avail Use% Mounted on /dev/sda1 23G 6.1G 16G 29% / udev 10M 0 10M 0% /dev tmpfs 397M 420K ...
4
votes
3answers
144 views

cycle through one command's output and use in another command [duplicate]

How can I use the output of one command - line-by-line - into another command? I know how to do this with the | pipe symbol, but this uses the entire output in one command. I would like to go line by ...
3
votes
1answer
1k views

How to merge two files in corresponding row?

Now,I have two files: aaaa.txt: a=0; b=1; c=2; bbbb.txt: d=3 e=4 f=5 I want to merge aaaa.txt and bbbb.txt to cccc.txt. cccc.txt as follow: a=0;d=3 b=1;e=4 c=2;f=5 So, what can I do ...
3
votes
4answers
9k views

How to find and replace string without use command Sed?

As we all know, sed is greatly efficient to find and replace string, for example find 'a' and replace it to 'b': sed 's/a/b/g'. Is it possible to do this with other command or shell script instead ...
5
votes
6answers
535 views

What is the easiest way to add a string on the beginning of every line of the file from the command line?

I am looking for a way to add some string to the beginning of every line (same string for every line). Not something customizable but rather something that will be easy to remember and available on ...
0
votes
1answer
976 views

shell script to print rows if there is a value in column 2

1 **2** 3 4 3 1 4 2 1 4 3 2 4 **2** 1 3 How can I print all rows, on which there is a 2 in the second column.
2
votes
3answers
142 views

How to extract the file size with du

I'm using du to get the file size of the result archive in a packager script: > du -smh archive.zip 51M archive.zip I'd like to assign just the 51M part to a variable, to be able to print: ...
1
vote
5answers
621 views

Print nth line before the matched line, Matching line and nth line from the matched line

I want Print nth line before the matched line, Matching line and nth line from the matched line where "n" is greater than 2. Here's an example of my data file (the line numbers below are not part of ...
0
votes
2answers
559 views

How to remove n number of lines from a file using sed command? [duplicate]

I want to remove a particular number of lines after the given pattern is matched. Can anyone please tell me how that can be done using the sed command ?
2
votes
2answers
364 views

Extract substring from 'list' of strings

I have a file with lines with IDs, and I'm extracting certain substrings from each line. I've seen this answer, which is great, but I don't know how to get it to execute on each line. I figure xargs ...
5
votes
5answers
2k views

How to append data of multiple files into single file?

For example: I have 01.txt and 02.txt files and need to append data of these files into new file new.txt. It should append data based on 01.txt followed by 02.txt files. Before this I have to delete ...
1
vote
3answers
143 views

Using makefile assignments from a shell script

For example: srcdir = ../libc # Src directory prefix = /usr/local // Installation path The first line is from a makefile and second from config.make. I need to access the variable in a ...
1
vote
1answer
83 views

What are the commands that you can use in the beginning of a sed command?

I initially learned to do sed search and replace like so: sed -e 's/<stuff>/<replacement>/' However I recently learned that these will do something different based on what command ...
2
votes
2answers
375 views

shell script to do some text manipulation of text file data structure and slight content changes

Apologies in advance for the wall of text, not sure how else to represent the existing data structure. I have been handed about a years worth of logs collected every hour from a server. Sadly, ...
0
votes
2answers
227 views

read file record by record and do transformation to the subsequent record based on above record and write into another file

Data file is fixed length file, and I want to read the file record by record and do transformations to the subsequent records based on the prior records (and write the results into another file). ...
11
votes
4answers
924 views

How to do a continous 'wc -l' with gnu texttools?

I know of course that cat logfile.txt | wc -l 120 will tell me the number of lines in a file. Whereas tail -f logfile.txt will show me the new lines that another program writes to logfile.txt. ...
3
votes
3answers
60 views

How to separate numerical values from identifiers

I'm currently writing a shell script that seperate values from their identifiers (retrieved from grep). For example, if I grep a certain file I will retrieve the following information: value1 = 1 ...
6
votes
2answers
5k views

How to cut (select) a field from text line counting from the end?

I know how to select a field from a line using the cut command. For instance, given the following data: a,b,c,d,e f,g,h,i,j k,l,m,n,o This command: cut -d, -f2 # returns the second field of the ...
1
vote
1answer
1k views

How to match 2 large file and print the difference in shell script

I have 2 files. File1 has 400k numerical records. e.g: 1 2 3 4 5 6 .. and so on File 2 also has 420k numerical records. e.g: 1 2 3 4 6 .. and so on Both these file are in unsorted manner. I want ...
3
votes
3answers
5k views

split long line on a delimiter

What command can I use to split input like this: foo:bar:baz:quux into this? foo bar baz quux I'm trying to figure out the cut command but it seems to only work with fixed amounts of input, like ...
6
votes
2answers
887 views

How can I use column to delimit on tabs and not spaces?

I'd like to use Unix column to format some text. I have fields delimited by tabs, but within each field there are also spaces. column delimits on white space (tabs and spaces). How can I make column ...
7
votes
10answers
1k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
2
votes
1answer
814 views

intersection of two files according to the first column

I have two files in file A, there are sequence_numbers in the other file B, there are many columns, and the first column is sequnce numbers, I want to get a files with all the lines in the B with the ...
17
votes
3answers
3k views

echo a file without the first and last lines

Is there a simple way I can echo a file, skipping the first and last lines? I was looking at piping from head into tail, but for those it seems like I would have to know the total lines from the ...
3
votes
2answers
417 views

Limit stdout from a continuously running process

I haven't had much luck finding an answer to my problem, but maybe I'm not asking for it correctly. I have a process I startup like the following: nohup ping 127.0.0.1 > log.txt >2>&1 & ...
4
votes
3answers
688 views

Find files that have words in common

What would be the best way to create a list of files that have common words with a given file. For example, if I had: $ ls mainFile file1 file2 file file4 $ cat mainFile exquisite malicious ...
2
votes
4answers
3k views

How to do an IF statement from the result of a executed command

I am trying to do an IF statement from the output of an executed commmand. Here is how I am trying to do it, but it doesn't work. Does anyone know the right way to do this? if [ "`netstat -lnp | ...
3
votes
2answers
1k views

Skip first 3 byte of a file

I am using AIX 6.1 ksh shell I want to use one liner to do something like this cat A_FILE | skip-first-3-bytes-of-the-file I want to skip the first 3 bytes of the first line, is there a way to ...
8
votes
2answers
6k views

How to print only the duplicate values from a text file?

Suppose there is a column of numeric values like following: File1: 1 2 3 3 3 4 4 4 5 6 I want the output: 3 4 That is, only the repeated lines. Is there any command line tools to find this ...
1
vote
1answer
397 views

Join problem: throwing error, join extra operand

I want to join 3 files on a column which has sorted unique numeric values (those files have only one column of values though) and starts with same prefix for an example "usi". Now, while I am doing ...
1
vote
2answers
460 views

Substitute pattern within a file with the content of other file

I have a text file (devel.xml). I added the word REPLACETHIS to it in order to replace this string with the content within a different file (temp.txt). The closest thing I have is this: sed -i -e ...
1
vote
1answer
369 views

What's the most appropriate way of parsing values from this output?

I have some netstat output I'm scripting and want to parse. What's the most efficient way of parsing values from the output below? It's important to point out that I'm using the old /bin/sh shell ...
4
votes
2answers
2k views

delete first line of file only if blank using sed

I have a solution for this in awk: awk '{if (NR==1 && NF==0) next};1' somefile but was unable to find one that worked in sed. E.g., sed -i.bak '/^$/{1,1d;}' somefile ended up deleting ...
3
votes
3answers
608 views

Using CSV line as command parameters

I have a CSV file like: Name,Age,Address Daniel Dvorkin,28,Some Address St. 1234 (... N ...) Foo Bar,90,Other Address Av. 3210 And I have a command that take this parameters: ./mycommand ...