Tagged Questions
3
votes
3answers
66 views
checking data in columns when a data or some may be missing or present?
I am not sure if this possible.
say i have columns like :
Team Colour Game Rainfall PlayerName
XYZ Blue Cricket Yes Kapil
suppose i need to search ...
4
votes
3answers
80 views
Why is sed giving me an error about an unterminated `s'?
I have a set of sed replacements in a bash script and am getting an error about an unterminated `s' command. Here's what the sed line looks like:
sed -n -e "s/TMPFOO1/$FOO1/" -e "s/TMPFOO2/$FOO2/" ...
1
vote
2answers
83 views
shell script to do some text manipulation of text file data structure and slight content changes
Apologies in advance for the wall of text, not sure how else to represent the existing data structure.
I have been handed about a years worth of logs collected every hour from a server.
Sadly, ...
2
votes
2answers
83 views
add “#” in begining to selected lines in File
Requirement : ADD someting (i.e #) in begining of every line in file A which is grep from file B
File A
abcd
abdc
sdfg
asdfa
jon
ram
File B
jon
abcd
grep file B from file A and add "#" ...
0
votes
2answers
70 views
read file record by record and do transformation to the subsequent record based on above record and write into another file
Data file is fixed length file, and I want to read the file record by record and do transformations to the subsequent records based on the prior records (and write the results into another file).
...
6
votes
7answers
621 views
Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”
Let's say I have a file that looks something like this:
23: a, b, c, d
24: b, d, f
25: c, g
and I want to get output like this:
23.a
23.b
23.c
23.d
24.b
24.d
24.f
25.c
25.g
Of course it's not ...
1
vote
3answers
72 views
Get separate used memory info from free -m comand
As a output of free -m command I got following.
total used free shared buffers cached
Mem: 2496 2260 236 0 5 438
-/+ ...
1
vote
3answers
161 views
How to get data count between two strings with multiple occurences in the same file
I have the data in my file as
START-OF-FIELDS
ID
NAME
DEPT
END-OF-FIELDS
START-OF-DATA
1|joy|cs
2|sam|ec
END-OF-DATA
START-OF-FIELDS
ID
NAME
DOB
DEPT
ADDRESS
END-OF-FIELDS
START-OF-DATA
...
4
votes
4answers
253 views
Count lines matching pattern and matching previous line
I want to count the number of times that a role is successfully deleted. The problem is that one line of the log file will show that the script is about to delete the role:
Prepare to remove role X
...
3
votes
2answers
89 views
Concatenate block of lines with following block of lines
I am trying to process some text exported from lecture slides using pdf2text. The bullet points of some slides show up like this:
title for the list
-
-
-
a bullet point text
another bullet point ...
2
votes
3answers
95 views
Linux command line that will fail if any standard out is produced
Is there a simple utility which I can pipe output to on Linux that will:
Return a success code if there is no output on standard out (and / or standard error).
Return a failure code if output is ...
5
votes
2answers
206 views
Find all files with a Python Shebang
I'm trying to run a PEP8 check against a large source tree. The tree is comprised of a mix of files in various languages. The idea is to check all the Python scripts without having to explicitly list ...
4
votes
4answers
745 views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
1
vote
1answer
248 views
How to remove trailing whitespace at the end of the line in given files (more than one)?
I'm sooo fed up with useless spaces in source files that I've even configured vim to show them.
The problem is that I'm very often (if not always) have to deal with useless spaces with my mates ...
1
vote
1answer
344 views
Reading the contents of the file and splitting using ksh
We're using a ksh script for installing one product.
I've another config file, I'd need to read this configuration file from my main script
Content of the Configuration file:
...
2
votes
2answers
506 views
Concatenate multiple files with same header
I have multiple files with the same header and different vectors below that. I need to concatenate all of them but I want only the header of first file to be concatenated and I don't want other ...
3
votes
2answers
628 views
sed: multi-line replace of config block
I have some configuration files that basically look like
(...content...)
# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY
(... more content ...)
# END DYNAMIC BLOCK
(... even more content ...)
Now, in ...
5
votes
4answers
255 views
Tool to create text files from a template
I have to regularly create 100+ text files from templates.
I currently use an overly complicated shell script. I think there is a smarter way to handle this, but I don't know how.
I have a "database" ...
2
votes
3answers
149 views
Indent like first line
How can I indent a file such as its first line?
Example:
A file containing
x=1+2+3+4+
5+6+7+8
+9+10+12
should be converted to
x=1+2+3+4+
5+6+7+8
+9+10+12
I need this inside ...
1
vote
3answers
178 views
Script matching literal pattern over multiple lines?
I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
0
votes
2answers
159 views
Easiest way to find/replace in a file using a list?
I have a file, foo.txt, and a regexp I want to find in that file. Each time I find the regexp, I want to take a line from another file, bar.txt, and substitute it in for the regexp match I found in ...
4
votes
3answers
261 views
Find files that have words in common
What would be the best way to create a list of files that have common words with a given file. For example, if I had:
$ ls
mainFile file1 file2 file file4
$ cat mainFile
exquisite malicious ...
1
vote
1answer
632 views
How to add header and footer to the flat file
I need to add a header and a footer to flat file - I just need to get values from the parameter like system date and report run date in the top line and for the footer I need to get the record count ...
3
votes
1answer
753 views
Identify the number of unique values and then the number of occurrences of those values in the file
I have a data file which contains 15000 lines, but only 400 unique values. I am looking for a way to identify the number of unique values and then the number of occurrences of those values in the ...
4
votes
1answer
691 views
How to find out common elements between two files?
For an example, I have 2 files having following info:
File #1:
12
13
14
15
File #2:
12 1
13 2
14 2
15 6
16 7
17 8
Output File:
1
2
2
6
In the output file, I want only the second column ...
2
votes
5answers
2k views
Bash iterate over a list of strings
I have this bash script:
for opt in string1 string2 string3 ... string99
do somestuff
It works, but I would like to replace the explicit listing of my strings with a file which actually contains ...
1
vote
2answers
113 views
How do I create a bunch of files with different parameters in the body of each file?
So, say, I want to create a bunch of files named this way (where I have an original file "albedo10solar100" with text inside), where I vary albedo from 10 to 60 in steps of 10 and solar from 90 to 100 ...
3
votes
2answers
4k views
How do I delete the first n lines of an ascii file using shell commands?
I have multiple files that contain ascii text information in the first 5-10 lines, followed by well-tabulated matrix information. In a shell script, I want to remove these first few lines of text so ...
-5
votes
3answers
107 views
Script that checks and saves text to file
Can you give example for a script where user is asked to enter name, surname and age. And then check if name and surname contains only letters and age contains only digits? And if the information is ...
9
votes
6answers
602 views
Text Manipulation Across multiple lines
I Have a file that has text like this:
AAAA
BBBB
CCCC
DDDD
1234
5678
9012
3456
EEEE
7890
etc...
And i want to match up the Alphabetic lines with the Numeric lines so they are like this:
...
2
votes
3answers
273 views
dynamic display of a running process?
I am writing a script and acording to that when i run it certain info will be displayed on the output screen.
for example say the constant data displayed is:
my name is mukesh.
i am 27 years old
...
4
votes
2answers
383 views
How to output file & ignoring lines that start with “?”?
I do svn status --show-updates and then I want to either
Q1:
ignore (not to display) lines that start with ?
Q2:
display only lines that start with * Note that there are few spaces before * occurs. ...
3
votes
2answers
262 views
need shell script to grock a csv
Need some pointing in right direction on script to fetch and regex or sed.
Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV ...
2
votes
3answers
2k views
How to search and replace text in all php-files in a directory and it's subdirectories
I am looking for a shell script that recursively traverses all .php files in a directory and performs a search & replace of a particular text pattern.
The search pattern is quite long ( > 5000 ...
4
votes
6answers
2k views
How can I convert tab delimited data to comma delimited data?
I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool:
ec2-describe-snapshots -H --hide-tags > snapshots.csv
The data looks something like this:
SnapshotId VolumeId ...
1
vote
1answer
241 views
Reading two files into an IFS while loop — Is there a way to get a zero diff result in this case?
I have a text file full of several hundred lines of sequences like this:
b 29.
b 52.
c 84.
c 83.
c 94.
c 93.
c 61.
b 38.
c 81.
c 92.
c 28.
c 37.
...
1
vote
3answers
386 views
To list out duplicate filenames in a listing?
bash-3.00$ cat f.txt
-rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt
-rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt
-rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
2
votes
1answer
439 views
How to read a hexdump byte by byte in bash or awk?
This is the hexadecimal output of an IPv6 TCP packet captured with tcpdump:
6000 0000 0018 0620 0000 0000
0000 0000 0000 0000 0000 0001 0000 0000
0000 0000 0000 0000 0000 0002 *0026 0026
0000 ...
1
vote
2answers
763 views
regex find and replace 0x0D, 0x0A characters
I have a text file of a database dump with some line break characters (0x0A0x0D) in the middle of lines. I want to replace them with commas, but I can't do it simply, because those characters are the ...
6
votes
4answers
977 views
How to prepend a license header recursively for all .h and .cpp files in a directory
I'm trying to add a license header to all header files and source files in a project directory using a for loop. This is not working, is there any other approach using sed?
3
votes
2answers
1k views
Replace all occurrences of a character in a variable using korn script
Part of the Korn script I am writing requires that I replace all occurrences of the ' character with two occurrences (''). I am trying to log some SQL that I am generating in this script to a column ...
3
votes
2answers
186 views
pulling strings out of mixed file of strings and binary
I'm trying to get my nokia backup file into an acceptable vcard format. It has valid vcard strings that appear in between binary data. For example:
!@#JUNKCHARACTERS!@#!@BEGIN:VCARD
VERSION:2.1
...
4
votes
1answer
770 views
awk: forcing a return status?
This is a followup to my earlier question.
I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
2
votes
1answer
134 views
reading input in sed
i'm new to shell scripting. I'm learning little by little. I'm more familiar with php which is working toward my disadvantage right now.
Anyway, i'm writing a shell script and i read in input with ...
2
votes
2answers
142 views
Need a script that generates files from words within a single text files
I have the following scenario and my lack of linux knowledge doesn't help a lot.
I have a text file that contains YML values, in the following matters:
coolregion:
min: {z: -99613.0, y: 45.0, x: ...
9
votes
5answers
1k views
Best way to follow a log and execute a command when some text appears in the log
I have a server log that outputs a specific line of text into its log file when the server is up. I want to execute a command once the server is up, and hence do something like the following:
tail -f ...
2
votes
1answer
1k views
Why is 'while.. read.. <<EOL' performing variable expansion, yet <file and | don't?
Aside from the variable expansion mentioned in the question's heading, I also experience another alarming issue when reading inline data to <<EOL... When the data contains a backtick `, it ...
7
votes
2answers
263 views
Strip // Comments From Files
What's the best way to strip all code comments in a given directory? I'd like to strip out all // ... EOL comments, and /* blah \*/ (or /** ... \*/) comments as well.
This is a PHP project, and I'd ...
