Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

5
votes
2answers
687 views

Use sed or awk to join lines in a certain pattern?

Not sure on how to turn this input: A B C apartner bpartner cpartner into this output: A apartner B bpartner c cpartner I have this pattern in a 80 line file that needs to be 40 line. Also, how ...
4
votes
3answers
218 views

Parsing columnized data

I have some columnized data which I would like to format a little further. It looks like this: $ getStats | grep ESTABLISHED | column -t all tcp 117.54.56.131:80 <- 10.42.100.211:63752 ...
5
votes
3answers
224 views

How to insert text after the last configuration line?

I have to insert a line into a configuration files automatically, but with the added caveat that it should be inserted before a multi-line footer comment and any preceding empty or whitespace-only ...
5
votes
1answer
753 views

Save HTML from clipboard as markdown text

I want to be able to highlight a section of a web page and copy it to the clipboard then save it to my local disk as markdown. I need an efficient way to do that. My current cumbersome method is: ...
2
votes
1answer
17k views

Copy multiple files into one (append, merge) in single invocation without shell redirection?

I'm looking for some sort of a command that I can use, to copy/append multiple files into one; but without shell redirection (I'd like to try it in call_usermodehelper, see similar issue in ...
3
votes
3answers
170 views

Delete lines containing repeated text

I have a file containing two paths on each line. I want to remove the lines containing the same path twice. I work on Linux and Solaris. I would like a one-liner in sed or awk or perl. Example input ...
3
votes
3answers
950 views

Help searching in files for regex patterns, recursively, with specialized output

My problem is that I need to: Find all lines matching regex_pattern in all files (deep search) in a given root directory For each line that matches, output: File name Line number containing the ...
6
votes
3answers
8k views

Editing config file via a bash script

I've been trying to write a simple bash script which I'll be using to install an application and update it's config file. I'm having hard time to have get it's config file modified. # ...
2
votes
3answers
2k views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
3
votes
2answers
3k views

Replacing text between two HTML Comments

I am new to the world of sed/awk and regex in general and have been studying their usage, but have been floundering trying to meet my need: I have an htm page that has a single line notice that needs ...
2
votes
3answers
416 views

Text to yaml conversion

I have a tab separated text file in this format: your-email your-order-id PayPal-transaction-id your-first-name your-second-name s@ess.com 12345 54321 sooky spooky t@tee.com 23456 ...
32
votes
7answers
3k views

Is there a tool that combines zcat and cat transparently?

When handling log files, some end up as gzipped files thanks to logrotate and others not. So when you try something like this: $ zcat * you end up with a command line like zcat xyz.log xyz.log.1 ...
3
votes
2answers
2k views

How to compile a selection of Markdown documents using a shell script

Technical authors are increasingly being encouraged to write in standalone topics. Topics are written so that they make sense when compiled into different packages or when they are read in ...
6
votes
2answers
743 views

bash: colorize second column of output

let's say, I have following output from ls: $ ls -lAhF /bin -rwxr-xr-x 1 root root 905K Apr 10 2010 bash* -rwxr-xr-x 3 root root 31K Dec 26 2011 bunzip2* -rwxr-xr-x 1 root root 505K Nov 15 2010 ...
3
votes
4answers
611 views

Using Linux text editor to prepare content for Microsoft Word

I need to prepare a long manuscript that must be submitted in a Microsoft Word document in order to meet the submission criteria. (Please no "Don't use Word" comments.) I want to use a text editor on ...
4
votes
2answers
79 views

How to add a line from a list into another file when a specified line is found?

Basically I have 2 files. One has a list of router and switch login prompts, like this: user@router1 user@router2 user@switch1 user@switch2 The other file has blocks of XML, like this: ...
5
votes
2answers
256 views

awk + paste for cleaning up PATH?

I have seen this code in .cshrc init files on a few machines. I went through a few awk tutorials in trying to understand how it works, but I am still unable to decrypt it. setenv PATH `echo $PATH | ...
2
votes
3answers
64 views

Replace certain strings with value from former line

I have the following text: Source IP,URL 1.1.1.1,example1.com Blank,example2.com 2.2.2.2,example3.com Blank,example4.com Blank,example5.com Blank,example6.com Blank,example7.com 7.7.7.7,example8.com ...
3
votes
3answers
206 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
3
votes
3answers
236 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
4
votes
2answers
536 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
6
votes
4answers
7k views

Why is sed giving me an error about an unterminated `s'?

I have a set of sed replacements in a bash script and am getting an error about an unterminated `s' command. Here's what the sed line looks like: sed -n -e "s/TMPFOO1/$FOO1/" -e "s/TMPFOO2/$FOO2/" ...
11
votes
1answer
599 views

How to do `head` and `tail` on null-delimited input in bash?

find command can output names of files as a null-delimited strings (if -print0 is provided), and xargs can consume them with -0 option turned on. But in between, it's hard to manipulate that ...
2
votes
2answers
731 views

shell script to do some text manipulation of text file data structure and slight content changes

Apologies in advance for the wall of text, not sure how else to represent the existing data structure. I have been handed about a years worth of logs collected every hour from a server. Sadly, ...
1
vote
1answer
821 views

Using awk to parse AIX's lsuser output in stanza format

I'm creating a simple awk script to parse AIX's lsuser -f ALL output and return information in one line. A sample of the output is something like this. root: id=0 pgrp=something ...
1
vote
2answers
58 views

Better string escaping for hg

So periodically I remove files that are listed as missing by the version control system hg. What I've noticed is that this (admittedly ugly) script I have does not properly escape strings: hg status ...
2
votes
6answers
207 views

How can I separate these two outputs using awk command

Here's my output: MessageID=3990592283244651750-30192b51.13df831d93a.7eb4;EsbTiming=2013-04-12T01:07:46.099 I want only the time. How can I separate them?
4
votes
5answers
213 views

Getting two fields from same line together

I've got a situation where I've got several lines, and I need two different fields out of each of them. To be specific, I have a list of references in a bibliography and I want to get the last name ...
6
votes
3answers
4k views

Split file into multiple small files, separate by the newline symbol

Is there a utility that split file by newline symbol? e.g if a file contains the following lines, aa bbb cccc If I want to split it to 3 files, the desired output would be: aa, bbb And cccc (in 3 ...
2
votes
2answers
98 views

Scripting to domain allocation to server

I have two text files: file #1: domain.txt cocacola.com airtel.com pepsi.com file #2: servers.txt 192.0.53.42 , 4 # 4 domains already allocated on server 192.0.53.42 192.53.1.2 , 1 # 1 ...
5
votes
3answers
9k views

How can I use SED or AWK to replace placeholders in a template file with variable content that contains special characters?

CentOS 6.3 I'm trying to get a small script to send an email containing a copy of email headers in the body (for the purpose of internal reporting). The template file contains the following: ...
4
votes
3answers
87 views

Replace all the occurences of a term in the files contained in a folder

In a folder on my system, I have a C++ project that I'm trying to compile. This process uses some C++ idioms which are not supported by my compiler version. In particular, often inside the code it is ...
4
votes
2answers
398 views

tac command's option creates strange output [comprehension question]

Say I have this file, containing nothing but a b c b a Using tac --separator=a file in BASH [on a Debian based Linux], I get this: # empty line # empty line b c ...
0
votes
1answer
1k views

comparing lines with awk vs while read line

I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line ...
4
votes
2answers
1k views

How to adjust column width in Linux command output

When I used this command on Red Hat Linux /usr/sbin/ss -i I get the following output below: State Recv-Q Send-Q Local Address:Port ...
3
votes
2answers
180 views

Can I make grep treat escaped multiline as a single line?

How can I make tools like grep treat multilines escaped with a backslash as a long single line? Let's say I have some files that looks like this: testvar=test1 \ test2 \ test3 ...
0
votes
0answers
120 views

sybase and iconv

I inherited an old Ubuntu 6.10 server running php 5.1.6 which is using sybase to connect to MSSQL. A user submitted some text containing an en-dash and crashed one of my script. It turns out sybase ...
3
votes
3answers
9k views

awk + print line only if the first field start with string as Linux1

how to print the line in case the first field start with Linux1 for example: echo Linux1_ver2 12542 kernel-update | awk '{if ($1 ~ Linux1 ) print $0;}' the target is to print the line , ...
2
votes
2answers
345 views

How to count the number of words and print the lines that match exactly a given pattern?

I want to count the number of words and print the matched pattern lines which matches exactly with following pattern: abc-ERROR: The input File contains: # abc-ERROR: xyxxkkfgfr # def-Error: ...
4
votes
5answers
3k views

How to remove character and space from a string

Hello I have one file with output Name : ABC Name : CDE Name : ZYS I want to get the result ABC CDE ZYS How can i get it through SED command. I have tried. sed s/'Name' but ...
9
votes
2answers
1k views

Deleting all C comments with sed

I am trying to write a script that will delete all comments and everything in between inside C files in my current directory. I've been using sed, and this is what I have so far: sed -i '/ * [^()] ...
0
votes
2answers
65 views

How to delete lines where the given part of the line is more than 100 chars?

I know I can delete lines that are longer than 100 chars with this: sed '/^.\{100\}..*/d' But I have the following example line: <a ...
0
votes
1answer
120 views

read line by line and take a string in line and assign that string to the subsequent line using perl [duplicate]

Data file is fixed length file, and I want to read file line by line and take a string from the current line, and then append that string to the subsequent line and write into another file. For ...
2
votes
2answers
429 views

add “#” in begining to selected lines in File

Requirement : ADD someting (i.e #) in begining of every line in file A which is grep from file B File A abcd abdc sdfg asdfa jon ram File B jon abcd grep file B from file A and add "#" ...
0
votes
2answers
372 views

read file record by record and do transformation to the subsequent record based on above record and write into another file

Data file is fixed length file, and I want to read the file record by record and do transformations to the subsequent records based on the prior records (and write the results into another file). ...
-3
votes
1answer
184 views

How to transform two delimited ASCII files

These are 2 common delimited, text file transformation problems I'd like to find scripts to do (I dont know if awk or perl is best for these tasks). csh is the default shell on my Solaris10 unix ...
1
vote
2answers
167 views

How to record every transaction using Bash

I have a file with debit credit and withdrawal transactions. I need a bash script that records the balance after every transaction. So file is like this: D:11/02/12:1000.50 C:11/03/12:300 ...
2
votes
4answers
6k views

print last field from line + alternative for awk

Due to technical reason on my Solaris machine, I can't use awk in order to print the last field in line. What are the other alternatives to awk that print the last field from line (using cut or tr ...
2
votes
2answers
396 views

Regex and piped commands with sed

I'm finding really hard to use sed command, plus I can't seem to find well written tutorials. Let me say that I worked with regular expression in other languages (Python, JavaScript, Java), so that ...
7
votes
7answers
839 views

Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”

Let's say I have a file that looks something like this: 23: a, b, c, d 24: b, d, f 25: c, g and I want to get output like this: 23.a 23.b 23.c 23.d 24.b 24.d 24.f 25.c 25.g Of course it's not ...