Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
3answers
2k views

Trouble getting awk output in for loop

I'm trying to create a script that will check a website for a word. I have a few to check so I'm trying to input them via another file. The file is called "testurls". In the file I list the keyword ...
41
votes
5answers
13k views

How can I wrap text at a certain column size?

I know that I can use something like cat test.txt | pr -w 80 to wrap lines to 80 characters wide, but that puts a lot of space on the top and bottom of the printed lines and it does not work right on ...
6
votes
4answers
435 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
3
votes
0answers
132 views

How to use regrex with AWK for string replacement in this example? [duplicate]

Possible Duplicate: How to use regrex with AWK for string replacement in this example? Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem ...
6
votes
3answers
36k views

How to use regex with AWK for string replacement?

Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem Statement and Basic Definitions 23" "#2") ("Exercises 31" "#30") ("Notes and References 42" "#34")) ) ...
2
votes
2answers
538 views

Remove copyright symbol

I'm trying to parse a RSS feed on the command line. The code works so far but the feed contains a copyright symbol which I try to remove (it is latin1 encoded). How do I remove the copyright symbol ...
2
votes
1answer
129 views

Is there a command that outputs a line-numbered file starting the numbering at a specific line?

cat -n file outputs file with line numbering, but how can I specify at which line to start the numbering? Is there a command that can do that? I've thought of perhaps splitting the process into a few ...
3
votes
2answers
259 views

join files with numbered index

I am trying this command: join -a1 -11 file1 file2 > file3 file1 looks like: 1 2 3 4 5 6 7 8 9 10 11 file2: 1 lkj klj lkj 2 lkj lkj lkj 3 7 lkj lkj lkj 8 9 11 ...
21
votes
5answers
5k views

Command line friendly spreadsheets

Does such a thing exist? Text-based spreadsheets that display well in a CLI environment. I'm aware that I could cat foobar.csvand do as I please, but it isn't particularly practical or attractive. I ...
2
votes
5answers
9k views

How to get netmask from bash?

I want to get my netmask on Linux. It is outputted with ifconfig but I want to extract the string.
20
votes
2answers
71k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
2
votes
3answers
2k views

Returning only the portion of a line after a matching pattern

I have a file (file_name) which contains exactly one occurance of the string Result:, at the start of a line. I want to print all the characters after the string Result: in that line until I encounter ...
1
vote
3answers
847 views

To list out duplicate filenames in a listing?

bash-3.00$ cat f.txt -rw-r--r-- 1 mukesh other 102 Nov 5 18:32 f1.txt -rw-r--r-- 1 mukesh other 19 Nov 5 18:32 f2.txt -rw-r--r-- 1 mukesh other 204 Nov 5 18:32 ...
4
votes
4answers
907 views

Find a string (like grep -q) within only a section of a file

I want to write some Bash that can verify that a string exists in a configuration file. I can't change the file format, it belongs to a different application. The file is subdivided into groups named ...
2
votes
1answer
624 views

How to insert a first column to many files + how to convert unix time to normal time

There are many-many files in a directory that has e.x.: the following content: INPUT: root 1324711901 sshd 1272725792 And I'm searching for a solution how to convert the content of the ...
1
vote
2answers
1k views

How can I right-justify variable length output?

So I don't want this: echo "9" 9 Rather I need this, with e.g.: 4 spaces before it: MAGICK "9" 9 So if I try it with 10: MAGICK "10" 10 then it will just have 3 spaces before it. How ...
0
votes
1answer
560 views

Making a simple PHP script process text in a pipe

Last week I wanted to convert some simplified Chinese characters into traditional ones. I found a deceptively nice looking shell one-liner, which today I found out was 20% error prone. So I did some ...
11
votes
4answers
8k views

How to insert a new line after every second line?

sed/bash/wharever :D INPUT: sadf asdf yxcv cxv eqrt asdf OUTPUT: sadf asdf yxcv cxv eqrt asdf
4
votes
4answers
5k views

How to convert hex chars to normal chars?

I tried, but I'm stuck at "escaping" the "sed's": sed -i 's/\\x0/NUL/g' $1 sed -i 's/\\x1/SOH/g' $1 sed -i 's/\\x2/STX/g' $1 sed -i 's/\\x3/ETX/g' $1 sed -i 's/\\x4/EOT/g' $1 sed -i 's/\\x5/ENQ/g' ...
2
votes
1answer
723 views

How to read a hexdump byte by byte in bash or awk?

This is the hexadecimal output of an IPv6 TCP packet captured with tcpdump: 6000 0000 0018 0620 0000 0000 0000 0000 0000 0000 0000 0001 0000 0000 0000 0000 0000 0000 0000 0002 *0026 0026 0000 ...
3
votes
1answer
463 views

Relative line offsets in awk (/pattern/+1 or /pattern/-1)

When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
1
vote
2answers
2k views

regex find and replace 0x0D, 0x0A characters

I have a text file of a database dump with some line break characters (0x0A0x0D) in the middle of lines. I want to replace them with commas, but I can't do it simply, because those characters are the ...
2
votes
2answers
8k views

extract middle section of lines of a text file?

I am writing a php script to parse a large text file do to database inserts from it. However on my host, the file is too large, and I hit the memory limit for php. The file has about 16,000 lines; I ...
4
votes
3answers
4k views

How to bulk convert all the file in a file system branch between Unix and Windows line break format?

Everybody knows :-) that in Windows plain text files lines are terminated with CR+LF, and in Unix&Linux - with LF only. How can I quickly convert all my source code files from one format to ...
9
votes
2answers
3k views

How to replace all the tabs in each file in a file system branch with 2 spaces each?

I code Scala and it's convention is to use double spaces per level for identiation. But using different editors makes my sources filled with a mix of spaces and tabs. I'd like to refactor all the ...
4
votes
1answer
428 views

How do I invoke this most basic usage of csplit?

For a while I've wanted to learn how to use csplit, so tonight I tried practicing. I created a file with some text, and where I wanted to split the text, I wrote two colons (::) surrounded by two ...
2
votes
1answer
242 views

XML text file with `^@` characters in it?

I have an XML file that I need to parse. When I open it in nano, nano give me the message (converted from Mac format). However between each character, there is a ^@ sequence, like so: ^@t^@h^@e^@ ...
5
votes
1answer
166 views

Inderect references aka value of a value in awk

I'm searching for a way to solve this problem in awk. Input: X 1 Y 2 Z 3 X 4 Y 5 The output should look like this: X 5 Y 7 Z 3 I see two problems here: the first one is the indirect reference in ...
12
votes
3answers
1k views

How can I redirect matching lines to a file, and non-matching lines to a different file?

Is there a script/program/utility already available for the following requirement in a optimised way? someCommand | tee >(grep "pattern" > LinesWhichMatch) | grep -v "pattern" > ...
4
votes
4answers
542 views

Adding a TeX macro to the beginning of a file using sed (why doesn't my method work?)

I need to add a line to the beginning of a file. The line in question is \def\submit{} I've got a semi-working solution using sed. I don't know sed, but got it from somewhere on the net, but it ...
2
votes
4answers
182 views

Lines 1,2,3,4,…,n-1, n to lines n,n-1, …,4,3,2,1

I just realized that I could solve this problem by reversing line numbers from 1, 2, 3,...,n to n, n-1, ..., 3,2,1 and then use the same logic as earlier. So I want to know how can I reverse the order ...
0
votes
2answers
151 views

Number of words from a file with the Linux shell

How do I find and many words of a file with the Linux shell? I have text example text example text example text example And I need text example1 text Example2 text example3 text example4 ...
4
votes
3answers
2k views

Inserting text at the beginning of a file with sed via the terminal in Linux [duplicate]

Possible Duplicate: How can I prepend a tag to the beginning of several files? How do I insert text at the beginning of a file via terminal?
2
votes
2answers
514 views

Script to prepend the files path to a file recursively

I work as a grader and files submitted to me end up in a folder named with the student's name which is itself inside a specific folder on my linux share. Currently I go through each source file and ...
2
votes
2answers
862 views

Comparing files and their properties

I get information of a certain set of files in my mail every day, which looks like this: 58623208 Sep 14 20:08 blbn_blfbe_drv 57904920 Sep 14 19:54 blbn_cycmn 55814208 Sep 14 06:02 clsa_Upd 38912000 ...
9
votes
4answers
3k views

How to prepend a license header recursively for all .h and .cpp files in a directory

I'm trying to add a license header to all header files and source files in a project directory using a for loop. This is not working, is there any other approach using sed?
2
votes
2answers
2k views

Extract several pages from a djvu file

I have a djvu file of multiple pages. I wonder how to extract a new djvu file that consists of only a subset of multiple pages? For example, a djvu file has 10 pages, and I would like to extract a ...
1
vote
1answer
234 views

Possible to extract title and pagenum of each page in a djvu file

This is a djvu version of the questions I asked before for pdf files. I have tried a similar idea as fred has given to the previous question. But I am now stuck at how to get the number of pages of a ...
11
votes
3answers
598 views

wc -L reports a line-length of 8 for a tab-char. bug or feature?

-L is a useful feature of wc, or so I thought. It prints the length of the longest line. For some reason it expands a single-byte tab-char to a length of 8. Is there some way to set this to not ...
13
votes
1answer
4k views

reformatting output with aligned columns [duplicate]

Possible Duplicate: A shell tool to “tablify” input data I have a program outputting something like this: abc defgh ijklm nopqr stu vw xyza bcde fghi which I'd like ...
2
votes
3answers
7k views

How to check if a file is tab-delimited and has 8 columns?

I have two things I need to check a text file for: If it is tab-delimited If it has exactly 8 columns How can I check those things and show an error message if either is false?
9
votes
3answers
8k views

How do I grep without leading whitespaces?

I'm greping through a large codebase, and leading whitespaces and tabulation seem to be quite annoying. Is there any way to get rid of it? grep -R "something" ./ E.g, instead of: foo/bar.cpp: ...
2
votes
1answer
432 views

Possible to extract title and pagenum of each page in a pdf file

I was wondering if there are some ways to extract title and pagenum of each page in a pdf file? Either by some applications, or by programming in some programming language with some pdf libraries? ...
2
votes
1answer
611 views

Change the order of displayed fields of arbitrary command output

How could I change the order of fields from a given command? As an example take the output of ls -l. By default the file name is displayed as the last field of the output. What could I pipe that ...
6
votes
4answers
1k views

Compare an old file and new file, but ignore lines which only exist in new file?

I have two files: oldlist - This contains a list of files and a md5 hash for each file. This was generated one year ago. newlist - This also contains a list of files and a md5 hash for each file. ...
9
votes
2answers
2k views

Why does a bash here-string add a trailing newline char?

The following examples show that a newline is added to a here-string. Why is this done? xxd -p <<<'a' # output: 610a xxd -p <<<'a ' # output: 610a0a
4
votes
1answer
195 views

Help improving my AWK skills

This program works as I intended but I feel like I used some clumsy methods to get the out put I desired(especially with my use of print commands and variable declarations) Could some one improve ...
5
votes
1answer
392 views

merge that works at word granularity instead of line

The old diff, patch, and merge programs, and their modern descendents in the form of version control systems work great for managing source code that can be edited concurrently by multiple people. ...
3
votes
5answers
2k views

How to replace a list of placeholders in a text file?

I have a text (configuration) file, but the program that reads the file unfortunately doesn't allow using any kind of variables. So I'd like to use a preprocessor that replaces a set of placeholders ...
0
votes
1answer
1k views

Convert text to image in centos from bash

I want to create an image [jpg, png, doesn't matter] from shell but I don't have convert command. The image is very simple - white background with a medium-size black text in the middle. Is it ...