Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
1answer
983 views

How can I add a line to multiple files in a directory tree structure?

Adding a line to multiple files in the same structure is not very hard, but I am stumped on how to do it over a possibly large directory tree structure. So lets say we have your project directory, ...
8
votes
2answers
1k views

Parsing XML's , JSON's and newer data file formats in UNIX using command line utilities

Unix environment has some excellent tools for parsing text in various forms. However off late the data is not in the regular set of formats(CSV,TSV,Record based or some other delimiter based) it used ...
3
votes
2answers
668 views

How is the octal 2-byte output calculated from od

I'm struggling to work out what the octal 2-byte output from the od command is. I understand the octal output (-b flag) but the octal 2-byte is a mystery to me (-o) Can someone shed some light on how ...
6
votes
4answers
6k views

Is it possible in bash, to start reading a file from an arbitary byte count offset?

I want to locate a date which is somewhere in an 8 GB log (text). Can I somewhat bypass a full sequential read, and first do binary splits of the file (size), or somehow navigating the filesystem ...
1
vote
1answer
135 views

svn command to see a tally of my changes

Is there an SVN command that I can run to see a tally of all of my contributions? Something like: added 1800 deleted 15
7
votes
2answers
712 views

What governs the limits of shell brace expansion?

In this example I refer to the expansion of a sequence of integers, but perhaps(?) the limits would be relevant to all aspects of brace expansion.. This more general view is also of interest to me. ...
5
votes
3answers
1k views

Fast way to build a test file with every second listed in YYYY-mm-dd HH:MM:SS format

I want to create a large test file with lines containg dates listed by the second, but my method is taking inordinately long... (or at least, that's how it feels :) ... 43 minutes to create only ...
14
votes
6answers
26k views

How to cut part from log file?

I have an 8 Gb log file (Rails production log). I need to cut it between some dates (lines). Which command could I use to do this?
7
votes
2answers
145 views

How does one find and replace text in all open files with jed?

How does one find and replace text in all open files with jed?
2
votes
1answer
336 views

How does one find and replace text in all open files with LibreOffice?

How does one find and replace text in all open files with LibreOffice?
2
votes
2answers
1k views

How does one find and replace text in all open files with gedit?

How does one find and replace text in all open files with gedit?
4
votes
1answer
2k views

How does one find and replace text in all open files with geany?

How does one find and replace text in all open files with geany?
2
votes
3answers
3k views

Extracting tokens from a line of text

Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...
11
votes
4answers
4k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
20
votes
11answers
9k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
9
votes
2answers
3k views

How to remove multiple lines per occurrence in a file?

Say I have this 857835 line file, containing stuff like this: a1 rubbish1 rubbish2 rubbish3 rubbish4 a1 rubbish5 rubbish6 rubbish7 rubbish8 And I wish to remove all occurences of a1 and the next ...
14
votes
10answers
3k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
3
votes
2answers
1k views

vim - but with scrolling, clickable text, and copy-paste

I like vim. I think touching mice is dirty. But sometimes when my hand is already on a mouse, it sure would be nice to be able to scroll the text or click a location in the text and have the cursor ...
20
votes
5answers
11k views

Filtering invalid utf8

I have a text file in an unknown or mixed encoding. I want to see the lines that contain a byte sequence that is not valid UTF-8 (by piping the text file into some program). Equivalently, I want to ...
11
votes
2answers
10k views

Parse html on Linux

I have a couple of hundred html source code files. I need to extract the contents of a particular <div> element from each of these file so I'm going to write a script to loop through each file. ...
6
votes
3answers
6k views

How do I insert a space every four characters in a long line?

I've got a long line that I want to insert a space every 4 characters, on a single lone line of solid text to make it easier to read, what's the simplest way to do this? also I should be able to input ...
3
votes
1answer
3k views

Remove files, which provided by pipe

I have this command chain: find . -print | grep php | xargs grep 'eval' -sl | xargs wc -l | grep ' [1-2][0-9] ' This provide me this output: 14 ./includes/js/calendar/lang/vgju.php 18 ...
8
votes
3answers
7k views

How to select specific processes with 'top'?

In case of one program having multiple instances, running pidof program gives: `1 2 3` top -p accepts comma-delimited arguments: 1, 2, 3. This means that top -p `pidof program` won't work: ...
3
votes
4answers
2k views

How do I check if a file already has line with “contents” in it?

I need to know if a file aready has line with contents X in it, if not append line. here's the code I've tried. if ! $(grep 'eval $(perl -I$HOME/foo/lib/perl5 -Mlocal::lib=$HOME/foo)' ~/.bashrc) then ...
16
votes
3answers
930 views

Looking for an old classical Unix toolkit textbook

I am looking for a book about the Unix command-line toolkit (sh, grep, sed, awk, cut, etc.) that I read some time ago. It was an excellent book, but I totally forgot its name. The great thing about ...
9
votes
3answers
458 views

Doing two things with output from a command

I have a program texcount that outputs the number of words in my LaTeX document. I can also pipe the output of this to sed to make the newlines TeX linebreaks and write this to a file which I can then ...
13
votes
2answers
6k views

Count how many times each line appears in a file

Say I have a file which contains: A A A B CC I want to have the output like this: A 3 B 1 CC 1
5
votes
2answers
2k views

Count the number of words of particular length from a file

I've used the below command to count the number of words in a file: tr ' ' '\n' < Filename | grep -c "WORD" This returns the word list with counter. Now I want to count the number of words of a ...
15
votes
2answers
2k views

How can I count the number of different characters in a file?

I would need a program, that outputs the number of the different characters in a file. Example: > stats testfile ' ': 207 'e': 186 'n': 102 Exists any tool, that do this?
5
votes
3answers
316 views

Searching a text file with a single line using regular expressions

as far as i know all unix text processing utilities are reading one line at a time and performing one action on this line. I have a huge file with a single line of text which contains several tokens ...
53
votes
11answers
25k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
2
votes
3answers
178 views

Solaris find previous log line

We are running solaris 10 at work. What I'd like to do is find the line previous to another line. Real live example. I can find out that some of our requests are timing out with the following query. ...
8
votes
1answer
783 views

How can I write a new syntax highlighter for gedit?

Some times I wish the particular language I happen to be writing in had syntax highlighting. How can I add that?
5
votes
2answers
2k views

sort output by column

I'd like to take this command find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" | wc -l; done ( from here ). which has an output of basically ./kennel: 11062 ...
2
votes
4answers
270 views

How can I manipulate the content of a file, by duplicating and changing some parts?

I want to manipulate a text-file. It contains many blocks of the form CLASS ...some stuff ... END I want to duplicate each of these blocks and add and remove a line of it's content. Can I script ...
1
vote
1answer
254 views

How can I grep through files whose name begins with `-`? [duplicate]

I've tried this. grep -r thanks * slave-iv grep: invalid option -- '@' Usage: grep [OPTION]... PATTERN [FILE]... Try `grep --help' for more information. Here's ...
21
votes
3answers
38k views

How to merge all (text) files in a directory into one?

I've got 14 files all being parts of one text. I'd like to merge them into one. How to do that?
7
votes
2answers
416 views

Strip // Comments From Files

What's the best way to strip all code comments in a given directory? I'd like to strip out all // ... EOL comments, and /* blah \*/ (or /** ... \*/) comments as well. This is a PHP project, and I'd ...
5
votes
4answers
231 views

How could I simplify this command to only use awk?

awk '/user/ {print $1 }' /etc/userdomains | sed 's/://' the format of /etc/userdomains is domain.tld: user otherdomain.tld: otheruser
8
votes
3answers
1k views

How do I take a list and remove it from a file?

I have a long list of domain names that I need to remove from /etc/remotedomains. They're probably not in any particular order in the file. Each domain is on one line. How could I iterate through the ...
12
votes
3answers
18k views

Grabbing the first [x] characters for a string from a pipe

If I have really long output from a command (single line) but I know I only want the first [x] (let's say 8) characters of the output, what's the easiest way to get that? There aren't any delimiters.
8
votes
5answers
510 views

What's the best way to take a segment out of a text file?

What's a good way of extracting say, lines 20 -45 out of a huge text file. Non-interactively of course!
4
votes
4answers
5k views

Adding a line of text to multiple files

So, I have a bunch of files in a directory, and I need to insert a line of text into each of them. They have essentially the following format: <VirtualHost *:80> ServerAdmin ...
10
votes
3answers
2k views

Parsing log files for frequent IP's

So, I hacked this together while undergoing a DDOS attack to pull naughty ips out of my logs. Anyone have any improvements or other suggestions to make it better? Here's the general idea: pull ...
14
votes
7answers
2k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?