3
votes
3answers
80 views

How can I delete text before a delimiter in Linux

I need to delete text from a line before delimiter. For example: (123434): hello::{apple,orange,mango}. I need to delete text before first :. i.e. (123434). Is there any command in linux to ...
2
votes
2answers
149 views

Remove all lines in file A which contain the strings in file B

I have a CSV file users.csv with a list of userNames, userIDs, and other data: username, userid, sidebar_side, sidebar_colour "John Lennon", 90123412, "left", "blue" "Paul McCartny", 30923833, ...
3
votes
5answers
163 views

How to prepend a particular text file contents to every text file in a directory and its subdirectories?

I hate to clutter my source code files with licensing/attribution comments but some times it is a requirement. So having dozens of source code files (organized in a tree of subdirectories) written I ...
2
votes
2answers
167 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
19
votes
2answers
634 views

Is there a reason why ls does not have a --zero or -0 option

This question was prompted by questions about ls' -1 option and the recurring tendency of people to ask question and answers that includes processing the output of ls. This reuse of the output ls ...
3
votes
1answer
119 views

Expanding comma-separated list into separate lines

I have a file with some data in this form: Prefix text: First Name, Second Name, Third-- The prefix differs by line. The number of names varies from one to several. The suffix (-- in the example) ...
1
vote
3answers
1k views

Count tabs per line in text file with utils

I have a tab-tabulated file. I would like to check if every line has the same number of tabs. For first step, I'd like to print the number of tabs for each individual line. I've tried grep -o '\t' ...
4
votes
3answers
281 views

OS X/Linux one-liner/script to find the largest recurring group of lines in a text file?

I have a log containing an execution trace where there is infinite recursion eventually terminating when the stack is too deep. There are enough lines and valid included recursion within the larger ...
5
votes
6answers
530 views

What is the easiest way to add a string on the beginning of every line of the file from the command line?

I am looking for a way to add some string to the beginning of every line (same string for every line). Not something customizable but rather something that will be easy to remember and available on ...
5
votes
2answers
419 views

Convert between Unicode Normalization Forms on the unix command-line

In Unicode, some character combinations have more than one representation. For example, the character ä can be represented as "ä", that is the codepoint U+00E4 (two bytes c3 a4 in UTF-8 encoding), ...
1
vote
1answer
442 views

Why does the paste command add line breaks? [duplicate]

I have 2 files: d and t. I would like to be able to combine these files so that the first line of file t is followed by a tab and then the first line of d. For shorter lines, paste t d seems to work ...
3
votes
2answers
206 views

Quickest way to find and replace a string in numerous HTML files

I have numerous HTML files all nested inside different folders contained in a single overall folder. In each of these HTML files I need to replace /contact/index.html With /contact/index.php ...
0
votes
2answers
114 views

Count lines with alphabet in linux

I have a file with a number of lines in a file filename. I want to count how many lines start with character 'a', with 'b' and so on in one go. What command i should execute.?
3
votes
1answer
3k views

Command to transpose (swap rows and columns of) a text file [duplicate]

Is there currently a generic command that will "pivot" input. e.g. #labeled.file name: bob title: code monkey name: joe title: pointy haired is converted to: name title bob code monkey joe ...
3
votes
1answer
256 views

convert bulk of text files to pdf with naming based upon header file

Knowing that "How to convert from text to .pdf" is already well answered here link and here link, I am looking for something more specific: Using Claws-Mail [website] and a Plug-In [RSSyl] to read ...
17
votes
3answers
3k views

echo a file without the first and last lines

Is there a simple way I can echo a file, skipping the first and last lines? I was looking at piping from head into tail, but for those it seems like I would have to know the total lines from the ...
2
votes
1answer
563 views

Regarding generating intersection and union of two csv files

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C. I would ...
3
votes
1answer
856 views

How to do df only on root partition?

How can get df results only for / partition. The partition name/identification (/dev/sda2, /dev/cciss/c0d0p1) could vary on different computers.
3
votes
3answers
272 views

Regarding separate a single file into multiple files according to line separation

Currently, I have a plain text file, A, such as lowest priority very high significance. outstanding very novel In this file, every line contains a sentence. I want to separate this file into ...
6
votes
4answers
359 views

extracting certain info from output

I'm trying to extract certain info from ffmpeg output. Sample ffmpeg output: configuration: --enable-memalign-hack --enable-mp3lame --enable-gpl --disable-vhook --disable-ffplay --disable-ffserver ...
0
votes
2answers
3k views

list the difference and overlap between two plain data set [duplicate]

Possible Duplicate: Linux tools to treat files as sets and perform set operations on them I have two data sets, A and B. The format for each data set is one number per line. For instance, ...
17
votes
5answers
4k views

Command line friendly spreadsheets

Does such a thing exist? Text-based spreadsheets that display well in a CLI environment. I'm aware that I could cat foobar.csvand do as I please, but it isn't particularly practical or attractive. I ...
1
vote
2answers
1k views

regex find and replace 0x0D, 0x0A characters

I have a text file of a database dump with some line break characters (0x0A0x0D) in the middle of lines. I want to replace them with commas, but I can't do it simply, because those characters are the ...
2
votes
2answers
4k views

extract middle section of lines of a text file?

I am writing a php script to parse a large text file do to database inserts from it. However on my host, the file is too large, and I hit the memory limit for php. The file has about 16,000 lines; I ...
8
votes
2answers
2k views

How to replace all the tabs in each file in a file system branch with 2 spaces each?

I code Scala and it's convention is to use double spaces per level for identiation. But using different editors makes my sources filled with a mix of spaces and tabs. I'd like to refactor all the ...
7
votes
3answers
4k views

How do I grep without leading whitespaces?

I'm greping through a large codebase, and leading whitespaces and tabulation seem to be quite annoying. Is there any way to get rid of it? grep -R "something" ./ E.g, instead of: foo/bar.cpp: ...
2
votes
1answer
288 views

rotating buffer type file logging utility

I'am trying to debug a server that throws a lot of logs on stdout/stderr. I need to redirect only last N lines to a file. Something like a rotating buffer feature there in tcpdump's -C & -W flags. ...
10
votes
2answers
994 views

Where has the trailing newline char gone from my command substitution?

The following code best describes the situation.  Why is the last line not outputting the trailing newline char?  Each line's output is shown in the comment.  I'm using GNU bash, version 4.1.5 ...
1
vote
3answers
1k views

Get line numbers for empty lines

I have a large pipe-delimited file where I need to find the line number of all lines where a certain field is empty. I can use cut -d \| -f 6 filename.txt to output just that column. What is a ...
31
votes
7answers
5k views

Linux tools to treat files as sets and perform set operations on them

Does anyone know of any linux tool specifically designed to treat files as sets and perform set operations on them? Like difference, intersection, etc?
2
votes
3answers
3k views

Printing unique lines

Is there some better solution for printing unique lines other than a combination of sort and uniq?
7
votes
1answer
340 views

Is there a way to make perl -i not clobber symlinks?

A friend of mine points out that if you do: perl -pi.bak -e 's/foo/bar/' somefile when "somefile" is actually a symlink, perl does just what the docs say it will do: It does this by renaming ...
8
votes
4answers
3k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
18
votes
10answers
6k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
13
votes
2answers
8k views

Filtering invalid utf8

I have a text file in an unknown or mixed encoding. I want to see the lines that contain a byte sequence that is not valid UTF-8 (by piping the text file into some program). Equivalently, I want to ...
6
votes
2answers
4k views

How do I insert a space every four characters in a long line?

I've got a long line that I want to insert a space every 4 characters, on a single lone line of solid text to make it easier to read, what's the simplest way to do this? also I should be able to input ...
14
votes
2answers
2k views

How can I count the number of different characters in a file?

I would need a program, that outputs the number of the different characters in a file. Example: > stats testfile ' ': 207 'e': 186 'n': 102 Exists any tool, that do this?
5
votes
3answers
290 views

Searching a text file with a single line using regular expressions

as far as i know all unix text processing utilities are reading one line at a time and performing one action on this line. I have a huge file with a single line of text which contains several tokens ...
47
votes
9answers
17k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
5
votes
2answers
1k views

sort output by column

I'd like to take this command find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" | wc -l; done ( from here ). which has an output of basically ./kennel: 11062 ...
2
votes
4answers
259 views

How can I manipulate the content of a file, by duplicating and changing some parts?

I want to manipulate a text-file. It contains many blocks of the form CLASS ...some stuff ... END I want to duplicate each of these blocks and add and remove a line of it's content. Can I script ...
5
votes
4answers
224 views

How could I simplify this command to only use awk?

awk '/user/ {print $1 }' /etc/userdomains | sed 's/://' the format of /etc/userdomains is domain.tld: user otherdomain.tld: otheruser
8
votes
2answers
844 views

How do I take a list and remove it from a file?

I have a long list of domain names that I need to remove from /etc/remotedomains. They're probably not in any particular order in the file. Each domain is on one line. How could I iterate through the ...
9
votes
3answers
11k views

Grabbing the first [x] characters for a string from a pipe

If I have really long output from a command (single line) but I know I only want the first [x] (let's say 8) characters of the output, what's the easiest way to get that? There aren't any delimiters.
8
votes
5answers
476 views

What's the best way to take a segment out of a text file?

What's a good way of extracting say, lines 20 -45 out of a huge text file. Non-interactively of course!