Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

9
votes
4answers
99 views

Script to extract selected entries from a bibtex file

I have a large bibtex file with many entries where each entry has the general structure @ARTICLE{AuthorYear, item = {...}, item = {...}, item = {...}, etc } (in some cases ARTICLE might be a ...
4
votes
3answers
120 views

Parsing string token with bash

I have this output (from a previous command): aaa something bbb someother ccc blabla For each line, I would like to create a file whose name is the first token, and whose ...
3
votes
2answers
232 views

Basic sed command on large one-line file: couldn't re-allocate memory

I have a 250 MB text file, all in one line. In this file I want to replace a characters with b characters: sed -e "s/a/b/g" < one-line-250-mb.txt It fails with: sed: couldn't re-allocate ...
4
votes
4answers
118 views

Howto find and move a text line inside a file?

I need to change a text line position inside a file; from one line position to another one, underneath a defined text. For example, I have the following text block (on /etc/pam.d/system-auth file) ...
16
votes
9answers
831 views

Grep lines but let the first line through [duplicate]

Consider the following output from df. Filesystem Size Used Avail Use% Mounted on /dev/sda1 23G 6.1G 16G 29% / udev 10M 0 10M 0% /dev tmpfs 397M 420K ...
0
votes
1answer
102 views

Copy file while replacing text in it

I'm writing a script to publish a webapp. While copying files, I must replace a placeholder current_date in a file with the current date. I would start with something like this to define the date ...
2
votes
6answers
1k views

bash - replace space with new line

How can I replace spaces with new lines on an input like: /path/to/file /path/to/file2 /path/to/file3 /path/to/file4 /path/to/file5 etc... To obtain the following: /path/to/file /path/to/file2 ...
2
votes
2answers
692 views

Convert csv to HTML table

I have a Medical.csv file with rows of following format, field: 'participation.type', displayName: 'program_type', type: 'String',path:'participation' field: 'participation.program', ...
8
votes
4answers
172 views

Split the file and put it in corresponding data structure?

I have a file which contains the single line below: {machineA=[0, 1024, 4, 1028], machineB=[1, 1025, 5, 1029]} in which I have two datasets: machineA=[0, 1024, 4, 1028] machineB=[1, 1025, 5, 1029] ...
2
votes
2answers
63 views

Write In A File at Specific Location

I am trying to create a script that will add a user to the UserDir enabled list or will remove it from UserDir disabled list in userdir.conf. For the sake of my question, let's say I have the ...
0
votes
1answer
90 views

script to calculate the average

We have a big file in which three columns are there that is hour, response time and service name. These fields are separated by space. We need write a script to calculate the number of occurrence of ...
5
votes
3answers
278 views

Extracting fixed width records with no delimiter from a single line

I need to extract strings of text from a single file containing one very long line of text with no delimiters. Using the sample line below, these are the following known facts: ...
1
vote
3answers
186 views

remove particular characters from a variable using bash

I want to parse a variable (in my case it's development kit version) to make it dot(.) free. If version='2.3.3', desired output is 233. I tried as below, but it requires . to be replaced with another ...
4
votes
2answers
88 views

Remove duplicates from basenames in two files

I have two long lists of filenames, and I want to merge these two by excluding duplicated filenames in the sense of basename. For example, I have something like ../data_folderA/file_1 ...
-1
votes
2answers
129 views

How to manipulate output from commands with shell scripts [closed]

Simple question, having a ls -l command it makes a list on the command prompt and if we want to manipulate it we pipeline it with awk command, for instance if we want to to show specific files. Is ...
4
votes
3answers
89 views

cycle through one command's output and use in another command [duplicate]

How can I use the output of one command - line-by-line - into another command? I know how to do this with the | pipe symbol, but this uses the entire output in one command. I would like to go line by ...
5
votes
5answers
508 views

Parsing a delimited text file in bash as command arguments

I have a text file split up like so: field1,field2,field3 xield1,xield2,xield3 dield1,dield2,dield3 gield1,gield2,gield3 Each of these columns will be a parameter to a program, and I would like ...
3
votes
1answer
88 views

terminal command that prints out tabs, spaces, and newlines as \t \s and \n

I wrote a terminal program in c that converts tabs to spaces. And I would like to see that it works and how many spaces were converted from tabs. The problem is when I create a file called input.txt ...
10
votes
3answers
234 views

Split text file into short lines for reading?

Is there a program that can take a plain-text file with long lines, and add a newline after a certain number of characters, only splitting at words, to make it readable? For example, take this: Lorem ...
2
votes
2answers
244 views

Bash: append columns to table

If you want to append a column (not a line) to a table, and the column has the same number of elements for sure, how can you do it? Given a set of files, which keeps changing each day, you want to ...
2
votes
2answers
95 views

Join two files with different number of columns and rows

I have two files. file_1.txt looks like this: R1 C1 C2 C3 C4 C5 R2 C1 C2 C3 C4 C5 R3 C1 C2 C3 C4 C5 R4 C1 C2 C3 C4 C5 ...
2
votes
2answers
115 views

stripping out the last two [end] with sed?

I'm working on a script that improves the fluxbox 'menu', and my script has this line: sed -i 's/\[end]//g' ~/.fluxbox/menu;sed -i '/^\s*$/d' ~/.fluxbox/menu but this has the unfortunate effect of ...
4
votes
4answers
281 views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
2
votes
1answer
98 views

Printing UTF-8 (including Chinese) text

I am trying to print a large quantity (several megabytes) of UTF-8 encoded text which consists of Chinese and Latin characters (and maybe a sprinkling of others). I would like to print it in several ...
4
votes
2answers
61 views

Concatenating fields from lines with different numbers of fields

Given input like this: x y a b c t p q w w t a b c d p q r I'd like to concatenate fields from field 3 up to but not including the last one. If field 3 is the last field, I'd like to insert a ...
1
vote
2answers
98 views

using awk to create a LaTex table from my data in a text file

So I have used awk a few times to add or remove text from each line of a text file. What I would like to learn how to do now is to add some characters to each line in multiple places. What this is ...
5
votes
3answers
200 views

removing redundancy from output columns

What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns A B B A A C A D C A D A B C C B A B and B A ...
4
votes
5answers
121 views

How to print all the lines not containing a string along with adjacent lines?

I'm trying to wrap my mind around this task. I want from the following output, to exclude all the lines that contain Auto-installed: 1 and the 2 lines before. I was naively thinking that grep -v -B2 ...
3
votes
3answers
246 views

Removing blank lines from a file

I want to remove all empty lines from a file. Even if the line contains spaces or tabs it should also be removed.
1
vote
1answer
44 views

List the lines with the highest scores

I have the following problem concerning the sorting of a content of a file. I have a file called points.txt and it contains the following table of data: Jim:4 Tim:8 Lin:6 Tom:1 Esko:2 Jaska:5 Erkki:7 ...
2
votes
2answers
114 views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
1
vote
3answers
78 views

a string representation using regex (modified)

I would like to use regular expression to represent a pattern with the following type of characteristics 1) It includes a continuous sequence of digit values. The length of this sequence it at least ...
1
vote
3answers
99 views

Extracting mail addresses

I have a file that contains lines like this: user=<xxxxxxx@asd.com.tr>, user=<yyyyyyy@asd.com.tr>, user=<zzzzzzz@asd.com.tr>, user=<ccccccc@asd.com.tr>, ...
1
vote
3answers
556 views

Count tabs per line in text file with utils

I have a tab-tabulated file. I would like to check if every line has the same number of tabs. For first step, I'd like to print the number of tabs for each individual line. I've tried grep -o '\t' ...
4
votes
3answers
249 views

how to emulate “replace a b b a” with a sed?

I am trying to emulate a replace command which swaps two characters, with a sed script, but the obvious choice: sed -e 's/a/b/g' -e 's/b/a/g' does no work. Can you point me how to do it correctly? ...
5
votes
5answers
189 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
2
votes
5answers
105 views

Read contents in a line in bash

I have a file with contents in this order which is separated by space 4096 1230 4560 1024.... n How can i read each number and pass it on to another command, eg divide each number by 1024.
3
votes
1answer
114 views

Simple Bash Script

I have a simple problem, but unfortunately I don't know where to even start (I'm just starting out). So, what I want to do is ultimately increase my vocabulary. I got the idea to strip the most ...
8
votes
2answers
271 views

Merge three file in to a single master file by excluding the header in 2nd and 3rd files

I have three file with n number of lines as given below sample1.txt ------------------------------ Date Time Name ------------------------------ 2013/10/12 12:09:09 Krish ...
0
votes
2answers
63 views

Extracting information between tags [duplicate]

I have a text file, the following shows a sample of file contents: 1234 A novel homeodomain-encoding gene is associated with a large CpG island interrupted by the ...
2
votes
6answers
220 views

Remove lines that do not start with a pattern from a given set of patterns

I have a file that contains data like this: report aaaaaaaa - .. -th bbbbbbbbb -to ccccccccc .. --. Question: I want to remove any line that does not start with following strings : report ...
0
votes
0answers
15 views

Extract in beteen tages in a text file [duplicate]

I have a text file that have tags between text such as the following <category="Modifier">Ovarian cancer</category> <category="SpecificDisease">coronary heart ...
2
votes
2answers
73 views

Extract information from a text file

I have a text file, i want to search for tags such as the following: <category="SpecificDisease">Type II human complement C2 deficiency</category> <category="Modifier">Huntington ...
4
votes
3answers
67 views

Fast access to often used document

For several applications I maintain my personal cheatsheets. I use them mostly for vim, latex, vim latex-suite and bash. For example, when I forget a key combination in vim, I browse to the ...
3
votes
1answer
584 views

How to merge two files in corresponding row?

Now,I have two files: aaaa.txt: a=0; b=1; c=2; bbbb.txt: d=3 e=4 f=5 I want to merge aaaa.txt and bbbb.txt to cccc.txt. cccc.txt as follow: a=0;d=3 b=1;e=4 c=2;f=5 So, what can I do ...
3
votes
5answers
215 views

A better paste command

I have the following two files ( I padded the lines with dots so every line in a file is the same width and made file1 all caps to make it more clear). contents of file1: ETIAM...... SED........ ...
1
vote
1answer
94 views

Insert a space before the first comma in a csv file

I have a huge CSV file which contains 100000 rows. However, I need to do some processing and so for that I need to have a single space before the first comma. My current data is something like below. ...
0
votes
1answer
217 views

Awk with multiple delimiters

I am trying to execute awk on something like this : <tag x="whatever" y="http://mysite.com/somewords/word1/word2/word3/word4"> I would like to extract word2 which comes forever after word1, ...
1
vote
3answers
198 views

compare files line by line and create new one bash programming

I have two text files. File 2 has logs over 1,000,000. File 1 has IP addresses line by line. I want to read file 2 lines and search these lines in file 1, I mean: file 1: 34.123.21.32 45.231.43.21 ...
2
votes
2answers
1k views

Insert text in specific lines of a file?

How can insert some text in specific lines of a file? What is most simplest method that I can use? (bash, sed, awk?) What I want to do, might be simple, but I don't know where I should start. It's ...