Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
13 views

iconv terminated by signal 7

I was running a little bash script on some text files: find . -name "*.dat" -exec iconv --from-code='UTF-8' --to-code='ASCII//TRANSLIT' {} --output={} \; My machine is a Ubuntu 14.04 LTS. After a ...
4
votes
5answers
149 views

What command(s) will feed a tab-delimited text file and cut each line to 80 characters?

I've got multiple-line text files of (sometimes) tab-delimited data. I'd like to output the file so I can glance over it - so I'd like to only see the first 80 characters of each line (I designed the ...
4
votes
1answer
77 views

How is ksh93 so fast?

So, in general, I tend to look to sed for text processing - especially for large files - and usually avoid doing those sorts of things in the shell itself. I think, though, that may change. I was ...
0
votes
4answers
64 views

Time calculation [on hold]

I have a text file containing flight times, hours and minutes one per line: eg 8 minute(s), 28 second(s), 870 milliseconds 19 minute(s), 24 second(s), 398 milliseconds 18 second(s), 876 milliseconds ...
4
votes
2answers
120 views

Replace or append a line with string within a file

I am looking for advice on how to search a part of a string within a file and replace the complete line OR append that string to that file if not found. I "played" with sed for a while now, but ...
4
votes
4answers
88 views

Remove a character between 2 known strings

I have a data set like below: \"XXX \ START sapiodj \\" aj \d 2387 END hddo\" START bbcc \\" END ss My Requirement: I want to remove all occurrences of backslash\ and double quotes " between ...
3
votes
4answers
133 views

tab separation of the file

I have a file with seq likes this ATCGTTTCTCAGCCTTTTGGCAAGACCAAGTGTAGTATCTGTTCTTATCAGATATTGGTATCTGCAGGCACTCCTGGAACCAAT ...
5
votes
3answers
100 views

Replace uppercase to lowercase except first letter of each word in specific line

I want to replace names that I received in uppercase letters only. The names are mixed with other information which should stay as it is. AUTH: at the beginning of the line identifies it as a line ...
1
vote
1answer
30 views

using bash script to sort text in a file

I would like to sort the output of multipath -ll into a csv type of file using bash script. For example # multipath -ll would give the following output 360060e80056fc30000006fc30000513c dm-41 ...
6
votes
3answers
191 views

sed with multiple expression for in-place arguement

I am trying to replace multiple words in file by using sed -i #expression1 #expression2 file Something 123 item1 Something 456 item2 Something 768 item3 Something 353 item4 Output (Desired) ...
5
votes
2answers
47 views

Merging/Combining 2 text files according to numeric field

I have 2 text files file_1: 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5 file_2: ...
0
votes
2answers
41 views

Extract multiple instances of text between two strings [closed]

I'd like to extract text between two strings in a text file that contains Spotify chart results Text file extract: ...
3
votes
1answer
62 views

sed repeating the last line in the stream

I have code in a script similar to this... smbconffile="/etc/samba/smb.conf" sed -i 's/.*[\[CMI\]]/\[CMI\$\]/' $smbconffile && echo "Success" || "Failed" sed -i 's/.*[\[LOCAL\]]/\[LOCAL\$\]/' ...
-2
votes
1answer
43 views

Output a colon delimited file to another file [closed]

I have the following example file: root:server1:2014-12-12:x:Open:adm bin daemon lp mail other root sys users sectmp:server1:2014-12-12:x:Open:users smbnull:server1:2014-12-12:x:Locked:ids ...
0
votes
2answers
36 views

Take the value of another column if the query column is between two column values

So i have a file with column that has information on salary (eg. 2674 dollars) - indi salary sam 2674 john 6375 max 9547 and another file with the scale of the salary ...
2
votes
1answer
36 views

Cut and Paste Commands

So I have: $ cat fruits 2 bananas 3 cherries 4 figs 5 dates 6 elderberries 7 apples 8 grapes and 1 $ cat prices 2 2.18 3 4.11 4 1.69 5 4.52 6 1.73 7 1.01 8 1.09 Every line from 'fruits' ...
6
votes
4answers
162 views

Need to move the last line of the file to second line of the same file

I have a file 'test' in linux with below data. aaaaaa bbbbbb cccccc dddddd eeeeee I need to cut the last line of this file and put it in the second position. It must look like below. aaaaaa eeeeee ...
1
vote
3answers
53 views

Calculate and round numbers up in bash

I'm struggling with a slight variation on: How to calculate by taking first field and How to round floating point numbers in shell? I have a file that looks like (space between columns): 1533 ...
0
votes
2answers
52 views

Command to see if a word is in a list of files and output how many files it is in

There is a similar question to mine already but mine has a slight difference. I want to see how many files contain each word. When a file has a word then it increments the counter and then continues ...
1
vote
3answers
167 views

Replace a string and replace it with a part of itself

I am cleaning up my data that is stored in text files. Each line starts with a category label followed by the actual data that I want to clean up. There are many text files in different subfolders, so ...
0
votes
2answers
31 views

output a variable value using echo command [duplicate]

I want to output number of lines in text file.So I am using echo command but I am getting filename along with number of lines when I run the following code: echo "$page" > t total="$(wc -l t)" ...
0
votes
1answer
55 views

Removing contents from a file

I have a file like this >gene1*ENSG24 CTTGGGGGGCTGGGGGCCAGGTGAAAGGGAAATGGAGGGCAGCACCCGCG AGCCCTCATTGCCTATAGTGGTTTCCATGGCGATCATGTAAGAGTCAATG TCGTCATTGGCAAAGTCGTCCGGGTGGGGTGTGCTGTAGGCAGAATCGGA ...
4
votes
3answers
91 views

Insert text from file inline after matching pattern in another file

I'm trying to take the contents of a file and insert it after a matching pattern in another file using sed. My question is very similar to this question, but I wish to insert the contents of a file ...
3
votes
2answers
57 views

How do I efficiently select specific line numbers from a list of records?

I want to take a list of numbers representing line numbers of a source file I want to filter and filter those lines from the source file. How I can build a unix pipeline to extract these lines from ...
1
vote
5answers
100 views

How to get the unique count of a particular part of a string

I have a set of data in a file. psf7433-nlhrms unit7433-nobody unit7333-opera bpx7333-operations app7333-osm unit7330-partners psf7331-pdesmond unit7333-projm mnp7330-redirect unit7333-retailbanking ...
1
vote
3answers
126 views

grep only matching word

I have a temp file, and I want to grep the only word that matches the pattern instead of whole word. i tried grep -o <pattern> file but its not giving me desired output Input xi29 vddf vss ...
-1
votes
2answers
31 views

Extract 2 fields from 2 different lines

I have the below lines in my file # Employee Id : ABC123, # Employee Name: Vivek # Employee Type: Temp Join Date: 31-jan-2014 Buz Code: 2098 # Employee Comments: # ...
6
votes
4answers
307 views

How to split text between separator into multiple files?

I have a text file contaning the following: "random textA" "random random textB" The separator is " How can I split the containt into multiple file as follow using a bash command? File 1 : ...
1
vote
7answers
104 views

repeat a line, splitting one field

I have a tab delimited file in which in every line I have this: K00001;K00004;K00008 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 36 0 0 52 0 0 0 6 0 I would ...
1
vote
2answers
47 views

Extracting list of patterns which are output of another command

I can extract list of patterns using following command, fgrep -A 1 -f patternlist.txt filename.fasta but, is there a way I can extract without creating another file (patternlist.txt in this case) ...
3
votes
1answer
23 views

Replacing " with pretty quotes on a lot of files

I have a lot of UTF-8 encoded files with words quoted like this "word" and I want to replace the quotes with “word” (the first quotes are upside down and the second is normal) Because the ...
2
votes
2answers
107 views

Remove escape characters using sed

I have input coming from pipe which contains special escape characters. For illustration, the stream looks like this: printf "foo.bar\033[00m" > file cat file How can I remove the trailing ...
0
votes
2answers
57 views

How to get portion of lines from all .txt files in a directory?

I have 5,000 text files of journal article citations. I am trying to extract only the abstract portion. Meaning that I want to keep the same text document and delete all the other text except for ...
4
votes
3answers
213 views

“parentheses not balanced” even though I escaped it?

Im trying to run this simple sed command sed '/VALUES\ \([0-9]/d!' It still gives me an error saying "RE error: parentheses not balanced" (yes the quotes are there). What can I do? I mean I could ...
0
votes
3answers
61 views

Strip the file base name, keep the directory part

I have a string which looks something like this dir/subdir/othersubdir/file.txt But it can also look like dir/file.txt or dir/subdir/file.txt Now, I am making a script where $P is the full ...
4
votes
3answers
162 views

Copy a specific percentage of each file in a directory to a new file

For example, we have N files (file1, file2, file3 ...) We need first 20% of them, the result directory should be like (file1_20, file2_20, file3_20 ...). I was thinking use wc to get the lines of ...
2
votes
1answer
77 views

Specify lines in awk?

Im going to use this awk script to delete duplicate lines from my text file: awk '!x[$0]++' However, I want to specify which lines that they are to be run on. There could be 10000 duplicates, but ...
0
votes
3answers
107 views

Shell script to add or replace block of text?

I'm looking for a shell script (Bash) that adds or replaces copyright text at the beginning of the file (for .c and .h files). It should search and add or replace the copyright text block for all the ...
2
votes
1answer
39 views

sed: append line and number output

In Linux, say one wants to add the following line before line number five (thus line nuber 5 becomes line number 6) and wants to have all the lines in the output numbered throught the sed 'tool' = ...
2
votes
7answers
215 views

How to print lines with blank 5th field in CSV

I have to check if the fifth field is empty in a CSV file. This is my file: 1,abc,543,87,DATA,fsg; 1,abc,543,87,,fsg; 1,abc,543,87,DATA,fsg; 1,abc,543,88,,fsg; 1,abc,543,,DATA,fsg; As you can ...
8
votes
5answers
470 views

Convert file contents to lower case

I have temp file with some lower-case and upper-case contents. Input Contents of my temp file: hi Jigar GANDHI jiga I want to convert all upper to lower. Command I tried the following command: ...
3
votes
1answer
46 views

Use paste with row-major input

I can create a file with multiple columns from a single-column input via paste like this: some_command | paste - - This works when some_command produces the data in column-major format. In other ...
1
vote
2answers
23 views

Find string then delete between two tags and repeat

This is probably a bit of a newb question, but here it goes. How can I find a string in a file, then find the first instance of a tag that precedes it and then delete everything between that tag and ...
1
vote
3answers
41 views

Splitting a string up to a specific location

I have a question and wondering how to address it with sed? How does one cut a variable up to a specific location, in my case _g? Example variable="This_is_good_g0r0s0_continues" I need to cut ...
3
votes
2answers
146 views

Sed: modify every non-first word repetition for every word in text

Can you help me to do something like that using sed? qq ab xyz ab qq aa ab Becomes: qq ab xyz +ab+ +qq+ aa +ab+
0
votes
2answers
57 views

problem with grep -f list of pattern

After searching, reading and trying possible solutions for much about grep lines with the list of patterns, hence I am writing. It is I know a very basic and have been attended in many forums. But I ...
4
votes
5answers
141 views

Use sed to print from the very first line until the line containing the last occurrence of a pattern?

I know the following commands will print up to the first occurrence of a certain pattern but will not include the later occurrences: sed -n '1,/<pattern>/p' <file> sed ...
2
votes
1answer
95 views

Replace certain text occurred within curly brackets

I have sample file as follow : WEA { Direction = Input Tag = WriteEnable PortId = "A" } MEA { Direction = Input Tag = MemoryEnable PortId = "A" } CLKA { ...
2
votes
2answers
44 views

Command to work on all files in a folder and output result to single file

I have a list of files, 10 to be exact, in a recipe folder. I must determine the number of occurences of each word in all the files and then order them according to these frequencies. I can get my ...
5
votes
1answer
133 views

How to make a list generated by md5deep in alphabetical order of relative paths?

If I want make a md5sum list recursively, then I would use md5deep, but it starts to pop up some problems such as it won't generate the md5sum file in alphabetical order. For example, $ cd ...