Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
1answer
36 views

seeking hints for correct escaping of sed commands WITHIN yaml

I am having a hard time finding the issue why following lines of "code" break my cloud-init config. Basically I am calling a son object as part of the DigitalOcean API which has a YAML format - I got ...
1
vote
2answers
74 views

How to obtain a plain (raw) binary dump of any file in Unix/Linux?

I have read many similar question on the Stack Exchange community but none of them address a 'clean' binary dump. I have used: xxd and sed combinations to try to obtain a clean binary dump from any ...
2
votes
0answers
139 views

Bash regular expression comparison fail for hex byte \x01

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly. Using GNU bash 4.1.5(1). Is this a bug, or is ...
0
votes
3answers
334 views

Remove special characters in a text file

I have a text file which contains in each line a sql query. For each line, I need to remove some special characters. Expl: if I have: select * from Users; insert into Users values ...
2
votes
4answers
137 views

Why isn't this awk command doing a full outer join?

Objective: Merge the contents of two files using common key present in the files file1.txt ========= key1 11 key2 12 key3 13 file2.txt ========= key2 22 key3 23 key4 24 key5 ...
2
votes
3answers
213 views

How to 'drop' characters from in front of a string?

I have a string that I would like to manipulate. The string is H08W2345678 how would I be able to manipulate it so the output is just W2345678? Similarly if the I wanted to drop the last 4 ...
0
votes
2answers
44 views

How to scrape multiple IP's and add in a script as a variable?

I am trying to add in my script an awk statement or a: if; else; then if eth0:4 is a match then put in eth0:4 IP How would you get the IP in your script as a variable if you have multiple IP's ...
0
votes
1answer
57 views

Vim: Count all occurrences (including multiple on the same line) of char or word

I try to count all occurrences of a char or word in a file with Vim. This means multiple occurrences of the same word in the same line should be taken into account as well. This has been discussed in ...
0
votes
2answers
102 views

How can I filter ping results by time?

How do I filter out any pings that are under a certain ms, ie, only echo ping responses above 500ms to the text file? IP=$1 SECONDS_Between_Pings=$2 ping -i $2 $1 | while read pong; do echo $(date) ...
0
votes
1answer
29 views

text processing

I have two files. file A : Chr1 Cufflinks exon 2903 3268 . + . gene_id "XLOC_000001"; transcript_id "TCONS_00000002"; exon_number "1"; oId "CUFF.1.2"; tss_id "TSS1"; ...
4
votes
3answers
137 views

Remove duplicate lines while keeping the order of the lines

[root@server]# awk '!seen[$0]++' out.txt > cleaned awk: (FILENAME=out.txt FNR=8547098) fatal error: internal error Aborted [root@server]# The ""server"" has: 8 GByte RAM + 16 GByte SWAP, x>300 ...
3
votes
4answers
246 views

Using sed for brackets

I would like to find out, if, in my file, there is a right number of brackets. I can make my file look like (((()))()(())) ((()))() How can I use sed to compute those brackets and print yes or no ...
3
votes
4answers
202 views

How do I extract the first integer from text string in a column of a tab-delimited file?

I work in Medical Genetics and often have delimited text files where in one column (ex. column 5) there is a text string with a "mutation" in our jargon: c.2458C>T or c.45_46delAA or ...
0
votes
1answer
40 views

How to find & replace pattern WITHIN specific pattern using sed

IT seems that 'sed' and I will never be friends... How can I find and replace a pattern within a range (within another pattern)? In words: Find: AllowOverride None within the following range and ...
0
votes
2answers
36 views

Vim: Substitute based on another match in the line

I have rather large files in which I need to substitute NX for N1 but only if the line already contains NX as a pattern. So far I do it manually by first searching for occurrences of NX with: /NX ...
2
votes
1answer
101 views

Using find and sed to copy 20th line of many files into one file

I have a folder called "PKA1" Within this folder I have several folders that look as: 1eV, 2eV .... 30eV Within each of these I have subfolders I have a file called PKA.dump I want to copy the ...
4
votes
2answers
193 views

Increment a variable using a Bash script

I have a file that contains the following text. // // test_file.h // test project // // Created by Test User // #ifndef test_file_constants_h #define test_file_constants_h // Generic includes ...
5
votes
5answers
401 views

How to replace a word with new line

I have a text file with following data and each row ends with |END|. T|somthing|something|END|T|something2|something2|END| I am tryig to replace |END| with \n new line with sed. sed ...
0
votes
1answer
56 views

Error while passing variables for keyword match using sed

I am a beginner in scripting. I am writing a bash script for text processing where this command is giving me error: sed -e "/\<$7\>/ /\<$5\>/$6/" < "$2/$1" > "file.tmp" This is ...
5
votes
7answers
507 views

Turn list into single line with delimiter

I have to take a list (loads) of IP addresses in this format: 134.27.128.0 111.245.48.0 109.21.244.0 and turn them into this format with a pipe in-between (IPs made up) 134.27.128.0 | ...
2
votes
6answers
185 views

Join lines of text with repeated beginning

I have a long text file (a tab-file for stardict-editor) which consists of lines in the following format: word1 some text word1 some other text word2 more text word3 even more and would like to ...
3
votes
1answer
91 views

Extract first word after pattern that includes `[…]`

I have an issue when I try to extract the first word after grep matches my pattern . Let me show you an example : cat test.txt [profile dev] xxx xxx [profile prod] xxx xxx now I want to extract ...
0
votes
3answers
53 views

How to find the word after a given pattern using cut?

I have a file that looks like this: -config1 -config2 param2 -config3 param3a param3b -config4 param4 and I can use the following to extract param2: echo $(<file) | cut -d' ' -f3 However I ...
1
vote
1answer
41 views

Trouble with Sed Syntax

I'm modifying raw PCL/PS files (mixed) and for some reason I can't get my Sed syntax right for the true beginning that I want it to grab. Here's a sample output from the strings command: *c50B *c0P ...
1
vote
2answers
51 views

SED replace after ignoring

There are many lines in a text file and I would like to change the value true to false using a one line script. bash-3.2$ grep online test.txt # online = false --> ONE SHOT import (delete all ...
0
votes
2answers
38 views

Select lines and format

Is there a way to format the following output so that only lines 1 and the respective lines that state the "rule" name print out? Depending on the configuation, the "rule" line could vary. It could be ...
0
votes
1answer
72 views

Modify part of line if keyword is found using sed? [duplicate]

I want to modify a line if a specific keyword is found in a line. CS_CODE, SM_NUM, PORT_NUM, DEV_PORT_NUM, DIRNUM, MAIN_DIRNUM, BILL_NO, ADR_NAME In above line, if 'PORT_NUM' is present then ...
2
votes
1answer
55 views

How to select lines and reformat command output?

Is there a way to format the following output so that only lines 1 and 4 print out? It would be best if the first line and fourth line could come out on the same line like this: sw pool test(no ...
1
vote
5answers
71 views

keep out line break

I have some files with this structure: 2015-03-25 17:08:17 sysUpTimeInstance 93474;^M 1.ValueforState=2500 I want to replace the line break, and leave the third line with the second line, I mean, ...
-2
votes
1answer
87 views

how to print new word after two lines using awk

I need to insert a blank line after two lines in a file that i did with cat rat hat pot NOT lot big big big sit sit sit awk '{print};(NR%2==0){print ""}' cat rat hat pot NOT ...
1
vote
1answer
44 views

How to match parameter and replace the value after “=” separator

the perl one liner command to replace Value in file is simple: perl -i 's/oldval/newval/g' file.txt but what should be the perl one liner syntax in case we want to replace the value on specific ...
1
vote
2answers
93 views

sort only the first column and uniq

I have a list as such below: 1,cat 1,dog 2,apple 3,human I'd like an output like this: 1,cat,dog 2,apple 3,human So value 1 from column 1 contains the value of cat and dog from ...
2
votes
3answers
133 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
2
votes
3answers
363 views

How can I extract the text between two strings in a log file?

I think I have a very similar question to this one but I see it was closed due to being unclear so I'll create a new question. I've got a log file that contains one-line entries with multiple ...
1
vote
2answers
106 views

Remove paragraph from file

I have a file (file.php) like this: ... Match user foo ChrootDirectory /NAS/foo.info/ ForceCommand internal-sftp AllowTcpForwarding no GatewayPorts no ...
1
vote
1answer
69 views

How to truncate text lines to N characters maximum? [duplicate]

Given a text file, or the output of a command, how can I truncate it so that every line longer than N characters (usually N=80 in a terminal) gets shorten to N characters maximum?
0
votes
1answer
61 views

Search a file for a string in multiple directories

Can someone please help me with a command to search a file for a string in multiple directories? I am searching for the VIP in httpd.conf in multiple httpd instances. I am using: find ./ -name ...
0
votes
4answers
80 views

Copy same specific parts of a file to another file

How can I copy same specific parts of a file to another file? In the first file, I have: user01: Id name nationality user02: Id name nationality . . . I want to copy each piece of information to a ...
4
votes
1answer
25 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
0
votes
0answers
47 views

Compare specific word counts without variables or expr

I am tasked to write bash command to count number of specific words (given in file) in 2 files and write an error message if number of these words in first file is more than 3 times biggen than in the ...
2
votes
3answers
107 views

How to remove a particular character after a specific word

How do I remove the , character after a specific word? For example, I want to delete the , character after the word IBM IBM, Inc.:APP-VM-Core:4.4.2.0:04, the expected results should be IBM ...
1
vote
1answer
59 views

Search for text inside java command output

I have a folder which have multiple Jar files. * dummy.jar * myjar-xx.jar * myjar-yy.jar Using find, I need to find first file starting with "myjar" Now, I want to grep the contents of ...
1
vote
2answers
70 views

filter blast output for e value

I ran a BlastX against the SwissProt database in with an -evalue threshold of 1e-3. The output looks like: >TCONS_00006082 sp|P43298|TMK1_ARATH 57.89 57 23 2 459 292 751 ...
0
votes
1answer
40 views

Using regular expressions to search a word list. Finding 2 letter words instead of 3. Why?

I am studying regular expressions and I am stumped. Here is my process so far.... I downloaded this file. wget http://www.mieliestronk.com/corncob_lowercase.txt mv corncob_lowercase.txt words.txt ...
1
vote
1answer
175 views

find common lines between multiple files

i have 4 files which are like file A >TCONS_00000867 >TCONS_00001442 >TCONS_00001447 >TCONS_00001528 >TCONS_00001529 ...
4
votes
4answers
405 views

How to fill a file with a stream from /dev/urandom with a specified number of lines ?

I am trying to fill a file with a sequence of random 0 and 1s with a user-defined number of lines and number of characters per line. the first step is to get a random stream of 0 and 1s: cat ...
3
votes
6answers
209 views

Remove everything between second and second last occurance of match

I have string which I would like to format. I would like to remove everything between second ; and second last ;. Input cellular ...
0
votes
1answer
83 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
3
votes
2answers
328 views

Nested 'awk' in a 'while' loop, parse two files line by line and compare column values

I need some help with a combination of awk & while loop. I have two simple files with columns (normal ones are very large), one representing simple intervals for an ID=10(of coding ...
1
vote
4answers
198 views

SED command to format text file

I need to create a SED command to modify an input file with some conditions explained below. I have an input file something like below. List item Rad# ; ID_KEY ; UNIT_ID 1 ; 30000000004 ; 8417920 ...