Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
3answers
25 views

Merge lines between keywords into one-line comma separated values

Between first occurrence of Cat to next occurrence of Cat, it should create a separate line with delimiter as ",". File input as below. Cat AA BB CC Cat AA-1 BB-1 CC-1 Output expected: Cat,AA,BB,...
1
vote
2answers
53 views

Finding the last character of a file and adding a string after that? [duplicate]

Appending an extra line is simple as I can use echo "line" >> file but what if I want to add a string right after the last char in the file without starting a new line? What are some good ways ...
3
votes
5answers
62 views

Non-greedy match with SED regex (emulate perl's .*?)

I want to use sed to replace anything in a string between the first AB and the first occurrence of AC (inclusive) with XXX. For example, I have this string (this string is for a test only): ...
1
vote
1answer
25 views

Replace the values of some keys in a single line text file

I have a large file called res.dat that contains certain data I need modified. All data is on one single line. For example: L1TRAIN="200" NEEDL2="14" L2TRAIN="300" NEEDL3="16" L3TRAIN="400" NEEDL4="...
1
vote
1answer
10 views

Splitting a file into multiple files based on 1st column value [duplicate]

I have seen solutions of splitting a file with respect to pattern matching and line matching but what I want is the following. The scenario is, let's say I have a file file1 - A.B|100|20 A.B|101|...
3
votes
2answers
45 views

how to get content within a specific text from a file

I have a file name demo.txt with content as follows: value -= [ "02|05|06|abc", ] /* Some other content other than value variable */ value -= [ ] value -= [ "0698|06|07|abc", ] I have lots of ...
0
votes
1answer
39 views

Problem with echo command

I want read a file and after if there is one "KO" in the file ill break the program and print "KO". If there are all "OK" i want that at the end of the program it will display me "OK". Here the code: ...
0
votes
2answers
32 views

Extract the value part from key-value pair separated by colon in Linux

I need to fetch arn:aws:sns:us-west-2:596873354795:demo from the below content in a file using sed command. What would be the command? Sample file: { "TopicArn": "arn:aws:sns:us-west-2:...
0
votes
3answers
48 views

How can I delete a line from a file when the line contains a specific string? [duplicate]

I have a file which is comma separated. I have to find specific text and then delete complete line. e.g. 123,aaa,bbb,888,8881 124,bbb,ccc,999,8881 125,ccc,ddd,888,8883 I have to delete the line ...
7
votes
3answers
572 views

Replace all occurences of two spaces after the end of a sentence with just one space

I'm stuck in including regular expressions with a sed command. Q: I want to replace all occurences of two spaces after the end of a sentence with just once space. Here is what I did: sed 's/^ ...
7
votes
6answers
927 views

How can I delete all characters falling under /* … */ including /* & */?

I did tried sed and awk, but its not working as the character involves "/" which is already there in command as delimiter. Please let me know how can I achieve this. Below is a sample Example.We ...
1
vote
3answers
39 views

Appending a character in the nth position of a matching string

I need to find a sed/awk command which will append a character to the nth position of a matching string HAVING LENGTH 10. For example, I need to search for the string starting with pattern 541 and ...
1
vote
1answer
28 views

Problem with reading records from a file

I have the following problem: Script #!/usr/bin/ksh for unitcaseno in `cat /pims/nigel/UNLOAD-DB/xaa` do echo "String = $unitcaseno" unitno=`echo $unitcaseno | cut -d'|' -f1` ...
0
votes
1answer
22 views

Remove last n characters from -print0?

Currently I have: @find . -type f -name "img*_01.png" -print0 | python script.py -f {} is there a way to trim the last few chars like this? @find . -type f -name "img*_01.png" -print0 | python ...
2
votes
1answer
45 views

Cut string from command output

I would like to cut 150.8 from this string temp1: +150.8°F (crit = +197.6°F). Here is my script for logging temperatures with the command sensors: #!/bin/bash now=$(date +"%m_%d_%Y") # ...
4
votes
2answers
207 views

Replace the first occurence of a pattern in a file that may contain a slash

Thanks to http://stackoverflow.com/questions/27787536/how-to-pass-a-variable-containing-slashes-to-sed I know how to pass a variable that contains slashes as a pattern to sed : sed "s~$var~replace~g"...
1
vote
4answers
73 views

Merge two files: two lines, partial line, two lines, partial line, etc [on hold]

Hello I am trying to use awk to merge two text files in a rather peculiar way, taking two lines from file1, a group of word(s) from file2 (but placed on a separate line), alternating ad infinitum.  ...
3
votes
2answers
27 views

Problem with reading text file encoded in Western encoding (ISO-8859-1)

I'm having a problem with encoding of ISO-8859-1 text file (subtitles in Polish language), which looks something like that: Mieszka³ sam,|¿adnej ¿ony, dzieci. It should be : "Mieszkał sam, żadnej ...
3
votes
2answers
80 views

Compare two files and print matching values without ignoring duplicates

I am trying to comparing two files and obtain the matching values. I tried this command: grep -Fwf file_1.txt file_2.txt > matched_output.txt However, this script extracts only unique values. ...
3
votes
5answers
111 views

How to repeat a block of text?

How do I use awk to repeat a block of text 100 times? ie string1 string2 string3 Would become: string1 string2 string3 string1 string2 string3 string1 string2 string3...
4
votes
4answers
590 views

Parse a txt file based on a specific delimeter and then tranform it into a CSV file

I have the following file named OpenSimStats.txt: TestreportsRootAgentCount=0agent(s) TestreportsChildAgentCount=0childagent(s) TestreportsGCReportedMemory=10MB(Global) TestreportsTotalObjectsCount=...
-1
votes
1answer
51 views

How to combine two files by shifting the value of the row file to its corresponding value in the column file?

I have a two files, the first has one row while the second with one column I would like to combine the two files by shifting each value in the row file to the same value in the column file, then ...
2
votes
0answers
54 views

Find the top 5 closest rather than just the closest? [migrated]

How would you find 5 numbers in a column of numbers that are the closest to a $VariableNumber? For example, if the $VariableNumber = 30 then: Example input file: 50 100 70 40 20 10 65 41 92 ...
1
vote
2answers
48 views

how to insert file contents on match using sed - first occurrence only [duplicate]

file contents: # cat file.txt ----- MATCH ----- MATCH ----- MATCH ----- # cat text.txt add this text file before first match sed commands: # sed '0,/MATCH/r text.txt' file.txt ----- add this ...
4
votes
1answer
59 views

Splitting file content into other files based on parsed parameter value in file [duplicate]

A regular file contains data organized in columns, separated by space. Based on the value of the second field, I want to split that initial data file into several other files. The data file has 54,...
4
votes
3answers
149 views

Transforming short repeated words in columns into numbers

I want to transform the short repeated words in columns into numbers. In the following example I want to change the words (with ONLY 2 LETTERS) in column 3 for numbers, so that AA is changed to 2, AB ...
2
votes
1answer
35 views

Appending a string to a file from remote ssh

I want to ssh to a node (which is passwordless) and append $spool_as_final_name true to a file /var/spool/torque/mom_priv/config. Note that is mandatory to put $ in front of the string. So, I wrote ...
2
votes
3answers
173 views

How to remove all lines that have one value in the line?

I have a file as the following input file 0.0 4 6 5 1 2 9 4 5 1 ..... 0.25 0.5 3 1 1.0 3 7 8 2 1.5 3 3 3 4 6 4 5 2.0 2.5 6 7 6 9 I want to delete all lines ...
-2
votes
3answers
27 views

How to select specific lines of multiple data files

I have many files like these: file1: 408.60 0.0847 370.03 0.1571 307.49 0.1492 297.12 0.0708 274.36 0.1989 251.27 0.1640 file2: 391.06 0.0589 368.55 0.2293 313.87 0.0369 299.94 ...
1
vote
3answers
64 views

How to grep/awk/sed for text in a log and display the chunk that has the text?

I'm looking for something pretty similar to this. The logs look like this: [09:44:22] [main] ERROR [url/location] - A ONE LINE ERROR [09:44:22] [main] ERROR [url/location] - Another ERROR [09:44:22]...
3
votes
2answers
277 views

How to parse an xml file in a shell script

I would like to know how can I split my data from the following format: <datas> <data> <name>Name1</name> </data> <data> <name>Name2</name> <...
1
vote
2answers
50 views

Replace particular character but not if it is inside ()

I'm looking for a one-line command to make a file more readable. I want to replace all ; characters with newline unless it is inside a set of (). This is on a firewall, so I can only use bash; no ...
-2
votes
4answers
69 views

Transforming letters in columns into numbers

I want to transform the letters in two columns into numbers. In the following example I want to change the letters in column 2 and 3 for numbers, so that A is changed to 1, B into 2, C into 3 and G ...
6
votes
1answer
220 views

Each grep result to different file

I have something like grep "keyword" -B 3 log. It show a lot of results like: some trash [GET] /orders/42 one more trash keyword and usefull info [GET] /orders/43 some trash one more trash keyword ...
2
votes
1answer
47 views

how to append words of two files word by word [duplicate]

sorry for my newbie question. I have two files with the same number of words and column fields in each line. I want to append these two files word by word. For example: File 1: A1 B1 C1 D1 E1 B1 C1 ...
2
votes
2answers
256 views

How to count values that match a certain condition?

I have file with number of columns for example input file 12 15 27 5 6 48 19 16 25 17 23 37 etc I want to count all the numbers (X) in column 2 for example that fulfill the ...
1
vote
2answers
29 views

Error in format when appending column with sed

I have a table and I want to add a column containing, for each line, the name of the file itself. I will post an example below: This is the input: SPATA17 1 217947738 LYPLAL1 1 219383905 FAM47E ...
0
votes
2answers
46 views

Command “paste - - - - ”, how to specify a big column number

In command paste - - - -, numbers of - is equal the future column number. In my case, I have 55,000 future column, for me wont need to put 55,000 - what I will can use? Example: title1:A1 title2:A2 ...
0
votes
2answers
46 views

copy every line from a text file that contains a number greater than 5000

Hi I've tried many solutions to similar questions and none have seemed to work for me. I have a text file where each line has an undefined length of numbers after the string " length_ ". How can I ...
1
vote
3answers
50 views

I need the counts of lines between two matching patterns

I have my input like below - [11/Jul/2016:13:40:43 +0000] "POST --" 200 - 7 [11/Jul/2016:13:40:43 +0000] "POST --" 200 - 7 [11/Jul/2016:13:40:47 +0000] "POST --" 200 - 7 [11/Jul/2016:13:40:47 +0000] "...
5
votes
4answers
240 views

sed/awk remove newline on two pattern matches

I have a .pem file that after some formatting it shows like this: -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAoK3D4mMIRnzPaYqHidgpBnDDzLlcYYd0GoB0pQGyGSHDW7KO3K+VeJP90GhE ZTEWJLp2N5DR/KT+5Vg7cgdx/...
0
votes
3answers
45 views

Print/export csv file into array

I have the following data in a very large csv: "sortorder","name","description" "1","B.1","Boiler room" "2","1.1","First office" "3","1.2","Second office" etc... Which I would like to end up in ...
1
vote
3answers
64 views

Output lines between two parameter dates INCLUDING the first parameter date but not the second

This questions has been asked before, except they were looking for text between two dates including or excluding those dates. I’ve changed one of the answers so that it selects and includes from first ...
7
votes
3answers
281 views

Side-by-side comparison of more than two files containing numerical values

I have three files containing a sorted sequence of numbers, one per line : file1 1 2 3 file2 1 3 4 file3 1 5 I want to "align" these three files side-by-side like the following : file1 ...
7
votes
7answers
343 views

grep the exact block of lines (content of file1) from file2

I have two files, file1 and file2. The sample content of file1 is: A B C D E F G H and the content of file2 is like: A B few other lines E F few more other lines A B C D E F G H few more other ...
3
votes
1answer
59 views

Parse (grepable) nmap output to print a list of IP\t[all open ports] with text utils like awk

I would like to find a way to print for each IP address found to have at least one open port, to print that IP address, followed by a list of open ports separated by commas. The ports and IP address ...
1
vote
1answer
35 views

read patterns from file1 and search them iteratively in file2

I'd like to read patterns from file1 and search them iteratively in file2, printing the matched line plus 3 rows, and print them to file3. Now I am doing: cat file1 | while read line; do awk '/$...
1
vote
2answers
61 views

Text Manipulation: Removing values x <= $input from sorted list & generalizing

My input text file contains a 1 line header, then a sorted list of lines containing: x y with x,y 64-bit integers separated by a space. The Input file is many million lines long. Now I want to ...
14
votes
3answers
415 views

Replace environment variables in a file with their actual values?

Is there an easy way to substitute/evaluate environment variables in a file? Like let's say I have a file config.xml that contains: <property> <name>instanceId</name> <...
1
vote
3answers
41 views

Selection of line from pattern, replacement with new line and incrementing number for each change

Hi I'm trying to work out how to select a line in a text file from a pattern/word string, and then replace that line with new text with an incrementing number for each line changed. I was ...