Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
37 views

Remove spaces in file under certain criteria

I want to clean up some files, and make make the way in which they are written more uniform. So, my input might look something like this: $a$h$l )r ^9 ^5 l \ urd The thing is, some spaces are ...
2
votes
2answers
54 views

Remove characters if they do not follow specified patterns

I want to clean up some files, and make make the way in which they are written more uniform. So, my input looks something like this: $a$h$l )r ^9 ^5 l \ urd The thing is, some spaces are ...
-2
votes
2answers
78 views

How do you remove newline character in the middle of file?

File structure: Col1|Col2|Col3|Col4 123|456|abcdefg Tel.: 689346589934, xyz lckhasfkl|abc I want to remove the newline character from 'col3' only.Please suggest
6
votes
4answers
218 views

CSV columnar reformat with SED (or anything other coreutil)

Given the following made-up segment from an output file from hashdeep: 7241,11111111111111111111111111111111,\01-data\file1 1237241,22222222222222222222222222222222,\01-data\file2 ...
0
votes
0answers
33 views

Fetch data from a CSV file based on multiple conditions

I'm trying a write a generic shell script which should extract data from a CSV file based on multiple conditions (generic) and write it to another CSV file. Sample CSV file: ...
2
votes
2answers
57 views

Extracting subset from fasta file

I have a fasta file which looks like this: >chr1 ACGGTGTAGTCG >chr2 ACGTGTATAGCT >chrUn ACGTGGATATTT >chr21 ACGTTGATGAAA >chrX GTACGGGGGTGG >chrUn5 TGATAGCTGTTG I just want to ...
3
votes
5answers
58 views

scanning a sequence and outputting a sub sequence

I am looking for a one liner for this if it is possible. I have a sequence like this CCGGTCTCTTCCGGTTCTGTCTTTTCGCTGG I want to get the output where I scan the sequence base by base and then take 20 ...
1
vote
3answers
38 views

Transpose paramter values from a file and Generate parameter files from it

There's a file containing a table with the parameter names as the first line, values in the following lines: How to perform the following using awk, sed or any shell scripting utility: generate a ...
1
vote
1answer
48 views

substitute from beginning of line with 1 of multiple patterns to the end of another line with 2nd pattern

I have a file like this: data data data $globaltext blah gibberjabber somemorestuff etc ); data data data $otherjunk yada gibberish etc etc more etc ); My desired result is this file: data data ...
4
votes
4answers
196 views

Get lines matching a pattern in one file and put them into a second file matching the same pattern

Let's say I have 2 files each containing lines which start with a 'b' character and I only want to merge these lines in the same order they appear in the first file. First File (1.txt) b 12 32 b 23 ...
1
vote
1answer
27 views

Dynamically computed string replacement with standard tools

Is there a UNIX mechanism for replacing matched strings dynamically--with a function of the matched string? E.g., say I want to replace URL matches with their URL-encoded counterparts or a convert ...
3
votes
3answers
44 views

Symmetric Difference Pipe?

I have a command that produces a lists of strings followed by newlines, a, and a file containing a list of strings followed by newlines, b.txt. I need a command that calculates the symmetric ...
1
vote
3answers
139 views

Number of occurrences in a text file where first character after 12th pipe is equal to 1?

I am fairly new to regular expressions and am looking for a sed/awk/grep/wc command to find the following in a pipe delimited text file the number of occurrences of the digit 1 after the 12th pipe. ...
1
vote
2answers
139 views

File manipulation; Insert a “;” every 12th, 44th and 60th character in every line of a text file

I have to make a csv-file from a textfile which contains data without any delimiters. Looks like: 2015021DUMMY001EAN1377689456777678945GERDE0002345 2015031DUMMY002EAN1345647876867867867TURTR0154565 ...
1
vote
3answers
47 views

remove files based upon diff of two text files

I have created two .txt files that look like: 5fba026dcf271c6fa174b9fc2fc1fefadf1370f9 ./.changeFiles/change/addImage1.change bb352fba5807f4d03ea11abb00e411b00ce38673 ...
3
votes
2answers
80 views

Merge and Append columns for more than 2 files

The files are tab separated. I have "N" number of files whose contents are as shown in the screenshot above. I want to merge them and append the 5th column. The first 4 columns are the same. I ...
1
vote
2answers
35 views

Return line conditonally on next line

I am trying to return a list of IP Addresses for computers that have a windows OS (for my security course). The output is in this format Nmap scan report for 192.168.xx.xxx Host is up (0.066s ...
1
vote
1answer
47 views

Simulate sed read command with awk

I want to stimulate use of sed: sed '3r awk.scr' awk.script with awk.scr: a b c d e f and awk.script as: hello there is hello i'am there is hello sdfdf dfdfdf aads 23213 3 434 Using awk ...
2
votes
2answers
59 views

How to duplicate a file a number of times while embedding an index in each file

I have a file named, for example, ascdrgi.txt, with the following contents: tiger lion cat I want to duplicate this file a (variable) number of times by changing the last character of the filename ...
3
votes
2answers
180 views

merge duplicate rows in columns

Given a file like this: x y y z x x x y z z y x x x z y y z z y x x x x x x x x I would like the output to be: x y+ z x x+ y z+ y x+ z y y z+ y x+ x+ Is that possible to do with awk or perl in a ...
1
vote
3answers
212 views

String cutting in Unix

How could I transform this text? Input file format: MOD MAXT:TID=TID190,MT=VCU,MAXT=100; Desired output format: MOD MAXT:TID,MT,MAXT MOD MAXT:TID190,VCU,100;
2
votes
2answers
58 views

How do I list all lines after two subsequent patterns are matched up to when they don't

I have files that are used to install files with specific owner:group and permissions. I'll ignore permissions for clarity and the details of installing files. Once an owner & group are set to a ...
-1
votes
1answer
42 views

row to column conversion in linux

I have a script which gives me output like this 10.230.146.10: Terminal ID Gol 10.230.146.10: Far End ID pol 10.230.146.10: Number of E1s 4E1s 10.230.146.10: Packet ...
1
vote
3answers
157 views

Read specified lines in file - BASH

I would like to read specified lines in file iptables. I want to read lines started with : and ends with space. File iptables # Generated by iptables 04:13:50 2015 *filter :INPUT ACCEPT [0:0] ...
1
vote
4answers
39 views

Cost efficiently pair each line with lines of another file

I have a very huge file (~10Gb data) which contains data in below format - '1','1' '2','2' '3','3' '4','4' '5','5' '6','6' '7','7' '8','8' '9','9' '10','10' and ...
2
votes
1answer
44 views

Extract URL from non-formatted text

I've found only examples on extracting substrings from formatted text like HTML files, but in my case I need to output a list of URLs for example: ... https://twitter.com/user1/status/xyza ...
4
votes
4answers
111 views

Extracting strings from between special characters

I have a large text file created from combining lots of html files. cat *.html > all_html_files.txt Inside the text file are specfic strings that I want to extract to another text file. For ...
4
votes
1answer
115 views

In POSIX sed, does the period (dot) match a newline in a multiline pattern space?

In GNU sed it works. For example, this matches two blank lines in a multiline pattern space (created with N) /^\n$/ Is this standard?
1
vote
2answers
14 views

bash regex to find and save string from file

I'm a noob, so thanks for your assistance. Need some bash help... My file, hi.html, has many lines of text. I need to hunt down the line that starts with Hello abc123 blah blah blah and then save ...
1
vote
1answer
42 views

Awk split and conditional printing

I have an ugly data file that I need to clean up - Date User Name Item Id Title Quantity Price ...
1
vote
5answers
146 views

Add quotation marks & comma to each word in a file

I have a list of words such as: string1 string2 string3 .... string12312 How do I convert them these words in such manner that output could be used as a JS array, i.e. "String1", "String2"..., ...
1
vote
3answers
34 views

awk to replace columns and print if condition statisfy

I have a file with below output Sending showtrans string ... Oldest redo log files necessary to restart Extract are: Redo Thread 1, Redo Log Sequence Number 29334, SCN 3364.4078507030 ...
5
votes
8answers
400 views

Generate two sequences of numbers separated by “|”

I'm using bash (CentOS 5) and looking to generate this output (I assume I could use seq or echo together maybe?): 1|1, 2|2, 3|3, .... 31|31, 32|32, 33|33, I googled seq examples for over two hours ...
0
votes
2answers
49 views

Find value in one column and print corresponding value from other one

I have text file with values as: 1 Peter 2 Paul I need a command/scipt that will take as a parameter value from the second column (e.g. Paul) and return the corresponding value from the first one ...
2
votes
5answers
135 views

How to make grep return value without comma [duplicate]

I want to extract a value from a json file so that I can process it and I try that using grep '"USDEUR" currs.json' | cut -d ':' -f 2 but it returns 0.918695, and the json file looks like this: { ...
1
vote
3answers
88 views

Compare two files with awk

I'm trying to compare two files and print the output if they match on some fields. File1.txt: bart:29482164591748 apu:29482164591748 smithers:68468468468464 lisa:68468468468464 ...
10
votes
8answers
290 views

sed in-place line deletion on full filesystem?

Due to an application bug as yet undiagnosed, I have several hundred servers with a full disk. There is one file that has been filled up with duplicate lines—not a log file, but a user environment ...
1
vote
2answers
66 views

Select filename with highest version number

highestVersion() { for TEST_PATH in `ls -a "$/TEMP"/*.flg` do echo " you are here" TEST_FILE=`basename "$TEST_PATH"` echo "$TEST_FILE" done } Below is the file ...
3
votes
1answer
128 views

How can I write data separately in many text files which contain the same fields?

I have a text file which contains around 9999999 lines. Here I'm pasting the few lines: 1874641047 Gazipur 1874646347 Jessore 1845105653 Chittagong 1845146123 Narayanganj 1845164162 Gazipur ...
5
votes
7answers
295 views

Remove lines from tab-delimited file with missing values

I have a large (~900MB) tab-delimited text file that I will be processing in a downstream program. I need to delete any line with a missing value. The correct number of columns are on each line (so ...
4
votes
3answers
167 views

How to replace text in a column

I have a huge file (more than 2 GB) where data is like below. 12,324,32342,E:fsdsf,23432,34534,45345,324 13,3224,342,E:werwefsdsf,23432,34534,45345,324 121,3244,33442,E:,23432,34534,45345,324 ...
1
vote
1answer
41 views

Don't run this sed when a string is matched?

I have this sed command for deleting numbers inside of tags: sed 's/\(<[^0-9>]*\)[0-9]*\([^0-9]*>\)/\1\2/g' But does anyone know how I can exclude certain strings from this? Now it would ...
3
votes
2answers
57 views

Replace only on the first matching line with sed

Using BSD sed (no GNU extensions), how can I perform an operation similar to the example provided below, but where instead of the line number, the replacement is performed on the first line in which a ...
0
votes
2answers
42 views

Remove both lines if column A values are repeated in next row

My input is like this: pak_c35079_g1_i1|m.14890 Pfam pak_c20797_g1_i1|m.3458 ProDom pak_c20797_g1_i1|m.3458 Pfam pak_c28265_g1_i3|m.9595 TIGRFAM pak_c28265_g1_i3|m.9595 ...
2
votes
2answers
25 views

How do I keep the lines with the highest number where a pattern is matched?

For example, if my file looks like this: string string1 string2 string4 string800 value2 value3 value5 value10 something18 something20 I want the output to be string800 value10 something20 ...
5
votes
3answers
237 views

egrep regular expression - same word in the beginning and end

I want to find all the lines that have the same word in the beginning and in the end of the line. For example: goodword fgdlakj 3t sfkl 43lk fkl goodword bad sfa;lk3t dgk;gs 34;kl ...
0
votes
1answer
28 views

arithmetic operation with columns in text file

I have a data file which consists of two columns. I need to perform arithmetic operation with these columns as follows; column_1=a(i) column_2=b(i) where i is the line number ...
1
vote
2answers
54 views

How to ONLY print the number that is being produced from du -sh /home?

I am new at unix systems and I wanted to know how to ONLY print the number that is being produced from: du -sh /home because if I execute that command it will print: 44M /home I only want the 44M ...
-2
votes
2answers
30 views

how to append a value into the empy text file

assume I have an empty text file, touch empty_file How can I insert a specific value into the first line of this empty_file?
1
vote
2answers
33 views

Find out most frequent strings in section of file

I have a file that looks like this: o99 ksad<tab>abcde $%!H$!(<tab>00 !sdaf as<tab>12 sarwa &!§! &% (where <tab> represents a real TAB character). For clarification: It ...