Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
1answer
54 views

A common way of combining two files in Solaris and Linux

I am trying to write a script that would combine two files into one and should work in both Solaris 5.10 and a Linux box. File1: dit710 dit710 dit720 dit720 File2: FacA D0000000000000001 FacA ...
0
votes
4answers
52 views

How do I insert the first line of one file into the first line of another?

I've got a head command to get the first line of a file and a sed command to insert before the first line of a file. How do I combine these so I can insert the extracted first line of one file before ...
-1
votes
3answers
42 views

How to filter log file for string inside single quotes?

I have a log file, which outputs information this way: 2016-01-01: foo bar fnord 2016-01-01: this is static 'this is of interest' some blob bar 2016-01-01: this is static 'this is of interest' some ...
1
vote
1answer
39 views

Split large log file into pieces based on date

I am trying to work out a Linux command to split a large log file into pieces based on date. Using ...
0
votes
2answers
32 views

Return part of string in text file

Background: All logs go to csv for mysql import. Have a great many logs with data like: Moon,Srv-1,2016-04-04 06:59:48,Entry #103 (s) test (AlphaNum_Need_This_32_Char_Long1),Msg On (ref2357 : Act) ...
0
votes
1answer
47 views

Column comparison using awk [duplicate]

I would like to compare the first column of file1 with the second column of file2 and only if they match it should display any matching lines in file2 only as the output. The columns are separated by ...
3
votes
2answers
701 views

In vi/vim, how do I append to a file instead of overwriting it?

I know I can write to a file by simply doing :w <file>. I would like to know though how can I write to a file by appending to it instead of overwriting it. Example use case: I want to take some ...
1
vote
2answers
40 views

How to treat remote txt file as stream?

A web service writes its output in a text file. I can fetch that log via: curl -u user:password -k https://log.my-app.com/log.2016-04-04.txt Yet this always only fetches a current "full" file even ...
0
votes
0answers
24 views

Remove lines existing in the first file from the second file and remove duplicate lines from a file [duplicate]

First Question Without having to deal with a perl or php script, can I do this efficiently with some common bash programs? With wich command? can you provide a sample? Second Question Can I take a ...
1
vote
2answers
30 views

Looking for a pattern with conditions

I have a huge file in which I want to look for a word, say pattern. I'm trying to illustrate my case with an example. Suppose I have in my file somewhere Sample-pattern="abc" I want to write a ...
0
votes
1answer
16 views

Separate values based on record

Using awk is it doable to separate values based on a certain record? Let's say we have following file: ACCJSH PREMIUM I5001 7 5976 7 @# 0.2 5195 5231 0.2 @# 39 5197 39 @# If we take the @# for ...
3
votes
3answers
96 views

Is it possible to use grep lookaroud multiple times?

<table name="content_analyzer" another-key="id9"> <type="global" /> </table> <table name="content_analyzer2" another-key="id12"> <type="global" /> </table> ...
0
votes
3answers
90 views

Print start and end between two patterns excluding end of range [duplicate]

I want to use sed -n "/START PATTERN/,/END PATTERN/p" file.txt pattern to search in a file. file.txt content is ~keyword~, ~output~. ~1.~ ~output~. ~2.~ ~output~. ~keyword~, ~output~. ~1.~ ~output~. ...
1
vote
2answers
41 views

Merge 2 files in one with a customized line in between

I have 2 Files A and B. I want to merge them in File C with a customized line in between. File A: Hello File B: Hi The merged file should contain: Records in File A is Hello Records in File B ...
3
votes
3answers
93 views

Replace multiple lines with a string when the line numbers are stored in a file

I have a file with few million lines, all the same. just for an example: Known Known Known Known Known Known ... I have another file with few thousand line numbers, for an example: 3 5 6 ... I ...
3
votes
3answers
126 views

Add mtime to grep -c output and sort the output by mtime

I have a directory full of logs named in the following style: info.log00001 info.log00002 info.log00003 ... info.log09999 info.log My current output (using grep -c) I need to analyze the ...
-1
votes
2answers
32 views

Search and output string from a file to xml

Received timestamp=1459434658969: ABC: Field id=0 double 11.4 DEF: Field id=1 string >def< GHI: Field id=2 string >g_hi< I would like to read a file that contains input in the above ...
1
vote
2answers
44 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and get the the output as following [duplicate]

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
2
votes
3answers
67 views

Splitting file based on last digit of first column in UNIX

I have a file sales_$date.csv. I want to split it into 10 files based on the last digit of first column (ITEM). so in reality file would be split in 10 files as sales_$date-01.csv, ...
2
votes
3answers
36 views

`pwd`, but omit first n directories

pwd gives the full path of the current directory, for example: /first/second/third/anotherdir/thefinaldirectory How can I omit the first n, let's say three directories, so that only ...
0
votes
1answer
54 views

Compare 1st column of the 1st file and 2nd column of the 2nd file and matching

File1: Excel file (.xls) UN ID St M1 M2 SE DOF PV PA FC 17127159 0 -5.9 297.3 765.7 0.22 4 0.003 0.00389231 2.57536 17127163 2 ...
1
vote
1answer
35 views

Interactive remove files listed with paths in text file

I am trying to interactively remove files listed with their paths in a text file. The command I am trying is: xargs rm -i <filelist.txt The error I get is: rm cannot remove ...
0
votes
1answer
42 views

sed cannot insert if a file ends with empty line

I have a file uca.xml (Thunar configuration for custom actions): <?xml encoding="UTF-8" version="1.0"?> <actions> <action> <!--some code--> </action> ...
3
votes
1answer
124 views

how to use awk to do subtraction with numbers in a large file

I have a large file like this AT5G44030,TAACARA,1000,+,200,206 AT5G44030,TAACARA,1000,+,418,424 AT5G44030,TAACARA,1000,+,773,779 AT5G44030,NGATT,1000,+,114,118 AT5G44030,NGATT,1000,+,267,271 ...
1
vote
3answers
65 views

sed delete all besides first and last line of many files

I'd like to process many *.txt files in a directory - they have a common structure (; separated, common headers) but vary in the number of lines each is comprised of i.e. some are just a single line, ...
0
votes
3answers
47 views

Command to find a string inside a pattern

Contents of log file are Caused by: com.ofss.fc.framework.exception.BusinessException: The memo start date cannot be earlier than the process date. at ...
0
votes
2answers
91 views

Curl and Grep to Parsing Content

I need to parse the output of a curl command: curl --user test:test http://192.168.1.1/security/pkm.html | egrep '@company|config.pkm.password' This returns: <input type="text" ...
-1
votes
2answers
29 views

Replace string if it has a common pattern

I have a file like the following: 0/1:34:89:09 0/0:985:083:67 0/1:687:90887:975 0/1:54:79:25 0/0:5:83:67 0/1:67:9087:97 0/1:574:779:225 0/0:275:73:627 0/1:67:47:77 I want to replace any ...
0
votes
1answer
36 views

Copy process output few lines to a file

I want to copy the xinput-calibrator output to the temp.txt file from my Qt application. I am starting the QProcess from the Application Using the command xinput_calibrator | tee log.txt, I am ...
0
votes
1answer
29 views

get unique values from 1 column and sum up 2 next columns [closed]

I have a text file that looks like this: A 500 100 ABC B 200 100 BBB B 200 100 ACC D 150 235 CDD A 500 100 RER C 210 653 DDD C 210 653 DWR D 150 653 DDW D 150 653 DER What I try to accomplish is to ...
2
votes
4answers
53 views

Not enough disk space: remove first n lines of text file without copying the file [duplicate]

I want to remove the first $rmv lines of a huge text file called $filename. This text file is so big that I cannot fit two copies of it on my hard drive. The following leaves me with a blank file ...
0
votes
1answer
29 views

How to implement config file expiration time?

I have a Debian server which runs a service using multiple config files (in a similar way to apache2) which I need to add an expiration date to. The config file that has an expiration date starts with ...
-2
votes
1answer
39 views

Replace character X with character Y in bash string

I am making bash script and I want to replace one character with another character in my string variable. Example: #!/bin/sh string="a,b,c,d,e" And I want to replace , with \n. output: ...
0
votes
2answers
53 views

Problem with grep -o

I have a text file as shown below. 192.168.10.1 192.168.45.45 192.168.255.255 192.168.0.5 Random text comes here 8.8.8.8 224.23.25.12 225.1.2.3 226.0.0.0 ...
2
votes
1answer
35 views

Split text file in random halves based on category

I have a text file that looks like this: n03250847/n03250847_0.JPEG n03250847 n03250847/n03250847_1.JPEG n03250847 ... n03250847/n03250847_499.JPEG n03250847 ... n03255030/n03255030_0.JPEG n03255030 ...
1
vote
2answers
52 views

Count times an expression is true in awk.

In unix/awk code Essentially, I need to iterate through the file (data1.txt) and count the number of times the substr($0,29,2) of each line of the file == "04". data1.txt: 6597243042 ...
0
votes
2answers
72 views

format command output with variable and fixed space

How can I achieve this best maintaining the hierarchy on the left and aligning the size value on the right? Also, at the same time I'd like to round the size to one or two digits. I tried with printf ...
2
votes
1answer
49 views

find and replace a string in a file without using temp file with SED

I have a text file called sample.text which looks like this Continent Lat/Lon: 46.07305 / -100.546 Country Lat/Lon: 38 / -98 City Lat/Lon:(37.3845) / (-122.0881) I want to process this text to get ...
2
votes
1answer
29 views

Insert string or line after last instance of a specific search variable, in a loop

I'm wondering if someone could help me with a specific coding question. I have a DNA sequencing file that reads something like this (as an example): ...
1
vote
2answers
64 views

Bash script - chomp parts out of a line

I have a data file (see sample three lines below) separated by pipes. What I want to be able to do is to "chomp" out the fields and put them into variables. They are of pre-defined lengths. I want to ...
3
votes
1answer
41 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
36 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
0
votes
1answer
43 views

Removing new line characters around patterns

I have a fasta file of millions of paired sequences, that looks like this: >7001289F:56:HKH3FBCXX:2:1101:1692:2074 1:N:0:CGATGT ...
0
votes
2answers
62 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
4
votes
3answers
134 views

Insert text at specific line number

I'm working on a bash script that will split the contents of a text document depending on the data in the line. If the contents of the original file were along the lines of 01 line 01 line 02 line ...
3
votes
2answers
55 views

sed pattern and multiline substitution

I need to parse a text file and replace a certain portion of it. I believe sed is the way to do it but not sure how to handle multiple lines with it The pattern I am looking is set cells { \ cell1 ...
1
vote
4answers
104 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
3
votes
2answers
47 views

Sorting by two columns

I need help with this homework assignment: Write a shell script named sales1 using nawk and sort commands to do the following: 1. print the file in two sections, east and west, with lines alphabetized ...
1
vote
1answer
72 views

What does :&lt;i&gt; code stand for in subtitles?

I downloaded a subtitle from subscene http://subscene.com/subtitles/the-5th-wave/english/1298314 and it contains some coding such as :&lt;i&gt; and &lt;/i&gt; which is not being ...
1
vote
1answer
56 views

Extract random sample of N lines based on pattern

I have a file formatted like this: train/t/temple/east_asia/00000025.jpg 94 train/t/temple/east_asia/00000865.jpg 94 ... train/s/swamp/00000560.jpg 92 train/s/swamp/00000935.jpg 92 .... ...