Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
2answers
24 views

How do I use newlines inside () with sed?

When I want to replace, say Dot with Dot ##empty line## I do this: sed 's/Dot/Dot\ /g' Yes, a literal new line, this is the way it works on BSD sed. But, I can't do the same when the sed is ...
3
votes
1answer
57 views

grep fixed string at end of line

I need to grep for string containing special characters and make sure it is at the end of the line For example, in list: abc defg[h]ij I want to grep for defg[h]ij$ xyz 123*456 I want to grep for ...
0
votes
1answer
23 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
3
votes
1answer
43 views

How to find out the biggest number in many documents that contains different numbers

For example, there are some temperature data at those folders in different time. temps.txt contains the temperature number. So how can I use bash script to find out the maximum temperature? (the ...
0
votes
1answer
31 views

script - recursively iterate over newline delimited list using a given depth range [on hold]

Years ago I encrypted a disk back-up image with an extremely long passphrase which I can no longer recall. However the passphrase is comprised of a number of smaller phrases which I do recall. ...
0
votes
0answers
30 views

Using s command to subsitute variables [migrated]

I need to replace a incorrect Illumina fasta header for formatting in QIIME, and this script doesn't quite work like I want. I need to able to put in variables and use the unix "s" command. Any ideas? ...
0
votes
0answers
48 views

Cut specific characters into columns [on hold]

I have a file as below: 2043630001007316220473533459926000000000003960CANDOVER INVEST S LONGHURST EXP 143292043630001007304400000000000000000000000000000 ...
1
vote
1answer
39 views

Add 10 to a value in large xml file

I have in a large xml file some hundred svg graphs where I want to change the <svg:g transform="matrix(1 0 0 1 59.06 88.74)"> 88.74 to 98.74 i.e to add 10 to every line that starts with ...
3
votes
7answers
169 views

Rows to column conversion of file

Suppose I have a file: File1: PAPER TEAM MANISH NISHA GARIMA JYOUTI ........etc File2 I want: PAPER TEAM MANISH NISHA GARIMA JYOUTI Rows to column conversion of File1.
4
votes
1answer
85 views

extracting a subset of lines from a large text file

I have a large file with 9 columns, tab delimited. This file is about 39MB, with about 250,000 lines. The last column, column 9, has information like this (note that this is all one column - thie ...
4
votes
8answers
561 views

Remove everything that is not a 4 character number

I need to remove everything that is not a 4 character number such as 9838, 6738, 1337 or 1889. I though that this command would work: sed 's/....[^0-9]//g' . Means any character in regex, and ...
1
vote
3answers
140 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
0
votes
3answers
66 views

How to print only certain characters?

I have to translate the permissions of a file this way: r ► c w ► s x ► e Then, I must extract the group of characters which are related to owner group. Last step is to print that group of ...
28
votes
3answers
2k views

Why is using a shell loop to process text considered bad practice?

Is using a while loop to process text generally bad practice in POSIX shells? As Stéphane Chazelas pointed out, some of the reasons for not using shell loops are conceptual, reliability, legibility, ...
0
votes
1answer
37 views

sed command to print all line starting from and end to specific words present in a file

Suppose I have one file File1 consist two Words File1 : PAPER TEAM Now i need to print all the lines of a file M starting from word PAPER end to word Team sed command to print between two words ...
4
votes
7answers
450 views

Command line tool to “cat” pairwise expansion of all rows in a file

Suppose I have a file (call it sample.txt) that looks like this: Row1,10 Row2,20 Row3,30 Row4,40 I want to be able to work on a stream from this file that is essentially the pairwise combination of ...
3
votes
4answers
284 views

How to use sed to selectively remove leading zeros

How can I substitute/remove 0's in this, using sed: 2001:0db8:03cd:0000:0000:ef45:0006:0123 to make it looks like this: 2001:db8:3cd:::ef45:6:123 There are 8 pieces of 4's divided by : in the ...
3
votes
1answer
92 views

Sorting lists when there are no leading zeros

I would like to sort this numerically and the sort command keys off the first character. CPL_V11.01.00_1 CPL_V11.01.00_10 CPL_V11.01.00_2 CPL_V11.01.00_3 CPL_V11.01.00_35 CPL_V11.01.00_36 ...
0
votes
1answer
47 views

No output from grep -P 'com\K\w+'

I have a source file saved off the internet which has various links I want to pull data from. For example https://blahblah.com/uniquetext I'm using grep -oP 'com\K\w+' source.txt > list.txt ...
1
vote
2answers
39 views

Oneliner which prints groups and their numbers which start with a '1'?

I have to write an oneliner which print system groups and their identifiers, for all groups whose identifiers start with '1'. For example, the result of cat command on /etc/group is this: I must ...
0
votes
3answers
55 views

Group by and sum in shell script without awk

I have a file like: $ cat input.csv 201,100 201,300 300,100 300,500 100,400 I want to add the values in column 2 which has same value in column 1. Expected output is as follows: $ cat output.csv ...
3
votes
1answer
78 views

How do I use awk to create an index of words in file?

I have a problem with my homework I need to create a shell script using awk. The script will output only the words in the file in an alphabetical order, and after the word is : after that a space then ...
1
vote
3answers
52 views

Merging multiple rows into a single one using awk

I'm working with a very large data set in the format: chr1 45162 . . C T chr1 45162 . . C T chr1 45162 A . . T chr1 45162 . . C T chr1 45257 A . . T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 ...
0
votes
1answer
33 views

How to Trim below highlighted lines from Shell output

How to Trim below highlighted lines from Shell output. Source shell program is below the output. $ ./checkhome.sh The Oracle base remains unchanged with value /opt/oracle Connecting ...
3
votes
2answers
54 views

How to remove specific keys from a wordlist?

I have generated a wordlist of keys starting from 03000000000 and ends at 03999999999 through crunch. So what I want is to delete a specific number of keys from it, for example, I want to delete keys ...
3
votes
3answers
24 views

Parsing Log Files with sed -e. Need to count unique class names

I have a file, let's call it filename.log, in it I have something like this (2014-11-18 14:09:21,766), , xxxxxx.local, EventSystem, DEBUG FtpsFile delay secs is 5 [pool-3-thread-7] (2014-11-18 ...
3
votes
5answers
166 views

removing extensions in a column

I have a file like this ILMN_1343291 TGTGTTGAGAGCTTCTCAGACTATCCACCTTTGGGTCGCTTTGCTGTTCG NM_001402.5 ILMN_1343295 CTTCAACAGCGACACCCACTCCTCCACCTTTGACGCTGGGGCTGGCATTG NM_002046.3 ILMN_1651209 ...
1
vote
2answers
77 views

Search for pattern and append line to another file

I have a file like this (five tab-separated columns) head allKO.txt Metabolism Carbohydrate metabolism Glycolisis K07448 Metabolism Protein metabolism protesome K02217 and I want to search for the ...
2
votes
1answer
31 views

How can I add four spaces before each lines of programs code in terminal

I usually go to stackexchange when I face programming terminal and I spend most of time trying to append four spaces before my pasted programming code. Is there any faster way to do it from a ...
0
votes
2answers
27 views

Grep matching cell of the csv file and index of that row

I have csv file and I want to see numbers that start with 0.99 and their row's index which is the first cell of that row. This is what I have so far: cat fil.csv | grep '0\\.99'| tee > (cut -d, ...
3
votes
4answers
267 views

Command to remove a portion of JSON data from each line?

I have test.json file with different lengths of rows. Some fictitious example: { a: 123, b: sd, c: x45, d: 1, e: '' } { a: 5, b: bfgg, c: x4c, d: 31, e: '' } I want to cut the whole substring after ...
3
votes
3answers
129 views

Grep for searching pattern in a file

I want to use grep to search pattern from one file in a second one. My pattern file is something like: K02217 K07448 KO8980 The file to search in is: >aai:AARI_24510 proP; proline/betaine ...
1
vote
1answer
40 views

Removing duplicate lines after comparing first pattern and special pattern

I'm looking for code to removing duplicate lines after comparing the first tab-delimited pattern: Example : (↦ = tab) car ↦ ab ↦ 38/43 car ↦ ca ↦ 2/34 house ↦ ca ↦ 34/34 house ↦ d ↦ 2/3 house ↦ ht ↦ ...
0
votes
3answers
37 views

How can I convert timestamps in a column to a date?

I have a file containing this: 1415602803,LOGIN SUCCESS,AUTH,user2,192.168.203.63,10.146.124.73,59996,22 1415602807,LOGIN SUCCESS,AUTH,user1,172.24.31.10,172.32.1.1,48191,22 1415602811,LOGIN ...
3
votes
4answers
61 views

Replace a marker in a text file with arbitrary new text

I have a file which contains some text and - on a single line - a marker which indicates where new content should be added foo bar %SUBSTITUTE% foo The line line substitute should be replaced by a ...
3
votes
4answers
124 views

Get pairs of lines and multiply [closed]

I'm trying to parse an html file for some values using bash. So I've got a big text file with numbers like this 3 2 4 2 What I want to do is get two lines, multiply both numbers and then store the ...
3
votes
2answers
199 views

Delete every Nth line in shell

I'm trying to delete a line after N lines using awk and I can't seem to get it right. The file format is like this YYYYYY XXXXXX XXXXXX YYYYYY XXXXXX XXXXXX The real example would be office3 3 1 ...
3
votes
2answers
146 views

Bash - echo line by line, ignoring the space between the lines

I had read the post on: bash - replace space with new line, but it does not helped to solve my issue. [My Directory] [root@testlabs Config]# ls Archive Backup_Files config_file.tgz ...
1
vote
1answer
29 views

Using “ps -C java -o pid,user,cmd” Can I reduce the output of cmd?

I am trying to get this command down to just the essential information that I need. In the command cmd comes back with this output... /usr/local/jre1.7.0_51/bin/java -Dwd.tag=boxect ...
8
votes
6answers
411 views

How to append Line to previous Line?

I have a Log file which need to be parsed and analysed. File contains something similar like below: File: 20141101 server contain dump 20141101 server contain nothing {uekdmsam ikdas jwdjamc ...
1
vote
0answers
31 views

How can i compare 2 files and add lines that aren't on the first one?

I have run into a problem while doing some coding, and I need to compare two files, here is an example input file inputfile1.txt: car php module so js node array and another input file: ...
0
votes
3answers
83 views

Simple way to extract value from HTML

I have a very simple html file with a value inside. Value is 57 in this case. <eta version="1.0"><value uri="/user/var/48/10391/0/0/12528" strValue="57" unit="%" decPlaces="0" ...
7
votes
4answers
309 views

Output part of each line to a separate file

I have a file like this: a AGTACTTCCAGGAACGGTGCACTCTCC b ATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCAT c ATATTAAATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCATCCACTCCACAC d ...
2
votes
2answers
33 views

Match missing values in text files

Referring to this question : Finding missing value in text file i have 2 files containing below data example Name Feature Marry Lecturer Marry Student Marry ...
1
vote
4answers
35 views

Sort datetime string in 12-hour format

I have a log file with contents like this: 11-12-2014 - 03:03:59AM lat = 41.990516; lon = -93.430704<br> 11-12-2014 - 03:05:15AM lat = 41.001546; lon = -93.443352<br> 11-12-2014 - ...
1
vote
2answers
21 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
31 views

How to replace a symbol in sed only if it occurs before a letter?

In my documents, I have lots of occurences like these: [["A [["b [["8 [["c [["F [["1... What I need to do is to replace double brackets with single brackets, but only if there is a letter after a ...
8
votes
8answers
503 views

Grep for pattern at start or middle of a line

I'll start by saying that I think this problem is a little less innocent than it sounds. What I need to do: check for a folder within the PATH environment variable. It could be at the start or ...
0
votes
3answers
42 views

Reduce length of specific column in delimited text file

I need for the 5th column in a delimited text file to be reduced to just the first 5 characters. All other columns must remain unedited. Input: file1.txt column1 column2 column3 ...
3
votes
2answers
77 views

Finding missing value in text file

I have a text file with following data. Name Feature Marry Lecturer Marry Student Marry Leader Bob Lecturer Bob Student ...