Tagged Questions

Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
3answers
39 views

Group by and sum in shell script without awk

I have a file like: $ cat input.csv 201,100 201,300 300,100 300,500 100,400 I want to add the values in column 2 which has same value in column 1. Expected output is as follows: $ cat output.csv ...
3
votes
1answer
66 views

How do I use awk to create an index of words in file?

I have a problem with my homework I need to create a shell script using awk. The script will output only the words in the file in an alphabetical order, and after the word is : after that a space then ...
1
vote
3answers
42 views

Merging multiple rows into a single one using awk

I'm working with a very large data set in the format: chr1 45162 . . C T chr1 45162 . . C T chr1 45162 A . . T chr1 45162 . . C T chr1 45257 A . . T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 ...
0
votes
1answer
26 views

How to Trim below highlighted lines from Shell output

How to Trim below highlighted lines from Shell output. Source shell program is below the output. $ ./checkhome.sh The Oracle base remains unchanged with value /opt/oracle Connecting ...
3
votes
2answers
48 views

How to remove specific keys from a wordlist?

I have generated a wordlist of keys starting from 03000000000 and ends at 03999999999 through crunch. So what I want is to delete a specific number of keys from it, for example, I want to delete keys ...
3
votes
3answers
22 views

Parsing Log Files with sed -e. Need to count unique class names

I have a file, let's call it filename.log, in it I have something like this (2014-11-18 14:09:21,766), , xxxxxx.local, EventSystem, DEBUG FtpsFile delay secs is 5 [pool-3-thread-7] (2014-11-18 ...
3
votes
5answers
161 views

removing extensions in a column

I have a file like this ILMN_1343291 TGTGTTGAGAGCTTCTCAGACTATCCACCTTTGGGTCGCTTTGCTGTTCG NM_001402.5 ILMN_1343295 CTTCAACAGCGACACCCACTCCTCCACCTTTGACGCTGGGGCTGGCATTG NM_002046.3 ILMN_1651209 ...
1
vote
2answers
66 views

Search for pattern and append line to another file

I have a file like this (five tab-separated columns) head allKO.txt Metabolism Carbohydrate metabolism Glycolisis K07448 Metabolism Protein metabolism protesome K02217 and I want to search for the ...
2
votes
1answer
29 views

How can I add four spaces before each lines of programs code in terminal

I usually go to stackexchange when I face programming terminal and I spend most of time trying to append four spaces before my pasted programming code. Is there any faster way to do it from a ...
0
votes
2answers
26 views

Grep matching cell of the csv file and index of that row

I have csv file and I want to see numbers that start with 0.99 and their row's index which is the first cell of that row. This is what I have so far: cat fil.csv | grep '0\\.99'| tee > (cut -d, ...
3
votes
4answers
252 views

Command to remove a portion of JSON data from each line?

I have test.json file with different lengths of rows. Some fictitious example: { a: 123, b: sd, c: x45, d: 1, e: '' } { a: 5, b: bfgg, c: x4c, d: 31, e: '' } I want to cut the whole substring after ...
3
votes
3answers
124 views

Grep for searching pattern in a file

I want to use grep to search pattern from one file in a second one. My pattern file is something like: K02217 K07448 KO8980 The file to search in is: >aai:AARI_24510 proP; proline/betaine ...
1
vote
1answer
39 views

Removing duplicate lines after comparing first pattern and special pattern

I'm looking for code to removing duplicate lines after comparing the first tab-delimited pattern: Example : (↦ = tab) car ↦ ab ↦ 38/43 car ↦ ca ↦ 2/34 house ↦ ca ↦ 34/34 house ↦ d ↦ 2/3 house ↦ ht ↦ ...
0
votes
3answers
30 views

How can I convert timestamps in a column to a date?

I have a file containing this: 1415602803,LOGIN SUCCESS,AUTH,user2,192.168.203.63,10.146.124.73,59996,22 1415602807,LOGIN SUCCESS,AUTH,user1,172.24.31.10,172.32.1.1,48191,22 1415602811,LOGIN ...
3
votes
4answers
59 views

Replace a marker in a text file with arbitrary new text

I have a file which contains some text and - on a single line - a marker which indicates where new content should be added foo bar %SUBSTITUTE% foo The line line substitute should be replaced by a ...
3
votes
4answers
115 views

Get pairs of lines and multiply [closed]

I'm trying to parse an html file for some values using bash. So I've got a big text file with numbers like this 3 2 4 2 What I want to do is get two lines, multiply both numbers and then store the ...
3
votes
2answers
194 views

Delete every Nth line in shell

I'm trying to delete a line after N lines using awk and I can't seem to get it right. The file format is like this YYYYYY XXXXXX XXXXXX YYYYYY XXXXXX XXXXXX The real example would be office3 3 1 ...
3
votes
2answers
138 views

Bash - echo line by line, ignoring the space between the lines

I had read the post on: bash - replace space with new line, but it does not helped to solve my issue. [My Directory] [root@testlabs Config]# ls Archive Backup_Files config_file.tgz ...
1
vote
1answer
28 views

Using “ps -C java -o pid,user,cmd” Can I reduce the output of cmd?

I am trying to get this command down to just the essential information that I need. In the command cmd comes back with this output... /usr/local/jre1.7.0_51/bin/java -Dwd.tag=boxect ...
8
votes
6answers
396 views

How to append Line to previous Line?

I have a Log file which need to be parsed and analysed. File contains something similar like below: File: 20141101 server contain dump 20141101 server contain nothing {uekdmsam ikdas jwdjamc ...
1
vote
0answers
30 views

How can i compare 2 files and add lines that aren't on the first one?

I have run into a problem while doing some coding, and I need to compare two files, here is an example input file inputfile1.txt: car php module so js node array and another input file: ...
0
votes
3answers
69 views

Simple way to extract value from HTML

I have a very simple html file with a value inside. Value is 57 in this case. <eta version="1.0"><value uri="/user/var/48/10391/0/0/12528" strValue="57" unit="%" decPlaces="0" ...
6
votes
4answers
288 views

Output part of each line to a separate file

I have a file like this: a AGTACTTCCAGGAACGGTGCACTCTCC b ATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCAT c ATATTAAATGGATTTTTGGAGCAGGGAGATGGAATAGGAGCATGCTCCATCCACTCCACAC d ...
2
votes
2answers
33 views

Match missing values in text files

Referring to this question : Finding missing value in text file i have 2 files containing below data example Name Feature Marry Lecturer Marry Student Marry ...
1
vote
4answers
33 views

Sort datetime string in 12-hour format

I have a log file with contents like this: 11-12-2014 - 03:03:59AM lat = 41.990516; lon = -93.430704<br> 11-12-2014 - 03:05:15AM lat = 41.001546; lon = -93.443352<br> 11-12-2014 - ...
1
vote
2answers
21 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
30 views

How to replace a symbol in sed only if it occurs before a letter?

In my documents, I have lots of occurences like these: [["A [["b [["8 [["c [["F [["1... What I need to do is to replace double brackets with single brackets, but only if there is a letter after a ...
8
votes
8answers
490 views

Grep for pattern at start or middle of a line

I'll start by saying that I think this problem is a little less innocent than it sounds. What I need to do: check for a folder within the PATH environment variable. It could be at the start or ...
0
votes
3answers
40 views

Reduce length of specific column in delimited text file

I need for the 5th column in a delimited text file to be reduced to just the first 5 characters. All other columns must remain unedited. Input: file1.txt column1 column2 column3 ...
3
votes
2answers
71 views

Finding missing value in text file

I have a text file with following data. Name Feature Marry Lecturer Marry Student Marry Leader Bob Lecturer Bob Student ...
3
votes
3answers
157 views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
1
vote
3answers
33 views

How to add a column from a file to the last column of another file using awk?

I want to add the second column of file2 to file1, that contains a number an unknown number of columns, so becoming the last column of file1. In this aim, I tried to create a file3 composed by pasting ...
0
votes
1answer
39 views

insert comma after the last letter

I use a txt file, and I need to convert it in csv file Saint Petersburg 0 10 0.1 - N Moscow - 9 0 - N Novgorod 0 7 1 30 Y In ...
3
votes
3answers
276 views

Take the first column from 2 files and write it to a 3rd file

Suppose I have 2 Files, ABC.txt & PQR.txt with the data shown below as an example: ABC.txt: ABC DEF PQR.txt: PQR XYZ I want to grep column 1 from both files and write into a third text ...
3
votes
2answers
227 views

random permutation of lines of text

If I have a file with following content: 0001 0002 0003 0004 0132 0137 0138 0141 How can I get a random permutation of them in bash?
7
votes
2answers
229 views

How can I delete all text between nested curly brackets in a multiline text file?

This question comes from How can I delete all text between curly brackets in a multiline text file? (just the same, but without the requirements for nesting). Example: This is { {the multiline ...
3
votes
3answers
192 views

How can I delete all text between curly brackets in a multiline text file?

Example: This is { the multiline text file } that wants { to be changed } anyway. Should become: This is that wants anyway. I have found some similar threads in the forum, but they don't seem ...
0
votes
1answer
35 views

Macbook System Profiler formatted

I want to format the output generated from macbook terminal command system_profiler and transpose the row values to columns under the same Application. Here's a snippet of the file: Firefox: ...
2
votes
5answers
141 views

Re-order text on each line in a file

I'm processing a log file and I need reorder each line (not sort). The log file looks like this: 11-06-2014 - 10:49:06PM lat = 41.858657; lon = -91.345142 11-06-2014 - 10:49:49PM lat = ...
3
votes
6answers
116 views

how can I use sed to replace the lists that begins with (*) in a file

Using sed how can I replace the lines that start with a star and replace it with numbers starting from 1? I need to replace the (*) in the fist of a list with numbers using sed for example > this ...
1
vote
3answers
47 views

How do I analyze report type of data in Bash?

What is the best way to read the following data in Bash? For example if the data is like a ls -l then I could read line by line and I have all the detail of a file within a line. In this case I have ...
2
votes
4answers
140 views

Concatenate different fields with different seperator

I have a tab delimited file like this: chr1 53736473 54175786 chr1 56861276 56876438 chr1 57512145 57512200 I want to concatenate the three fields result like this: ...
3
votes
4answers
247 views

Club together N consecutive line in shell?

Let say I have a text file in this format field1a field2a field3a field1b field2b field3b I want to club 3 (or in general case N) consecutive lines, how will I do it with sed or other command line ...
2
votes
2answers
56 views

Parse several thousand lines of txt into lines and columns

The file at hand lists names, credentials, roles, and privileges for thousands of users. It was passed onto me as .xls but the lines and columns are not arranged in proper form. I have reformated ...
6
votes
8answers
725 views

How do I construct a list with numbers like this?

How do I construct a list like this: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Where I run command 15 or something. Or if I specify 100 it would make it with 100 numbers or 10000 and it would make it ...
0
votes
0answers
20 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
1
vote
1answer
26 views

convert row into column based on particular key

I have file like Below. But header are in different line the header Values are printed in down to the hear. I want all the header in single row & value to the Particular header to be print to ...
6
votes
4answers
568 views

Modify a file without creating another file

Is there a way to modify a file without writing the contents to other file, without sed and awk? For example, $ cat test.txt aaa aaa bbb ccc ddd Replacing using sed with -i option, sed -i ...
5
votes
2answers
305 views

delete a line if same line exists in the previous line

I want to remove lines which exist already in the previous line from the command line in UNIX. I have the following data in a file. <xref ...
2
votes
6answers
87 views

Getting the matched word exactly

I have looked through a number of question but they do not satisfactorily reply to my question. I need to print words that match the pattern exactly not the lines containing that word. For example: ...