Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
1answer
33 views

How to read and do a min/max/avg from a NMEA stream

I have a weather station attached to my device spitting NMEA 0183 on /tty/USB0 I have alreeady sorted out how to parse the data and echo what I want in a script. What I need to do now is write the ...
1
vote
4answers
49 views

optimize command with or and pipe to parse the output of ifconfig

I have this command line, but it's doubling the grep and awk ifconfig eth1 2> /dev/null | grep "inet " | awk '{gsub("addr:","",$2); print $2 }' || ifconfig eth0 2> /dev/null | grep "inet " | ...
1
vote
3answers
33 views

compare multiple files(more than two) with two different columns

I have four different files: file1, file2, file3, file4. Each file has 2 different columns separated by tab. I want to match first column of file1 (as reference) with the first column of second file, ...
1
vote
1answer
38 views

Pick columns from a variable length csv file

I have two CSV files, which share a column that is unique to each row in each file, like an ID. The files don't have a header. The file_2 has variable length columns e.g. 1,2,3 4,5,6,7,8 I joined ...
1
vote
1answer
46 views

validate the sequence number in the trailer of a file

We have a file that arrives daily and has a sequence number in the trailer like 9999 TR SAP _AP Distribution 0000000000000001 where 0000000000000001 is the sequence number and the file that ...
3
votes
3answers
117 views

Undo letterspacing with sed

I have a source text file containing text where some words are l e t t e r s p a c e d like the word "letterspaced" in this question (i.e., there is a space character between the letters of the word. ...
1
vote
4answers
76 views

“head” only printing one line?

I've got a CSV file that's generated by saving as CSV from Excel. If I do "head" (or indeed "grep" or anything else) it only prints the first line: head -n 10 messy.csv 10,15,11,21 But if I open ...
-3
votes
1answer
29 views

Extract single information from a column in a file

I have a file with tab-separated fields in the following format: 2-micron 251 1523 R0010W . + SGD gene . ...
2
votes
4answers
67 views

Print alternating lines with conditional

Still struggling with messy logfiles. What I want is to find lines that first matching string X and then string Y. I then want to print them together. The problem is that, sometimes there is X, but no ...
2
votes
1answer
26 views

Move up every other line to end of line containing character

I have ran 2 xpath queries piped together to retrieve certain information. However the results came out formatted on separate lines when I want them to be on the same line. So for example the lines ...
0
votes
0answers
14 views

I have two arrays roles added and roles deleted, want to merge them and write in this order in perl [migrated]

I have another scenario. My file has the following fields: Username | Roles | Type |date | abc|admin |added | 01072015 abc|developer |deleted |01072015 abc|deploy |added |01072015 xyz |admin |deleted ...
7
votes
4answers
296 views

How to grep-inverse-match and exclude “before” and “after” lines

Consider a text file with the following entries: aaa bbb ccc ddd eee fff ggg hhh iii Given a pattern (e.g. fff), I would like to grep the file above to get in the output: all_lines except ...
1
vote
4answers
106 views

Delete everything before / on every line

I am trying to delete certain that appears before / on every line. I have something like: testing.db.com/7fad416d-f2b3-4259-b98d-2449957a3123 testing.db.com/8a8589bf-49e3-4cd7-af15-6753067355c6 ...
4
votes
1answer
51 views

How can I repair a tar file where stderr got mixed with stdout?

I have a tar file that I'm trying to repair. The problem is that tar's stderr got mixed in with its stdout, so the file looks like this: % head android-1435613730.tar tar: removing leading '/' from ...
1
vote
3answers
91 views

Merge duplicated lines which have the same fist three filed

I have the below input (thousand lines). I want a sed command to merge duplicated lines with the same fist three filed but adds only different fields and delete N/A: D04005;4;279;0;0;SSM-4-1 ...
0
votes
1answer
56 views

What is the cause of the sed error?

In my grunt file I have the command like this "sed -i index 's/API_CONTEXT_URI/http:\\/\\/localhost:5557/g' www/index.js" This is not a file I have written; this has to come to me as a maintenance ...
0
votes
1answer
48 views

How to extract particular fields from lines using shell commands?

File1: 27/07/2014 10.10.03 Breaker1 was closed by Mr. A.Basu Spectrum ui6dms 29/07/2014 04.50.53 Breaker1 was opened by Mr. D.Gupta Spectrum ui6dms and one note added 30/08/2014 03.20.01 EarthSwitch ...
-6
votes
1answer
64 views

Shell script to get common elements from two files [closed]

My problem is as follows: I have two files in the following format: File 1: 1000 LSI Logic / Symbios Logic 0001 53c810 0021 53c1010 66MHz Ultra3 SCSI Adapter 1000 1000 ...
2
votes
2answers
36 views

Sort a list and separate item with commas instead of newlines

I want to list all the filesystems in a single comma-delimited list, in dictionary order. I'm using this command: cat /proc/filesystems | sed 's/nodev//' | sed 's/,$//' And my output looks like ...
3
votes
2answers
320 views

Extract only chinese characters

I am working on a text written in Italian and Chinese and I need to extract only Chinese characters using AWK. How can I do this? I tried: [The range of Chinese Unicode chars is 4E00 thru 9FFF (344 ...
2
votes
1answer
34 views

Why should you avoid “stringently columnar” input formats?

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]: (ii) Expect the output of every program to become the input to ...
4
votes
4answers
169 views

How can I convert two-valued text data to binary (bit-representation)

I have a text file with two (2) only possible characters (and maybe new lines \n). Example: ABBBAAAABBBBBABBABBBABBB (Size 24 bytes) How can I convert this to a binary file, meaning a bit ...
-1
votes
2answers
24 views

To sed match and some text in newlines

You want to visualize maximum 10 marks but minimum 1 mark after the sequence 65 72 62 65 in a newline where the dummy data source is hexdump -v /usr/bin/telnet and the less is a viewer. Code hexdump ...
2
votes
2answers
67 views

Compare 2 Files if it falls between a range of numbers

I have 2 files, File 1: 4 7 2 File 2: 1 5 Alpha 6 10 Bravo 11 15 Charlie 16 21 Delta 22 30 Echo I want to compare every entry in file 1 if it lies between col 1 and col 2 of file 2 (should be ...
2
votes
2answers
33 views

Workaround for busybox head that drops NUL from data

TL;DR: Latest busybox (1.23.2) provided head is buggy and will happily drop NUL characters in the provided data. Replacing the binaries is not an option. I'm using head -[nc] -X in my scripts, is ...
1
vote
2answers
36 views

Extract lines from a file using vim or teminal

I have the following in a file. I'd like to use Vim or terminal to extract title and its content like this. How can I achieve this? Topic 1 - Core: Algebra 1.1 Arithmetic sequences and series; sum ...
-3
votes
2answers
41 views

Transform text records using only a “for” loop

I have an input file that is text, consisting of lines that look like: 352|C|PAID|7036|VOICE|01-FEB-12 I need to split each line into fields at the | (vertical bar) characters and output each field ...
4
votes
2answers
185 views

How to append the lines of a file to the end of the lines of other file?

File #1: I have foofoo You have foobar she/he has foo File #2: bar foobar barfoo Final: I have foofoobar You have foobarfoobar she/he has foobarfoo
-1
votes
3answers
76 views

sed - Remove lines that contain “hotmail” in the third field

I am trying to delete all emails that contain the word hotmail in the third field from my .txt file. I am currently trying to use the following but it only deletes some lines. I am not sure why sed ...
1
vote
2answers
77 views

Extracting information from entries in a LDIF file

I've a file that contains entries like this: ddn: cn=312548,ou=Entities,ou=Active,ou=Vault,o=abc member: cn=312548001,ou=Unnamed,ou=Identities,ou=Active,ou=Vault,o=abc member: ...
-1
votes
1answer
17 views

Matching 2 different files by first column

So I have 2 different files. file1.txt is: abcdefjeiireiiioe:data:otherdata rijirjeiwrieeoooe:datamore:otherdatamore file2.txt is: abcdefjeiireiiioe:data:otherotherdata ...
1
vote
1answer
55 views

Bash script to edit text in csv-format

As part of an inventory bash script I am checking SFP modules and adding them to a .csv-file. The format currently looks like this, with varying lengths: RO01;1000BaseSX SFP;1000BaseSX SFP ...
1
vote
2answers
46 views

Using df and trying to awk a specific column

Among other problems with this script I am trying to make (this is the first unix/linux script I have ever made), I am trying to use df to get the % of disk space used and email it to the root. Our ...
7
votes
7answers
346 views

Removing numeric values in certain columns whilst keeping minus signs?

I put my hands up to being a complete beginner, so apologies if I'm doing this wrong. Say I have the following data frame that continues indefinitely horizontally and vertically with negative numbers ...
1
vote
2answers
38 views

Concatenate multiple fields separately based on one (key) column

I have a tab-separated test data like this: A 1 2 B 3 4 B 5 6 C 7 8 I want to concatenate (aggregate) both column#2 and #3 based on duplicate values ...
2
votes
4answers
45 views

Sum of alternate values in a column using either sed or nawk

foo.txt: 1 10 11 2 20 22 3 30 32 4 40 42 5 50 52 6 60 62 7 70 72 8 80 82 9 90 92 10 100 110 Desired Out.txt : 1 10 11 2 20 22 3 ...
1
vote
1answer
17 views

vim: Automate context-sensitive parenthesis replacement with surround.vim

I am using vim with the surround.vim plugin and want to replace all occurrence of sin(...) with sin[...] in a (very long) mathematical expression (containing matching parenthesis). Example input: ...
1
vote
2answers
145 views

Another awk(ward) question - mapping substrings with multi-keys

I have two files that look like this: mapping.txt: 100178 L08/2015 100180 L08/2015 103822 J03/2012 103823 A06/2012 103823 J02/2012 103823 J03/2012 ... input.txt: 100180011|225| ...
0
votes
3answers
45 views

match same lines

I've an file which I need to know if this file has entries which are the same The file contains entries like this: dn: cn=ccb2fa1a-6efb-4f29-b18b-72e226d76935,ou=Named,ou=Identities,ou=Active,o ...
2
votes
3answers
68 views

Removing duplicates every X number of lines

Is it possible to remove only the AWD015F11 duplicate using Awk? I have a large file with one column. This happens every 24 rows. I have tried counting the rows in search for a pattern that I could ...
2
votes
2answers
37 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
1
vote
1answer
36 views

How to add a new line before a specific string

I have a file like this (it shows in single line) Status: 012, Id: -1434606409zzzzz, Number: +9257884Status: 013, Id: -1434606409efwe, Number: +92465745Status: 014, Id: -1434606409fwe6, Number: ...
1
vote
4answers
47 views

Find text in a file and copy it to another file

Is there any command in Linux to find text in a file and if found then copy it in another file? Using sed -i we can find text but how to copy whole line in another file?
0
votes
5answers
87 views

Print everything between two patterns, then delete first and last line of the resulting output

otherdata otherdata start_data one two three four end_data otherdata otherdata The resulting output should just be: one two three four This looked like a job for sed to me: sed -n ...
6
votes
1answer
105 views

How can I extract/change lines in a text file whose data are separated into fields?

How can I manipulate field-based data from the commandline? For example How can I print only lines whose Nth field is foo? How can I print only lines whose Nth field isn't foo? How can I print only ...
1
vote
3answers
46 views

Delete lines that contain X in specific column .csv [duplicate]

I currently use the following to delete all emails that contain EXAMPLE sed -i '/EXAMPLE/d' newname.csv However that deletes lines that have EXAMPLE in any of the columns. I want it to only delete ...
1
vote
2answers
59 views

Keep line that contains X within the first two fields

I have a .txt converted from .csv that looks like this Smith","example","example","example","example" example","Smith","example","example","example" example","Smith","example","Smith","example" ...
3
votes
6answers
121 views

Case-insensitive count duplicate line, remove duplicate by choose case with highest duplicate

The duplicate is combination of different case text. I need to count number of duplicate (case-insensitive) and then I need to remove duplicate by choose case with highest duplicate. Below example: ...
4
votes
4answers
484 views

How to insert all lines from one file to another file?

File1: .tid.setnr := 1123 .tid.setnr := 3345 .tid.setnr := 5431 .tid.setnr := 89323 File2: .tid.info := 12 .tid.info := 3 .tid.info := 44 .tid.info := 60 Output file: .tid.info := 12 ...
1
vote
5answers
89 views

Only keep .csv column containing X Y Z data?

How can I keep .csv line data that only contains the words dog or cat and only check for them on the second column, if the word "dog" appears on any other column but not the 2nd then it should be ...