Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
35 views

Group by Functionality on Mac Address

I'm looking for a solution that should do grouping of identical mac address(device ids). I tried using sort -k 2, but I'm not confident of the result. Data Set looks like: 1616.93,ac:22:0b:a6:22:c3 ...
0
votes
2answers
55 views

Merging large data set in fastest possible way

I have a large data set ~100 GBs having smaller csv files ~ 100,000. I wish to merge all .csv files together in the fastest possible way. Files are located in two different folders. Someone ...
1
vote
1answer
48 views

bash ignorecase match pattern in file and color that line, print all to screen

I have almost zero idea about coding or bash scripting, so please bare with me and excuse my horrible formatting. What I want is to cat a log file and highlight certain lines with color based on a ...
11
votes
9answers
2k views

How can I delete the 5th word of every line in a file?

I want to delete the 5th word of each line in a file. The current content of the file: File is not updated or and will be removed System will shut down f within 10 seconds Please save your work ...
3
votes
1answer
43 views

Store specific output to a shell variable

I am looking to store the output of the svn info command, more precisely the revision number into a variable for further processing. I just need the 4 stored. Can anyone help me please? $ svn info ...
-1
votes
2answers
69 views

Shell append string after last match

I would like to insert a string after the last match, example: String to insert after last instance of banana: Orange Input Apple Banana Apple Banana Output Apple Banana Apple Banana Orange I ...
1
vote
3answers
84 views

Bash convert string to array of strings?

I've got a string like this: 8080 "ac ac df asd" 9019 "f v adfs" 1 "123 da 123x" Is there a clever way to convert this into arrays like this using Bash? 8080 "ac ac df asd" 9019 "f v adfs" 1 "123 ...
0
votes
0answers
21 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
1
vote
2answers
71 views

How to extract 1st, 3rd, 5th of group of three lines by sed or awk?

Say I have a file: Line1 Line2 Line3 Line4 Line5 Line6 Line7 Line8 Line9 Line10 Line11 Line12 … I want to get: Line1 Line2 Line3 Line7 Line8 Line9 Line13 Line14 Line15 It means that I want ...
2
votes
4answers
67 views

How to parse table for pattern if pattern includes spaces

A continuation of this question: parse first column of command output, get corresponding second column value Say I have a command that outputs a string formatted as a table, as shown below. What if ...
0
votes
1answer
105 views

parse first column of command output, get corresponding second column value

I am fairly new to BASH scripting for Linux. I am executing a command that outputs a table, as such: firstValue: a secondValue: b.y7 thirdValue: c-x-g 10:42 GMT Say I need the secondValue. I ...
3
votes
6answers
435 views

Grep a file on specific field

I have two files, let's say File1: Locus_1 Locus_2 Locus_3 File2: 3 3 Locus_1 Locus_40 etc_849 3 2 Locus_2 Locus_94 * 2 2 Locus_6 Locus_1 * 2 3 Locus_3,Locus_4 ...
0
votes
1answer
35 views

How to make PDF viewer responsive to changes in Debian

I have a fresh Debian 8.1 installation and am trying to make its usability feel like OSX. I run xelatex document.tex && evince document.pdf & but it only successfully runs xelatex and ...
1
vote
2answers
38 views

Copy text only contained within quotes “” from a file

/etc/opendkim/keys/"$(hostname)"/default.txt contains the following default._domainkey IN TXT "v=DKIM1; r=postmaster; g=*; k=rsa; ...
8
votes
6answers
607 views

How do I display the nth result of an ls command?

I used this command to display the first result of files in my directory. ls | head -n 1 My simple question is, how can I modify this command to display say the nth result? Thanks!
2
votes
1answer
47 views

How to read and do a min/max/avg from a NMEA stream

I have a weather station attached to my device spitting NMEA 0183 on /tty/USB0 I have alreeady sorted out how to parse the data and echo what I want in a script. What I need to do now is write the ...
1
vote
4answers
82 views

optimize command with or and pipe to parse the output of ifconfig

I have this command line, but it's doubling the grep and awk ifconfig eth1 2> /dev/null | grep "inet " | awk '{gsub("addr:","",$2); print $2 }' || ifconfig eth0 2> /dev/null | grep "inet " | ...
1
vote
3answers
55 views

compare multiple files(more than two) with two different columns

I have four different files: file1, file2, file3, file4. Each file has 2 different columns separated by tab. I want to match first column of file1 (as reference) with the first column of second file, ...
1
vote
1answer
93 views

Pick columns from a variable length csv file

I have two CSV files, which share a column that is unique to each row in each file, like an ID. The files don't have a header. The file_2 has variable length columns e.g. 1,2,3 4,5,6,7,8 I joined ...
1
vote
1answer
56 views

validate the sequence number in the trailer of a file

We have a file that arrives daily and has a sequence number in the trailer like 9999 TR SAP _AP Distribution 0000000000000001 where 0000000000000001 is the sequence number and the file that ...
3
votes
3answers
122 views

Undo letterspacing with sed

I have a source text file containing text where some words are l e t t e r s p a c e d like the word "letterspaced" in this question (i.e., there is a space character between the letters of the word. ...
1
vote
4answers
92 views

“head” only printing one line?

I've got a CSV file that's generated by saving as CSV from Excel. If I do "head" (or indeed "grep" or anything else) it only prints the first line: head -n 10 messy.csv 10,15,11,21 But if I open ...
-2
votes
1answer
33 views

Extract single information from a column in a file

I have a file with tab-separated fields in the following format: 2-micron 251 1523 R0010W . + SGD gene . ...
3
votes
4answers
92 views

Print alternating lines with conditional

Still struggling with messy logfiles. What I want is to find lines that first matching string X and then string Y. I then want to print them together. The problem is that, sometimes there is X, but no ...
2
votes
1answer
37 views

Move up every other line to end of line containing character

I have ran 2 xpath queries piped together to retrieve certain information. However the results came out formatted on separate lines when I want them to be on the same line. So for example the lines ...
7
votes
4answers
574 views

How to grep-inverse-match and exclude “before” and “after” lines

Consider a text file with the following entries: aaa bbb ccc ddd eee fff ggg hhh iii Given a pattern (e.g. fff), I would like to grep the file above to get in the output: all_lines except ...
1
vote
4answers
123 views

Delete everything before / on every line

I am trying to delete certain that appears before / on every line. I have something like: testing.db.com/7fad416d-f2b3-4259-b98d-2449957a3123 testing.db.com/8a8589bf-49e3-4cd7-af15-6753067355c6 ...
4
votes
1answer
59 views

How can I repair a tar file where stderr got mixed with stdout?

I have a tar file that I'm trying to repair. The problem is that tar's stderr got mixed in with its stdout, so the file looks like this: % head android-1435613730.tar tar: removing leading '/' from ...
1
vote
3answers
92 views

Merge duplicated lines which have the same fist three filed

I have the below input (thousand lines). I want a sed command to merge duplicated lines with the same fist three filed but adds only different fields and delete N/A: D04005;4;279;0;0;SSM-4-1 ...
0
votes
1answer
68 views

What is the cause of the sed error?

In my grunt file I have the command like this "sed -i index 's/API_CONTEXT_URI/http:\\/\\/localhost:5557/g' www/index.js" This is not a file I have written; this has to come to me as a maintenance ...
0
votes
1answer
54 views

How to extract particular fields from lines using shell commands?

File1: 27/07/2014 10.10.03 Breaker1 was closed by Mr. A.Basu Spectrum ui6dms 29/07/2014 04.50.53 Breaker1 was opened by Mr. D.Gupta Spectrum ui6dms and one note added 30/08/2014 03.20.01 EarthSwitch ...
2
votes
2answers
54 views

Sort a list and separate item with commas instead of newlines

I want to list all the filesystems in a single comma-delimited list, in dictionary order. I'm using this command: cat /proc/filesystems | sed 's/nodev//' | sed 's/,$//' And my output looks like ...
3
votes
2answers
354 views

Extract only chinese characters

I am working on a text written in Italian and Chinese and I need to extract only Chinese characters using AWK. How can I do this? I tried: [The range of Chinese Unicode chars is 4E00 thru 9FFF (344 ...
3
votes
1answer
48 views

Why should you avoid “stringently columnar” input formats?

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]: (ii) Expect the output of every program to become the input to ...
4
votes
4answers
197 views

How can I convert two-valued text data to binary (bit-representation)

I have a text file with two (2) only possible characters (and maybe new lines \n). Example: ABBBAAAABBBBBABBABBBABBB (Size 24 bytes) How can I convert this to a binary file, meaning a bit ...
-1
votes
2answers
26 views

To sed match and some text in newlines

You want to visualize maximum 10 marks but minimum 1 mark after the sequence 65 72 62 65 in a newline where the dummy data source is hexdump -v /usr/bin/telnet and the less is a viewer. Code hexdump ...
3
votes
2answers
85 views

Compare 2 Files if it falls between a range of numbers

I have 2 files, File 1: 4 7 2 File 2: 1 5 Alpha 6 10 Bravo 11 15 Charlie 16 21 Delta 22 30 Echo I want to compare every entry in file 1 if it lies between col 1 and col 2 of file 2 (should be ...
2
votes
2answers
35 views

Workaround for busybox head that drops NUL from data

TL;DR: Latest busybox (1.23.2) provided head is buggy and will happily drop NUL characters in the provided data. Replacing the binaries is not an option. I'm using head -[nc] -X in my scripts, is ...
1
vote
2answers
41 views

Extract lines from a file using vim or teminal

I have the following in a file. I'd like to use Vim or terminal to extract title and its content like this. How can I achieve this? Topic 1 - Core: Algebra 1.1 Arithmetic sequences and series; sum ...
-3
votes
2answers
45 views

Transform text records using only a “for” loop

I have an input file that is text, consisting of lines that look like: 352|C|PAID|7036|VOICE|01-FEB-12 I need to split each line into fields at the | (vertical bar) characters and output each field ...
4
votes
2answers
195 views

How to append the lines of a file to the end of the lines of other file?

File #1: I have foofoo You have foobar she/he has foo File #2: bar foobar barfoo Final: I have foofoobar You have foobarfoobar she/he has foobarfoo
-1
votes
3answers
87 views

sed - Remove lines that contain “hotmail” in the third field

I am trying to delete all emails that contain the word hotmail in the third field from my .txt file. I am currently trying to use the following but it only deletes some lines. I am not sure why sed ...
1
vote
2answers
92 views

Extracting information from entries in a LDIF file

I've a file that contains entries like this: ddn: cn=312548,ou=Entities,ou=Active,ou=Vault,o=abc member: cn=312548001,ou=Unnamed,ou=Identities,ou=Active,ou=Vault,o=abc member: ...
-1
votes
1answer
20 views

Matching 2 different files by first column

So I have 2 different files. file1.txt is: abcdefjeiireiiioe:data:otherdata rijirjeiwrieeoooe:datamore:otherdatamore file2.txt is: abcdefjeiireiiioe:data:otherotherdata ...
1
vote
1answer
74 views

Bash script to edit text in csv-format

As part of an inventory bash script I am checking SFP modules and adding them to a .csv-file. The format currently looks like this, with varying lengths: RO01;1000BaseSX SFP;1000BaseSX SFP ...
1
vote
2answers
59 views

Using df and trying to awk a specific column

Among other problems with this script I am trying to make (this is the first unix/linux script I have ever made), I am trying to use df to get the % of disk space used and email it to the root. Our ...
7
votes
7answers
407 views

Removing numeric values in certain columns whilst keeping minus signs?

I put my hands up to being a complete beginner, so apologies if I'm doing this wrong. Say I have the following data frame that continues indefinitely horizontally and vertically with negative numbers ...
1
vote
2answers
59 views

Concatenate multiple fields separately based on one (key) column

I have a tab-separated test data like this: A 1 2 B 3 4 B 5 6 C 7 8 I want to concatenate (aggregate) both column#2 and #3 based on duplicate values ...
2
votes
4answers
68 views

Sum of alternate values in a column using either sed or nawk

foo.txt: 1 10 11 2 20 22 3 30 32 4 40 42 5 50 52 6 60 62 7 70 72 8 80 82 9 90 92 10 100 110 Desired Out.txt : 1 10 11 2 20 22 3 ...
1
vote
1answer
19 views

vim: Automate context-sensitive parenthesis replacement with surround.vim

I am using vim with the surround.vim plugin and want to replace all occurrence of sin(...) with sin[...] in a (very long) mathematical expression (containing matching parenthesis). Example input: ...