Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
1answer
6 views

Move up every other line to end of line containing character

I have ran 2 xpath queries piped together to retrieve certain information. However the results came out formatted on separate lines when I want them to be on the same line. So for example the lines ...
0
votes
0answers
14 views

I have two arrays roles added and roles deleted, want to merge them and write in this order in perl [migrated]

I have another scenario. My file has the following fields: Username | Roles | Type |date | abc|admin |added | 01072015 abc|developer |deleted |01072015 abc|deploy |added |01072015 xyz |admin |deleted ...
6
votes
4answers
233 views

How to grep-inverse-match and exclude “before” and “after” lines

Consider a text file with the following entries: aaa bbb ccc ddd eee fff ggg hhh iii Given a pattern (e.g. fff), I would like to grep the file above to get in the output: all_lines except ...
1
vote
4answers
99 views

Delete everything before / on every line

I am trying to delete certain that appears before / on every line. I have something like: testing.db.com/7fad416d-f2b3-4259-b98d-2449957a3123 testing.db.com/8a8589bf-49e3-4cd7-af15-6753067355c6 ...
4
votes
1answer
49 views

How can I repair a tar file where stderr got mixed with stdout?

I have a tar file that I'm trying to repair. The problem is that tar's stderr got mixed in with its stdout, so the file looks like this: % head android-1435613730.tar tar: removing leading '/' from ...
1
vote
3answers
79 views

Merge duplicated lines which have the same fist three filed

I have the below input (thousand lines). I want a sed command to merge duplicated lines with the same fist three filed but adds only different fields and delete N/A: D04005;4;279;0;0;SSM-4-1 ...
0
votes
1answer
54 views

What is the cause of the sed error?

In my grunt file I have the command like this "sed -i index 's/API_CONTEXT_URI/http:\\/\\/localhost:5557/g' www/index.js" This is not a file I have written; this has to come to me as a maintenance ...
0
votes
1answer
41 views

How to extract particular fields from lines using shell commands?

File1: 27/07/2014 10.10.03 Breaker1 was closed by Mr. A.Basu Spectrum ui6dms 29/07/2014 04.50.53 Breaker1 was opened by Mr. D.Gupta Spectrum ui6dms and one note added 30/08/2014 03.20.01 EarthSwitch ...
-6
votes
1answer
64 views

Shell script to get common elements from two files [on hold]

My problem is as follows: I have two files in the following format: File 1: 1000 LSI Logic / Symbios Logic 0001 53c810 0021 53c1010 66MHz Ultra3 SCSI Adapter 1000 1000 ...
2
votes
2answers
32 views

Sort a list and separate item with commas instead of newlines

I want to list all the filesystems in a single comma-delimited list, in dictionary order. I'm using this command: cat /proc/filesystems | sed 's/nodev//' | sed 's/,$//' And my output looks like ...
3
votes
2answers
315 views

Extract only chinese characters

I am working on a text written in Italian and Chinese and I need to extract only Chinese characters using AWK. How can I do this? I tried: [The range of Chinese Unicode chars is 4E00 thru 9FFF (344 ...
2
votes
1answer
33 views

Why should you avoid “stringently columnar” input formats?

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time [McIlroy78]: (ii) Expect the output of every program to become the input to ...
4
votes
4answers
167 views

How can I convert two-valued text data to binary (bit-representation)

I have a text file with two (2) only possible characters (and maybe new lines \n). Example: ABBBAAAABBBBBABBABBBABBB (Size 24 bytes) How can I convert this to a binary file, meaning a bit ...
-1
votes
2answers
23 views

To sed match and some text in newlines

You want to visualize maximum 10 marks but minimum 1 mark after the sequence 65 72 62 65 in a newline where the dummy data source is hexdump -v /usr/bin/telnet and the less is a viewer. Code hexdump ...
2
votes
2answers
64 views

Compare 2 Files if it falls between a range of numbers

I have 2 files, File 1: 4 7 2 File 2: 1 5 Alpha 6 10 Bravo 11 15 Charlie 16 21 Delta 22 30 Echo I want to compare every entry in file 1 if it lies between col 1 and col 2 of file 2 (should be ...
2
votes
2answers
33 views

Workaround for busybox head that drops NUL from data

TL;DR: Latest busybox (1.23.2) provided head is buggy and will happily drop NUL characters in the provided data. Replacing the binaries is not an option. I'm using head -[nc] -X in my scripts, is ...
1
vote
2answers
36 views

Extract lines from a file using vim or teminal

I have the following in a file. I'd like to use Vim or terminal to extract title and its content like this. How can I achieve this? Topic 1 - Core: Algebra 1.1 Arithmetic sequences and series; sum ...
-3
votes
2answers
40 views

Transform text records using only a “for” loop

I have an input file that is text, consisting of lines that look like: 352|C|PAID|7036|VOICE|01-FEB-12 I need to split each line into fields at the | (vertical bar) characters and output each field ...
4
votes
2answers
183 views

How to append the lines of a file to the end of the lines of other file?

File #1: I have foofoo You have foobar she/he has foo File #2: bar foobar barfoo Final: I have foofoobar You have foobarfoobar she/he has foobarfoo
-1
votes
3answers
74 views

sed - Remove lines that contain “hotmail” in the third field

I am trying to delete all emails that contain the word hotmail in the third field from my .txt file. I am currently trying to use the following but it only deletes some lines. I am not sure why sed ...
1
vote
2answers
75 views

Extracting information from entries in a LDIF file

I've a file that contains entries like this: ddn: cn=312548,ou=Entities,ou=Active,ou=Vault,o=abc member: cn=312548001,ou=Unnamed,ou=Identities,ou=Active,ou=Vault,o=abc member: ...
-1
votes
1answer
17 views

Matching 2 different files by first column

So I have 2 different files. file1.txt is: abcdefjeiireiiioe:data:otherdata rijirjeiwrieeoooe:datamore:otherdatamore file2.txt is: abcdefjeiireiiioe:data:otherotherdata ...
1
vote
1answer
51 views

Bash script to edit text in csv-format

As part of an inventory bash script I am checking SFP modules and adding them to a .csv-file. The format currently looks like this, with varying lengths: RO01;1000BaseSX SFP;1000BaseSX SFP ...
1
vote
2answers
46 views

Using df and trying to awk a specific column

Among other problems with this script I am trying to make (this is the first unix/linux script I have ever made), I am trying to use df to get the % of disk space used and email it to the root. Our ...
7
votes
7answers
344 views

Removing numeric values in certain columns whilst keeping minus signs?

I put my hands up to being a complete beginner, so apologies if I'm doing this wrong. Say I have the following data frame that continues indefinitely horizontally and vertically with negative numbers ...
1
vote
2answers
33 views

Concatenate multiple fields separately based on one (key) column

I have a tab-separated test data like this: A 1 2 B 3 4 B 5 6 C 7 8 I want to concatenate (aggregate) both column#2 and #3 based on duplicate values ...
2
votes
4answers
44 views

Sum of alternate values in a column using either sed or nawk

foo.txt: 1 10 11 2 20 22 3 30 32 4 40 42 5 50 52 6 60 62 7 70 72 8 80 82 9 90 92 10 100 110 Desired Out.txt : 1 10 11 2 20 22 3 ...
1
vote
1answer
17 views

vim: Automate context-sensitive parenthesis replacement with surround.vim

I am using vim with the surround.vim plugin and want to replace all occurrence of sin(...) with sin[...] in a (very long) mathematical expression (containing matching parenthesis). Example input: ...
1
vote
2answers
143 views

Another awk(ward) question - mapping substrings with multi-keys

I have two files that look like this: mapping.txt: 100178 L08/2015 100180 L08/2015 103822 J03/2012 103823 A06/2012 103823 J02/2012 103823 J03/2012 ... input.txt: 100180011|225| ...
0
votes
3answers
45 views

match same lines

I've an file which I need to know if this file has entries which are the same The file contains entries like this: dn: cn=ccb2fa1a-6efb-4f29-b18b-72e226d76935,ou=Named,ou=Identities,ou=Active,o ...
2
votes
3answers
68 views

Removing duplicates every X number of lines

Is it possible to remove only the AWD015F11 duplicate using Awk? I have a large file with one column. This happens every 24 rows. I have tried counting the rows in search for a pattern that I could ...
2
votes
2answers
36 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
1
vote
1answer
35 views

How to add a new line before a specific string

I have a file like this (it shows in single line) Status: 012, Id: -1434606409zzzzz, Number: +9257884Status: 013, Id: -1434606409efwe, Number: +92465745Status: 014, Id: -1434606409fwe6, Number: ...
1
vote
4answers
46 views

Find text in a file and copy it to another file

Is there any command in Linux to find text in a file and if found then copy it in another file? Using sed -i we can find text but how to copy whole line in another file?
0
votes
5answers
85 views

Print everything between two patterns, then delete first and last line of the resulting output

otherdata otherdata start_data one two three four end_data otherdata otherdata The resulting output should just be: one two three four This looked like a job for sed to me: sed -n ...
6
votes
1answer
105 views

How can I extract/change lines in a text file whose data are separated into fields?

How can I manipulate field-based data from the commandline? For example How can I print only lines whose Nth field is foo? How can I print only lines whose Nth field isn't foo? How can I print only ...
1
vote
3answers
46 views

Delete lines that contain X in specific column .csv [duplicate]

I currently use the following to delete all emails that contain EXAMPLE sed -i '/EXAMPLE/d' newname.csv However that deletes lines that have EXAMPLE in any of the columns. I want it to only delete ...
1
vote
2answers
59 views

Keep line that contains X within the first two fields

I have a .txt converted from .csv that looks like this Smith","example","example","example","example" example","Smith","example","example","example" example","Smith","example","Smith","example" ...
3
votes
6answers
116 views

Case-insensitive count duplicate line, remove duplicate by choose case with highest duplicate

The duplicate is combination of different case text. I need to count number of duplicate (case-insensitive) and then I need to remove duplicate by choose case with highest duplicate. Below example: ...
4
votes
4answers
481 views

How to insert all lines from one file to another file?

File1: .tid.setnr := 1123 .tid.setnr := 3345 .tid.setnr := 5431 .tid.setnr := 89323 File2: .tid.info := 12 .tid.info := 3 .tid.info := 44 .tid.info := 60 Output file: .tid.info := 12 ...
1
vote
5answers
88 views

Only keep .csv column containing X Y Z data?

How can I keep .csv line data that only contains the words dog or cat and only check for them on the second column, if the word "dog" appears on any other column but not the 2nd then it should be ...
3
votes
1answer
31 views

Match pattern, do calculation and print only 3 or 4 floating values max

I would like to convert below to GB and print only 3 floating values v copvol2 fsgen ENABLED 204800 - ACTIVE - - v copvol3 fsgen ENABLED 204800 - ...
10
votes
3answers
407 views

How can I combine values from two columns?

I have a file in the following format: $ cat /tmp/raw 2015-01 5000 1000 2015-02 6000 2000 2015-03 7000 3000 Now, what I want is to get the combined value from columns 2 and 3 in each ...
7
votes
6answers
183 views

How to efficiently split up a large text file wihout splitting multiline records?

I have a big text file (~50Gb when gz'ed). The file contains 4*N lines or N records; that is every record consists of 4 lines. I would like to split this file into 4 smaller files each sized roughly ...
0
votes
2answers
41 views

How to find and replace only a particular word (which is at the end of the file) using shell command?

File1: Dms 01.01.2015 feeder1 6kv close at 04:30 Updated Dms 01.01.2015 feeder1 6kv open at 06:20 Updated Dms 04.02.2015 feeder10 6kv close at 17:23 Updated Dms 04.02.2015 feeder12 6kv ...
0
votes
1answer
33 views

Bash: How to append a range of lines from a file to another

Let's say I have file contents similar to the following (fileA): lorem.ipsum=dolor sit amet. donec.non.nibh=ut tortor nam.mattis.lacus=et rhoncus sodales get.placerat=neque fringilla. What are the ...
0
votes
2answers
58 views

Find all characters “[1-777])” at the beginning of the line

Pseudocommand perl -pe 'g/^[1-777])/g' data.txt where I try to all numbers at the beginning of a line with ")" following them etc 1), 67) and 777). Data where all question numbers should (etc 1), ...
1
vote
2answers
48 views

Match beginning of a line with something there?

I want to match the beginning of lines where the line contains something. Command in Perl/Sed 's/^.?/\section{}/g' but the problem here is that it replaces also the first character of sentences. ...
0
votes
0answers
15 views

Why this exclusion not working for long sentences? [migrated]

Command perl -ne 'print unless /.[240,]/' input.txt > output.txt which includes some sentences which are longer than 240 letters. Why? Example data Development of World Funny Society ...
5
votes
2answers
317 views

Read multiple lines from text files in bash

When I am shell scripting, the majority of what I am doing is wrapping the I/O of other modules in python, matlab, etc. To do so, I usually use text files or something of that nature with the ...