Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
1answer
55 views

Create table with frequency of unique names retrieved from multiple .csv files

I have 32 CSV files containing fetched information from a database. I need to make a frequency table in TSV/CSV format, where the names of the rows are the name of each file, and the names of the ...
0
votes
3answers
32 views

split lines and print it to new line for certain character

I need to split lines and print it to next line when pattern match. Like I have: ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955 And I want it to print to new line when ABC pattern will come: ...
2
votes
2answers
81 views

How can I append a # in front of each line of a block of text?

This is part of my PHP Myadmin's configuration file: Alias /phpMyAdmin /usr/share/phpMyAdmin Alias /phpmyadmin /usr/share/phpMyAdmin <Directory /usr/share/phpMyAdmin/> AddDefaultCharset UTF-...
-1
votes
4answers
50 views

How to filter the table names from .txt file [duplicate]

select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by using: awk '/select/{a=1} a; /;/{a=0}' ...
2
votes
2answers
91 views

pull out lines that contain 1 in 3rd column

I've got a text file with 633K lines all have 3 columns 768844 30050038638556 0 768892 30050018569904 1 768967 30050028970902 0 How do I pull out in a file those lines that have "1" in the ...
1
vote
3answers
50 views

Elegant way of counting how many times patterns from a file occur in another file

Imagine that we have two for example files. The first file is filled with unique names of employees created by combining the first two characters of the first name and the last 2 characters of the ...
1
vote
1answer
47 views

convert hexadecimal to binary

I have a text file with lines (each of 631 characters) consisting of 16 hexadecimal numbers 0,1,2,...,9,A,B,...,F. I would like to replace each number with the corresponding 4 bits of 0 and 1, i.e. 0 ...
-4
votes
3answers
53 views

I need a command to filter the statements from a .txt file

I have a file with the following contents: select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by ...
2
votes
2answers
78 views

Find (and strip) common initial whitespace from file/pipe

I am looking for a way to find and remove the common initial whitespace from a text stream or file. I do not want to remove all leading whitespace (that would be a simple sed 's/^[[:space:]]*//'). ...
0
votes
1answer
57 views

Only one line break in fasta file

I have a fasta file like: grep -A 7 -B 4 COSN229024 wrong.fasta: >COSN9627597 CGCTGGGCTCGCCTCCAGCCTGGCCTGCATTCCCAAATCTA >COSN8175610 CAAGAGAGAAATTCTGACACCTCCTAAGTCTACCAAGCTTT >COSN229024 ...
0
votes
2answers
34 views

Replacing string in HTML with value from lookup file

Situation: I have a source HTML file, with some values which must be completed with data from another file. Value that must be enriched is between a unique tag. <Uniquetag>Mystring1</...
-4
votes
3answers
61 views

linux : find a lastname that has 5 characters only in a text

Name, lastname, phone number .This is the text : Hace, Apoio, 703-2101 Yoliot, Apostolos, 4222-1124 Jenny,Thano,105-4122 Heriom,Koun, 510-4122 Dekon, Atolo, 502-2522 Feow, R,685-3431 I ...
3
votes
1answer
80 views

Use awk as a stream editor

awk is the swiss army knife of text manipulation. If, however, I need to change small bits in a text, I reach out to sed generally. While it might be the best tool for the job, it is worth it to ...
2
votes
2answers
157 views

What's the fastest way to get n-th line of a text file [duplicate]

Given a file with m lines, how to get the n-th line. m can be smaller than n sometimes. I have tried: method1: sed -ne '10p' file.txt method2: sed -ne '10p' <file.txt method3: sed -ne '10{p;q;}' ...
-1
votes
1answer
44 views

How to delete a specific paragraph in text file?

I have this following html file. {% load staticfiles %} <html> <head> <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css"> ...
0
votes
2answers
100 views

Why does `read` not pick up all my vars?

My apologies for the very basic question, but can someone please help explain why the following does not print 3 2 1 6 5 4? echo '1 2 3 4 5 6' | while read a b c; do echo result: $c b a; done ...
-3
votes
1answer
49 views

Removing entire line with sed not working? [closed]

I have a line which looks like this: </File2>. I want to remove it using sed, but nothing I tried worked, I tried sed '/^<\/File2>$/d' so I thought maybe theres special characters at ...
1
vote
1answer
37 views

Join two files in a batch mode

I have many files with the following names: 0001mm.xyz , 0002mm.xyz , 0003mm.xyz ... 0001qm.xyz , 0002qm.xyz , 0003qm.xyz ... I want to join the contents of 0001mm.xyz and 0001qm.xyz ...
3
votes
4answers
77 views

Find matches on adjacent lines

I want to find adjacent matching lines, e.g., if the pattern matches are $ grep -n pattern file1 file2 file3 file1:10: ... file2:100: ... file2:1000: ... file2:1001: ... file3:1: ... file3:123: ... ...
2
votes
1answer
39 views

grep a range of unix timestamps from a logfile

I have a logfile where every log event has a unix timestamp: 1460332800 event a 1460332999 event b 1460342800 event c 1460417039 event d Is there a clever unix-style way to grep all log events ...
0
votes
1answer
33 views

Removing a pattern from a string and then storing the whole string into one index in an array

So I have a file of many fields which start with "Content:" like below... NOTE: The content goes over multiple lines Content:Wonderful time- even with the snow! What a great experience! From the ...
6
votes
3answers
83 views

sed/awk subscript numbers in chemical formulas using markup tags

I have hundreds of text files that include chemical formulas together with narrative including numerical values. The formulas are always preceded by white spaces but can be followed by white spaces, ...
1
vote
1answer
25 views

Appending a value to all columns

I have a "simple" question yet I have not found an answer to it nor have I gotten my code to work. I have a bunch of columns, and I want to append the average of the last column to ALL columns. For ...
1
vote
4answers
56 views

Bash file - find value from command output and keep as variable

I am trying to make a script that will conntect to a ups and get the battery value and store it in a variable. I have managed to connect to the ups and get infomation from it but I am strugling to ...
4
votes
2answers
82 views

How Do I Remove Duplicate Words With Suffixes?

How do I compare a list of unsorted words of variable length and remove duplicate words that have a suffix? Example word list: iron curl curled railroad curler ...
11
votes
8answers
2k views

Move first N lines of output to end without using temporary file

Imagine an output of a command like 44444 55555 11111 22222 33333 how can I yank the first N lines (first two in the example above) and append them at the end, but without using temp file (so only ...
3
votes
2answers
92 views

Search, insert, move and insert

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's AGE line and move that line to the top ...
1
vote
1answer
49 views

I want to search and replace a pattern

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's age and insert a line age NIL at the end ...
0
votes
1answer
25 views

How to assign variables associated with list of ranges to another table with integers

I have a list of integers from 1 to 6k, and I have a list of ranges (start/stop) with a third value I'd like to assign to each integer within those ranges. The list of ranges might look like this, ...
15
votes
12answers
2k views

Delete duplicate lines pairwise?

I encountered this use case today. It seems simple at first glance, but fiddling around with sort, uniq, sed and awk revealed that it's nontrivial. How can I delete all pairs of duplicate lines? In ...
1
vote
1answer
53 views

How to selectively remove columns and rows with bash?

I currently have a data set that looks like this: INPUT Hybridization REF TCGA-FA-8693-01A-11D-2399-05 TCGA-FA-8693-01A-11D-2399-05 TCGA-FA-8693-01A-11D-2399-05 TCGA-FA-8693-01A-11D-...
1
vote
1answer
44 views

Capture text between curly braces in C preprocessed file

I'm working with a preprocessed C source file. I want to extract one function for additional processing. I can find the start of the particular function with: gcc -DNDEBUG -march=native -E source.c &...
-3
votes
2answers
43 views

Extract a word from a line

Created a file employees: 1. Fred 2. Billy 1 3. Sally 1 4. Jim 2 5. Jane 2 6. Sue 3 7. Meg 3 Created a file “managers”: 1. Fred 2. Bill 3. Sally I want to print something like this: Fred ...
4
votes
4answers
396 views

Command for keeping the lines with a certain length in a file

I would like to keep the lines with exactly 639 characters in my .txt file. What is the command to do so?
2
votes
4answers
314 views

print specific line from multiple files

I have a collection of 5000+ files and I just want to create a output.txt file which contain 27th line of all file along with filename. well I am noob at linux level so what i got from internet is to ...
-3
votes
1answer
27 views

AWK interval expressions to clean IP [closed]

Credit to a great many entries from you online and manuals. It works, but, is it efficient enough? Do you approve? {Reading logs +- 5k items fast as I can see} Source data from file "myAwk3.txt": ...
1
vote
1answer
44 views

Sed, Awk Multiline log file, a number {0-999] identify relation

Short version of issue: Need to extract from logs: {Using sed and awk on Linux} Time, PlayerName{Can have many special characters}, GUID, IP from log example at end. Longer version: How to combine 2 ...
1
vote
2answers
19 views

text to find and replace in all files and in all directories

I want to replace all occurrences of 1800 102 6022 with 1860 266 2666 in all files. I have different files in different directories which can contain 1800 102 6022. I am using: perl -pi.13.04.2016 -...
2
votes
3answers
52 views

Show character at position in a file

I would like to print the character at a given position using only the command line. E.g.: <command> 5 Would output a if the 5th char of that file was a. Since I am dealing with big files, ...
0
votes
1answer
57 views

How do I calculate the column percentage of a file?

I have a tsv file that contains some values. I want the sum of each column and total number of values and percentage values. Eg: file.tsv contains x 1 1 0 1 x x 1 x ...
5
votes
3answers
298 views

Grep exact number of digits and some other characters

I'd like to parse a file containing 5 digit numbers separated by comma or dash, lines like : 12345,23456,34567-45678,12345-23456,34567 My goal is to find lines which have incorrect formatting eg. ...
3
votes
1answer
49 views

Pattern matching and text processing for lists of song files

I have two files and I want to generate a new file with the lines found in file2 but not in file1. The original files are not sorted. Now, here is the tricky part. Some of the lines in the files ...
1
vote
3answers
52 views

How to grep “block” strings out of a weird file

Command on AIX is: [root@hx042:/home/user1]$ lqueryvg -Atp hdiskpower13 0516-1396 lqueryvg: The physical volume hdiskpower13, was not found in the system database. Max LVs: 256 PP Size: ...
3
votes
3answers
55 views

Grep that works with overlapping patterns

I want to count the number of repeating patterns aa in a very big file. So I use the following command: grep -o "aa" ./bwt/dblp.txt | wc -l This works, except when I have a string such as aaa, ...
3
votes
1answer
143 views

Replace with sed until match in a line

I have to replace all "." characters before the "=" character in every line in a file. The lines are like this: one.two.three=something four.five=1 six.seven=127.0.0.1 eight.nine.ten.eleven=...
3
votes
4answers
278 views

how to make list of strings from file where this strings are in own line

I have file: xxx.lst with values: 111 222 333 I just need to make one line with: 111 222 333 into a variable or standard output.
0
votes
2answers
71 views

Select rows based on the values of a specific column (not the first column)

I would like to read a file.csv file and keep the rows (i.e., all the columns) of the file if and only if the sixth column of that file matches values in user_ids_to_keep.csv. The file.csv file I ...
2
votes
3answers
62 views

Print up to (and including) the nth occurence of a pattern in a line

Let's say I have multiple lines similar to below in a file. Turbo is a cat. cats are good. cats are not dog. Coco is a black cat. cats are furry. cats are not dog. now, if want to grep all the ^.*...
3
votes
2answers
135 views

How to extract strings from one file into multiple files

I have a file, xxx.lst with the following data: aaa 111 bbb 222 ccc 333 I need to make 3 files called 111.lst, 222.lst and 333.lst respectively with aaa in 111.lst bbb in 222.lst and ccc in ...
-1
votes
2answers
36 views

awk print all matching lines including duplicates [duplicate]

How can I print all matching lines between two files including the duplicates sourcefile: SNP value RS1 0.55 RS2 0.52 RS3 0.25 RS4 0.22 queryfile: SNP RS1 RS2 RS2 RS3 RS4 RS4 Desired output: ...