All Questions

Tagged with
Filter by
Sorted by
Tagged with
0
votes
1answer
36 views

File read, compare and replace based on comparison in text file

Hi i need guidence to write code for file processing. i have one file which having ~ as delimeter and each line is keyword is 1st column. First i have to find the lines which matches with ^6999 Then ...
0
votes
2answers
44 views

Don't print the same string again if it was found between two files

I'm looking for the string SUCCESS into the file_1 & file_2 which in this case looks for the string and prints it based on whether its there on only file_1 or on both file_1 and file_2 while I'm ...
0
votes
1answer
61 views

sed/awk/python replace all spaces between characters after string match

I want to delete all spaces between characters after an initial match. An example is exmpl 0 t h i s a r e t he spaces to d e l e t e exmpl 1 m o r e spaces to d e l exmpl 2 o r s m t h completely d ...
-1
votes
1answer
29 views

Sort based on second column and list from highest to lowest

TGAAACATGCCTTTCACCTCCTGCCATGATTGTGAGGCCT 2 GTGTTGCCCAGGTTGGTTTCAAACTCAGCTCAAGTGATCT 1 GAGACTCCTTCAGGAGACTGGTCCCCTGTCCTCGCCTCTC 3 CATGTTCTCCTGGAAGCCATGTGAGGAAGGCACATCATGG 8 ...
1
vote
2answers
89 views

How can I extract only the pid column and only the pathname column in the lsof output?

$ sudo lsof -u t | grep -i "\.pdf" evince 1788 t 37r REG 8,4 176328 134478 /home/t/some/path1/white space/string1 + string2 string3.pdf evince 3737 t 36r ...
1
vote
1answer
41 views

Compare two text files, extract matching rows of file2 plus additional rows

have been mocking around with this too long and tried grep, join, awk but I cant get the parameters right. I need to get the command right. I have got two text files. cat file1 @ABC:11:ABC:1:...
1
vote
1answer
73 views

Scripting for text processing: Delete a set of lines only if entire pattern matches

I want to delete a set of lines (globally) only if the entire pattern matches. Pattern Description: Line1:^[#]+ .* Line2:^[[:space:]]*$ Line3:^-[[:space:]]*$ Line4:^[[:space:]]*$ Line5:^[#]+ .*$...
-3
votes
1answer
72 views

Comparing two fields from two files and printing the common items from the files with the count from each file

I have two files in a below-mentioned way File1 Emp1 Dec10 12Am Emp2 Dec10 2Am Emp3 Dec10 4Am Emp2 Dec10 6Am Emp1 Dec10 8Am File2 Emp1 Dec11 12Pm Emp2 Dec11 2Am Emp1 Dec11 3Am Emp3 Dec11 ...
-1
votes
3answers
86 views

replace multiple strings in a file in python3 [closed]

I have test file where i have some entries like below and i need to change memberUid to nisNetgroupTriple along with the names $ cat testfile2 memberUid: tony memberUid: jacob memberUid: zenny ...
1
vote
2answers
182 views

Awk or Sed: If pattern is found: delete line && export variable

In my bashscript I want Sed to find debug: Debug in a file ${SourceP} and if found delete that line and export to the environment Debug=Debug sed -i '/debug: Debug/,+0 d' ${SourceP} && export ...
-3
votes
1answer
107 views

Extract data between two patterns from a huge (forced) text file

I have filename.json. If I parse it in terminal with file filename.json output is: filename.json: UTF-8 Unicode text, with very long lines wc -l filename.json 1 filename.json If I parse it ...
0
votes
1answer
50 views

count number of lines provided to Python script before and after task

Input: {"case":"0901","emailed":"yes","vote":1,"accepted":"no"}, {"case":"0908","emailed":"yes","vote":8,1"accepted":"yes"}, {"case":"0911","emailed":"no","vote":10,1"accepted":"yes"}, {"case":"0090",...
1
vote
1answer
2k views

passing python variable into shell script

Here is my script: cmd=(''' ssh $servername /usr/local/bin/pstat $1|awk 'FNR==5 {print $9}' %s ''' %int(var)) p=subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ...
0
votes
2answers
143 views

Compare one column from one file with all columns in another file

I have two files. File 1 has a pattern in the form of a single column that I want to compare with all columns in file 2 to ultimately count how many columns in file 2 show that pattern. The number of ...
5
votes
1answer
109 views

The right tool to reverse the sorting order of thousands of elements in HTML file [closed]

I have a HTML file containing thousands of <div class='date'></div><ul>...</ul> blocks of code like below: <!DOCTYPE html> <html> <head> </head&...
1
vote
2answers
36 views

sed delete all after the first period followed by numbers inclusive

.bill.sally.foo.bar.1.3.21.3.43.nupkg I want to remove all after the first period (inclusive) followed by numbers so output is: .bill.sally.foo.bar This will be embedded in a python script via ...
-1
votes
3answers
50 views

sed or awk reformatting a line by copying part of it to the end of the line

I have a CSV file I wish to edit before importing it into a SQLITE DB. It has thousands of lines in it and I want to copy part of the line and append it to the end with a pipe "|" so that it can ...
0
votes
1answer
140 views

Pattern recognition and summing columns between two csv/excel files

I have two files, in which one has data on each person's start date, increment, promotion and other details, and the other file has their research output over the years. Can a script be written in ...
-1
votes
1answer
745 views

reading output into variables inside the os.system command

I have a python script which runs in Linux. I need to capture a command's output and store into variables and then should print the output. Code below - #!/usr/bin/python import os, time systime=os....
4
votes
5answers
960 views

‌How to select rows with minimum value in each group based on first column as ID?

I have a file looks like this : 1 7.8e-12 1 7.8e-12 1 1.0e-11 2 9.3e-13 2 3.5e-12 2 3.5e-10 2 3.1e-9 3 3.0e-11 3 3.0e-11 3 1.7e-08 For every ...
2
votes
2answers
110 views

counting occurences of pattern from an input file to match against a huge file

I have a list of colleges in a text file, and in a separate file I have a list of publications with affiliation. I want to write a script, to check, how many times a publication is repeated and to ...
0
votes
2answers
363 views

Print file line-by-line, but handle the file changing mid-execution

The following bash command reads the input file then prints each line with a 15 second pause between them: IFS=$'\n';for line in $(cat file.txt | tail -n +2); do echo $line && sleep 15; done; ...
2
votes
3answers
442 views

Customized Sort and remove double quotes join by underscore of multiple columns from tab delimited file

Custom Sort based on subject column order should be Maths, English, Science when I use this below command awk -F',' '{if (NR!=1) {print $2,$3,$5,$4}}' myfile.csv on my myfile.csv I am getting ...
0
votes
1answer
42 views

Select every two rows if they begin with the same name

I have a table that looks like this: name something 1 100036498|F|0--20:T>G something 2 100036501|F|0--44:C>T something 3 ...
1
vote
1answer
431 views

pattern file as an input to RS,FS in awk/sed/grep to recognize and add columns

I have a file which has the patterns that I have to find in the given data set and add the numbers in a columns of the data. I want to write a script or use awk/sed/grep to automatically query the ...
0
votes
2answers
443 views

Find lines between two patterns and append lines with pattern

We have the following file where a bunch of records (million records) of interest are between the start and end of patterns. The start pattern always start with an A and the end pattern always starts ...
3
votes
4answers
197 views

Selecting lines in a file that do not contain the value in the other file

I have two files. One is a CSV of data with 60490 lines. Each line is a set of values such as a customer name, service dates, etc. One of these values in the first file is a VIN value. I have a ...
17
votes
5answers
2k views

How can I sort a list with major.minor.patch level and sometimes rc correctly?

I have to sort the following list with a shell script and make the latest version appear on the bottom or top. How would I do that with shell tools only? release-5.0.0.rc1 release-5.0.0.rc2 release-5....
0
votes
2answers
287 views

How to print the matrix part of a file [closed]

I have a file that has strings and float numbers. I want to load this file and then print the matrix part of the file to can access to the matrix elements. file.text laimfnllvbms ksdvkdv=ksmgfgmf ...
4
votes
3answers
852 views

How to print same lines multiple times with variables changed using Sed/Awk/ anything?

a. Have set of lines as given below. cr_v8_sel0 : cross cp_v8_en, cp_sel0 { ignore_bins ig_v8_sel0 = binsof(cp_v8_en) && binsof(cp_sel0) intersect {[0:3],[5:$]}; } b. I wanted to print ...
-1
votes
3answers
69 views

Insert array of years into a list of URLS

I have a list of URLS like the example below. /test/test I also have an array of years [1999, 2000, 2001]. I am trying to find a way to automate the task of printing out a list of URLS with the ...
0
votes
2answers
34 views

placing words corresponding to a key in a singe row

I have a symbol table of the form (key-value pairs) 01 aaluu desii 01 aaluu cipsa 01 amaruuda ilaahabaadii 02 hai 02 nahii 02 txamaatxara To start with, I am supposed to place words(...
33
votes
5answers
70k views

How do you list number of lines of every file in a directory in human readable format.

I have a list of directories and subdirectories that contain large csv files. There are about 500 million lines in these files, each is a record. I would like to know How many lines are in each ...
0
votes
2answers
68 views

Data arrangement

I have data which i would like get output in below format using shell/perl script. Data: Virtual Machine Test status Running. Assigned Server OVS001.local Virtual Disk Test_DISK1 (30) size 61GB ...
0
votes
1answer
523 views

Python output in XML format

I have a python script which compare IP list with IP Database and gives output of matching criteria. (I got the script from this forum). convert.py #!/usr/bin/python import socket,struct ...
1
vote
1answer
39 views

calculate numbers on end and merge to one line

So in a file i have lines like these: thisdata:thisother:1337 thisdata:thisother:1800 thisdata:thisother:1500 thisdata:thisother:46984 I would like to output: thisdata:thisother:51621 for all ...
2
votes
1answer
101 views

how can I add an extra character after a word search [closed]

I have the following field names demelog voyapro naisjdf naismc decoide decoccm travide travccm equiccm mariccm Below is a sample of my file. There are more columns before the first fck=83... fck=...
14
votes
4answers
19k views

Replace multiline string in files

I have a number of files I want to update by replacing one multi-line string with another multi-line string. Something along the lines of: * Some text, * something else * another thing And I want ...
1
vote
1answer
724 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
1
vote
4answers
2k views

Pattern Matching and Delete the whole line

I want to delete all the Lines of File 1, if Column1 of File1 matches exactly with Column 1 File2. File 1: r001:21:10 21 AAAAAATTTGC * = XM:21 r002:21:10 21 YAAAATTTGC * ...
2
votes
1answer
1k views

Safely convert unicode strings to printable characters

I have many HTML files containing mixed unicode strings like \303\243 and printable characters like %s. What I'd like to do is converting the unicode strings into printable characters in a safe way. ...
4
votes
6answers
974 views

What is the easiest way to check if column A and B values goes both ways with AWK? [closed]

What is the easiest way to check if column A and B values goes both ways? Output to check: Mike John John Mike Pamela Barbara Barbara Pamela Mike Paul Roger Paul Desired output Mike <-> John ...
3
votes
2answers
740 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given input....
3
votes
5answers
619 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...