Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
2answers
32 views

Can I convert fields in XML to tags using xmlstarlet?

I want to convert fields in Tags to tags within that tag for example <book name="Data Structure" price="250" pages="350"/> To <book name="Data Structure"> ...
4
votes
2answers
42 views

Replace field in one file based on match with a different field in another file

appreciate any help with this task: File A has 4 fields and has 90K lines. The first field (column 1) has a value that needs to be changed if criteria below met. The fourth field (column 4) has the ...
1
vote
3answers
38 views

Remove words (letters followed by space) from a specific column

I have a file and the format is as follows: cat dog AHF123432 | 123432 | dhfshfjdh lion AFG23412 |23412 | dfshjhfjdhj I need to remove those words from first column which contains only alphabets ...
0
votes
1answer
24 views

Timestamp comparison in awk

Considering below file, how can I compare two timestamps, noting that the time part is negligible. foo,boo,2038-01-18T12:00:00+02:00 foo,boo,2015-09-12T01:31:24+03:00 ...
4
votes
1answer
175 views

In AWK, is it possible to specify a range for operation?

Example: Given a comma separated file "foo" with 50 fields per line, I want to print $0 if $10 != the range from 1001 to 1010. I can do it using below code: awk -F, '{if($10!=1001 && ...
4
votes
2answers
93 views

Adding column based on matching of second column

I have two files. The first file has the following format: 10D0325 2465 0 0 -9 -9 10D0598 2567 0 0 -9 -9 10D0562 2673 0 0 -9 -9 10D0175 2457 0 0 -9 -9 10D0241 2209 0 0 -9 -9 10D0954 2312 0 0 -9 -9 ...
-2
votes
4answers
27 views

Exctract several lines from large text file [duplicate]

I have large text file. I need quickly pull a bunch of lines, say from #14600 to #14700, from this file, as a separate file. How it could be done?
1
vote
3answers
178 views

How to add an attribute in a tag in XML file by command-line?

I am trying to add field at the end of tag using sed script. Suppose I have a tag in XML file: <book name="Sed tutorial" price="250"/> Now I want to add field as Book_Width="A" after end of ...
0
votes
1answer
24 views

Transcribe a csv column in awk, accounting for empty fields

Let's say I have a csv file with several columns. NF is not constant. At the moment, to replace columns > 4 with lowercase, I'm doing: #!/usr/bin/awk #call with `awk -f lowercase.awk in.csv > ...
1
vote
2answers
41 views

join two file based on column when there is no one by one corespondness in bash script (awk, grep , sed)

file1.txt 112|9305|/inst.exe 112|9305|/lkj.exe 112|9305|/dje.jar 112|9305|/ind.pdf 112|9306|/ma.exe 112|9306|/ngg.pdf 112|9307|/jhhh.dat 112|9312|/ee.dat 112|9312|/qwq.dll file2.txt ...
0
votes
1answer
49 views

What is the correct way of inserting \u using sed?

I'm trying to find a string that contains \u and adding something to it. For example, $ echo \user | sed 's/\us/\\usX/' - correctly produces \usXer However, if I have the contents in a file and ...
2
votes
3answers
48 views

merge two different files having different line number of lines

I have two huge files and number of lines in both of the file is not same. file1 61346877 41724134 85406965 59647779 25199749 86213 45417131 41905714 19415458 1828594 56543876 70603415 File2 212 ...
3
votes
3answers
116 views

Print man pages with fixed width

With the example command man apropos > outputfile a text file is generated which contains the formatted man page of apropos (with some little differences with respect to man apropos directly ...
0
votes
1answer
43 views

Array with two awk input files

I have two files, one with NF=7 and the other has NF=47 I need to check the number of occurrence of $2 of file with NF=7 if the occurrence is <=2 then I need to make an array from the same file ...
4
votes
1answer
55 views

Recursively grouping symlinks by original file

Pardon me, but I don't actually know where to begin with this - I will update if I get further. Sorry. Basically, I've got a list in a text file similar in structure to the following: ...
2
votes
2answers
65 views

Delimiter being changed with fields

I have a file with values organized in columns, and separated by commas, as below file: 324,01,1,113333600000,1,, 016,01,1,134954200000,1,, 770,01,1,109069200000,1,, 853,01,1,111518800000,1,, When ...
0
votes
2answers
25 views

Total value for a field?

Considering below file: 0,2,,,10 0,2,,,15 0,1,,,984 0,2,,,9 1,14,,,5 Using awk, I need to calculate the total value in $5 per each $2. The desired output would look like below: 2,34 1,984 14,5 ...
-1
votes
2answers
100 views

How to remove a column or multiple columns from file using shell command?

My file, ARCHIVE B1_NAME B2_NAME B3_NAME ELEMENT INFO_NAM WERT PROCID -------- -------- -------- -------- -------- -------- ---- ------ ...
3
votes
3answers
88 views

Organize entries as a spreadsheet (csv) using awk

I have information for entries (N= 1000) saved on a computer in a delimited text file/per entry ( every entry has one text file). The information recorded in the same format as the following: ...
0
votes
0answers
21 views

To list only the file names in a present working directory using unix shell command [duplicate]

There are some links,directories and files in my path. I want to list only the file names eg. logfile1 redolog1 dumplog1 logfile2 lrwxrwxrwx spsy spec Jun 23 2343243 jpeg --> ~/home/s/jpegs ...
5
votes
4answers
153 views

How to find duplicate lines in a text file, while some may be commented out or have different tokens at the beginning?

I have a text file with lines that are a mixture like this: ### Comments # Comments 86.242.200.81 banana.domain.net # comment 86.242.200.3 orange.domain.net 31.28.225.81 ...
4
votes
5answers
190 views

Remove characters while naming files or folders

I have a text file with multiple rows. In every row I have the date in the following format: 12/2/2015 15/9/2013 08/3/2011 02/5/2005 . . . I want to create a folder for every line in this text file ...
0
votes
2answers
41 views

How can I calcalate the average of each row excluding the first value of each row?

I need your help to calculate the average over each row but without including the first value. Example: 1 12 13 14 15 16 17 18 2 21 22 23 24 25 26 27 28 29 ... -------- I want the ...
4
votes
2answers
66 views

Find if any line of a file is a subset of data in another file

I have a file file1.txt with the following data: MIME_HTML_ONLY SUSPICIOUS_RECIPS SORTED_RECIPS HFILTER_HELO_5 and another file file2.txt with lines as: HFILTER_FROM_BOUNCE FORGED_OUTLOOK_HTML ...
0
votes
0answers
27 views

Excel - Keep data on field that starts with

I have an excel file with the following data example 1 E Eastland Care Ctr pbrotherton@voa.org 2425 Kimberly Pkwy E Columbus OH 43232-4271 2072822617 6148689306 805101 Nursing & ...
1
vote
3answers
39 views

Scripting batch text mangling

I often find myself trying to do trivial text manipulations on quite large files. It seems like there ought to be a way to script this with one or other of the various Unix text mangling tools, but I ...
1
vote
4answers
45 views

Linux replace last character in a csv file to new string

I am new to shell scripting. I created a script that extracts data from a database and loads it to a csv file. I am using SunOS hstz1454 5.10 system. Data looks like this below: 124,Y,Y,Y,Y,S ...
2
votes
4answers
150 views

How to take the absolute value using awk?

If I have below two dates: 2015-09-12,2015-08-13 And I need to get the number of days between them, I will use the below code: awk -F'[-,]' '{print 360*($4-$1)+30*($5-$2)+($6-$3)}' The output ...
2
votes
6answers
264 views

How to Remove last comma of each line on csv using linux

I need to remove the last comma of each line. I have only one csv file, which looks like this: 98,N,N,N,N,S, 99,N,N,N,N,S, 101,Y,Y,Y,Y,S, I have the following script, but it is not working: for ...
3
votes
1answer
59 views

Shuffle two parallel text files

I have two sentence-aligned parallel corpora (text files) with about 50 mil words. (from the Europarl corpus -> parallel translation of legal documents). I'd now like to shuffle the lines of the two ...
0
votes
2answers
49 views

Grepping from a parameter file for non matching text

I am new to shell scripting. I have a paramameter file and I'm trying to grip a word and then remove only that word leaving everyting elese. For instance: An Example would be: #Parameter File ...
4
votes
2answers
192 views

grep string where next line does not contain string

I want to search all files inside a directory and its subdirectories for lines containing a certain string, but I want to exclude those results that contain a different certain string in the line ...
1
vote
2answers
46 views

Apply same field from different records to an operation.

Considering below file: 4,5,8,3,41,86,32, 0,0,0,0,38,0,8,8, 9,8,8,9,42,8,9,8, 1,1,7,1,11,2,5,1, How can I compare each $5 with the same field from the next record. Ex for the first two records: ...
1
vote
1answer
30 views

Get the filename that the record come from

I have a huge number of .gz files, which being uncompressed using zcat together. I am looking to get from where each record came from, as below example: File1: 1,2,3,4,5,6,7,NEW,9,10,11,12 ...
1
vote
1answer
32 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: ...
2
votes
1answer
94 views

Using portion of a field in awk

How to use portion of a field for awk script? The input file: 123456789,0,0,4,1,6,8,6 987654321,0,0,8,4,6,9,8 251335549,8,9,6,3,1,8,8 897465419,8,1,8,9,4,7,9 I need to select the 3rd character ...
1
vote
3answers
31 views

Using awk to merge 2 files

I would like to merge 2 files based on column 1, 2, 3. I have tried the awk command below but is not working. awk 'NR==FNR {h[$1FS$2FS$3]=$4; next}{k=$1FS$2FS$3; if (k in h) print $1,$2,$3,$4,h[k] ...
1
vote
1answer
33 views

How to add hostname, date, and fix disk size

I'm collecting disk usage data and want to add the date, time, and hostname to this command. How can I do it in this format: YYYY-MM-DD HH:MM:SS,HOSTNAME,DISK_USAGE_DATA Command I have so far: ...
1
vote
2answers
42 views

Extract columns of a string using grep

I need to extract from /proc/net/dev the second and tenth columns separated by spaces. In other words, the number of received and transmitted bytes. Is it possible to do this using grep and ...
3
votes
4answers
109 views

Concatenate table without header

I would like to concatenate several tables and just keeping the header in the first line (and thus not having a new header each time that a new table has been concatenate). Does anyone know how to do ...
21
votes
9answers
3k views

How to delete every second line from a file?

File: Data inserted into table. Total count 13 No error occurred Data inserted into table. Total count 45 No error occurred Data inserted into table. Total count 14 No error occurred Data inserted ...
-1
votes
1answer
44 views

How to add a word at the first of every lines? [duplicate]

File1: Data will be inserted on Thursday Messages has been sent at 6pm Office hours is from 9am to 6pm All the employees are present Output what I want: DMS Data will be inserted on ...
2
votes
1answer
71 views

Show nth occurrence of a record

Using awk, how to show the nth occurrence of a record and delete the other occurrence, for example" Input File: 79933,2 79933,4 79933,5 55251,1 49494,4 49494,2 49494,2 49494,1 in this example, ...
2
votes
5answers
150 views

How to sort file by character occurrences per line?

I'm quite new to Linux, and I've found quite a bit of useful information on how to do character counts in a file, but is there a way in Linux/terminal to sort a text file by the number of times a ...
1
vote
5answers
362 views

Check number of words in a file that contain a specific letter

Bash command to check number of words in a file that contain letter “a”
1
vote
2answers
34 views

Append mth and nth columns of a file with the columns of another file

I have 2 files having different number of columns and with huge number of rows: bash-3.00$ cat fileA a1,a2,a3,a4,a5,a6,a7,a8,a9 q,w,,,,y,u,, a,z,,,,q,n,, ......................... z,p,,,,w,e,, ...
2
votes
3answers
176 views

Remove the ^L (aka \f/FF/form-feed/page-break) character

How do I remove the ^L character and the blank lines that come after it on a unix file? I have tried the below and have been able to remove the VT and spaces but am failing to remove the ^L character ...
3
votes
3answers
158 views

If statement string comparison

I'm reading a csv file with 14 values per line. I want to convert the HEX value from 10th [f10] value to a DEC value. In order to do this if the 7th [f7] value must hold EnHr or EnSt or SpJb or Chem ...
1
vote
3answers
49 views

Adding a pattern to each field of the following line

I need some help here please. I have the following input: MX04A; DMX04A; DMX04A; LMX04A; LMX04A; -17.2; -15.3; -14.3; -13.6; -16.8; -15.4; -16.0; -15.3; LH36A; DLH36A; DLH36A; -11; -117.2; -11; ...
2
votes
5answers
74 views

bash expanding to same value instead of line by line value in sed command

I have a file containing numbers in it say numbers.txt as follows. 9374541632553 51243747879841 32030098896914 84654557358238 11000656847765 I want to convert it to the following format. ...