Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
4answers
40 views

Filter Require Capital words from a file( not all capital words )

I want Output of this file only AVDDPLL1V8 AGNDPLL1V8 DVDDPLL1V1 DGNDPLL1V1 Please help me in this to find solution. Here is my input: 6.1.2 Power and Ground Pins The following ...
1
vote
1answer
12 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
47
votes
10answers
17k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
0
votes
1answer
32 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
0
votes
5answers
69 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [on hold]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
-1
votes
1answer
42 views

To append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append

I have four CSV files as examples given below; I want to append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append. file1: ...
2
votes
4answers
82 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
9
votes
3answers
517 views

Why does “wc -c” print a value of one more with echo?

When running echo abcd | wc -c it returns 5. But the word abcd is only 4 characters long. Is echo printing some special character after the word abcd? And can I prevent echo from printing that? ...
0
votes
2answers
43 views

How do I validate a settings(ini) file for the values expected using bash?

After I check if the ini file exists where it is expected to be, which I've learned how to do, I would like to check the file for proper formatting. I'd like to validate any number of "site" entries ...
0
votes
7answers
1k views

Unix sed command to find multiple string patterns

Scenario here is I need to replace "$$file_name=" value.. I have some 150 file names in a file as mentioned below.. Eg: $$file_name=abc.bbb.ccc to_be ---> $$file_name=abc.bbb.cccV1 AS_IS : ...
1
vote
2answers
91 views

Editing INI-like files with a script

I'm writing a script to automate setting up Puppet agent configuration files in Docker. Basically, I need to ensure that the following section is in /etc/puppet/puppet.conf: [agent] ...
2
votes
1answer
26 views

Command to print few consecutive lines from middle of a file [duplicate]

For a file containing 20 lines, lines 6-10 can be printed using following command: head -10 filename | tail -5 Can this exactly same thing be done without using 'head' and 'tail' commands ?? ...
1
vote
3answers
39 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
2
votes
2answers
22 views

How to escape a Whole Variable?

In the answer of thisquestion: https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters It said that we have to put an arg of a function in ...
14
votes
3answers
9k views

How can I find out how many lines a text file contains without viewing it?

How can I find how many lines a text file contains without opening the file in an editor or a viewer application? Is there a handy Unix console command to see the number?
-3
votes
0answers
46 views

please let me know where to attach the files here? [closed]

Below is the requirement. Ignore lines containing pattern "SET SNAP_ID" or "Shadow Copy Components:\" . Ignore lines with column 2 till 8 is empty. print lines containing pattern "Failed". ...
3
votes
1answer
54 views

Tokenize string from $REPLY in bash script

This is my first post, and I have no idea how I managed anything before StackExchage, Google, Wiki, GNU, Internet, the list goes on :) I am trying to find a way to construct a SQL database generator ...
3
votes
3answers
47 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
3
votes
2answers
30 views

Extract text between two specific lines

How can I get the part of the output of a command between two specific lines? A dummy example: Command: git for-each-ref --sort='*authordate' --format='%(tag)' refs/tags | grep -v '^$' Output: ...
1
vote
3answers
121 views

Joining files on disk

Say I have several file with the following naming: 000001 000002 000003 ... Each of these files is a csv file (may include escape characters). In total the folder has ~20GB of data. How can I ...
6
votes
6answers
220 views

Obtain values of a section of a file

I have a configuration file in the following format. <Title> [part1] A.File = a A.Val = val1 B.File = a B.Val = val1 [part2] A.File = a1 A.Val = val2 B.File = a B.Val = val1 ...
2
votes
4answers
172 views

Iteratively edit a single file

I have a single file with a statement that looks like this : myName_tx_1 VARCHAR(255) myName_in_1 VARCHAR(255) myName_tx_2 VARCHAR(255) myName_in_2 VARCHAR(255) myName_tx_3 VARCHAR(255) myName_in_3 ...
0
votes
1answer
42 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
2
votes
6answers
425 views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
4
votes
5answers
74 views

Insert Lines after Multi-Line Regex Match

I've got an XML file I need to append to after a specific set of tags. Basically the file structure looks like this: <key>servers</key> <dict> ... Server Details ... </dict> ...
5
votes
2answers
489 views

How to append multiple lines to a file without last newline?

I have a very long string that is split in chunks. I want to append them to a file without putting the newline character using bash. Example: First append cat >> abc.sh << EOL echo "bla ...
1
vote
5answers
63 views

How to overwrite stdout using echo? [closed]

We know to overwrite bar.txt, we can use this: echo "foo" > bar.txt but how can we overwrite to the standard output? I tried this command: echo "foo" > but it failed. I want this because ...
3
votes
2answers
117 views

Remove duplicate fields in a given column (after ID check)

I'd like to remove from a given column ($2 in the example) the duplicate fields (comma separated), but only if the ID column ($1 in the example) is the same. Input: A 1,2,3,4 A 8,9,10,11 A ...
5
votes
4answers
425 views

How can I “lazily” read output from xrandr?

I have a bash script that I use to adjust my monitor brightness that uses xrandr --verbose to get the current brightness. It works fine, but using xrandr is kind of slow on my machine, as you can see ...
0
votes
4answers
65 views

How print only the current logged in “root” user IP address?

Currently I can see my (root) IP address on top of the output for this command: last But what I want is just that the IP address is printed out to me in a clean way. root pts/1 1.1.1.1 ...
2
votes
1answer
28 views

Calculate Apache's response time for last 10 seconds

I have the next apache's log format: LogFormat "%h %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" where's %D - The time taken to serve the request, in microseconds. I want to ...
1
vote
2answers
26 views

Transform custom tags with arguments

I want to write my diploma theses with markdown. Sadly, there is no perfect editor and so I have to improvise. For bibtex citations I want to transform my own definition with some sort of bash-filter ...
-3
votes
0answers
24 views

Parsing the log file test into tabular format

How to display below test file in tabular format as below: Output should be like: ma1-test1.cor.dns.com cmd - ...
4
votes
2answers
741 views

How to display TSV (csv) in console, when empty cells are missed by: `column -t -s $'\t' `

I have file with columns spearated with tab. I have file when some rows have empty cells (on begining, in middle). In such cases column -t -s $'\t' simply fails: Input: $ echo -e ...
8
votes
4answers
3k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
3
votes
3answers
118 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
-2
votes
3answers
59 views

How to sed two same value without affecting the other

I have an xml file and I want to change one value on that file: param name=swam_license_ports_overdraft_thresh 99 param name="strike_license_ports_overdraft_thresh" 99 The outcome I want is to ...
2
votes
2answers
27 views

removing each . after ] appear in a line in complete text file

I have a file in which I need to eliminate . after the ] on every line e.g xyz[1.2]35.23 xyz[1.2.3]35.234 output should be xyz[1.2]3523 xyz[1.2.3]35234
3
votes
4answers
471 views

Delete a block of lines with a condition on the last line

I have a log-file where at the end of a series of lines you can see if this block is relevant. Now I'm looking for a command like sed to delete de blocks ending with "Content-Length: 0" and beginning ...
2
votes
2answers
209 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...
0
votes
2answers
4k views

Replacing one test string with another in particular column

I would like to replace "CC" with "C" and "AA" with A" in a particular column of Tab delimited file (using awk probably).
5
votes
4answers
210 views

How to replace all the first occurences in matched lines with incremental number?

I have a file like this ... 1562 first part 1563 H col3 H col4 1564 H col3 H col4 ... 3241 H col3 H col4 3242 third part ... I want to replace only the first H in every line to H#, with ...
0
votes
5answers
67 views

More concise methods for wrapping a file than using sed

Introduction script: sed -i '1i <?xml version=\"1.0\" encoding=\"utf-8\"?>' $1 sed -i '/<?xml version=\"1.0\" encoding=\"utf-8\"?>/a<hello>\n\t<world>' $1 sed -i ...
2
votes
1answer
23 views

Get each column which contains multiple spaces?

My file: subnet 172.25.10.0 netmask 255.255.255.0 { option domain-name-servers 172.25.10.9, 8.8.8.8; default-lease-time 300; max-lease-time 7500; range dynamic-bootp 172.25.10.10 172.25.10.30 ; ...
0
votes
1answer
22 views

Compare strings

I have 2 txt files, in one i have alias lists and the other is the servers name. So what im trying to do is to loop from the alias txt file line by line and where it will find the name that is on the ...
2
votes
1answer
62 views

How to convert .doc to ODF with a command line?

I'd like to convert many Microsoft Word documents to ODF (the OO.org formata with a command line (because there are many of them). (I need them for reading afterwards on a device where reading OO.org ...
5
votes
4answers
73 views

Search a pattern and print preceding lines starting with another pattern

I need to grep for a pattern and once I find it, I need to look for another pattern preceding that and print all those lines in between those. Better example: 1 a 2 a 3 a 4 a 5 a 6 b 7 c 8 d 9 xyz ...
0
votes
1answer
78 views

Parse lines of output from bash loop

I'm trying to parse the output of commands run in a bash loop. Here is an example: $ for i in `git log --format='%H'`; do echo $i ; git branch --contains $i; done | head -n 8 ...
3
votes
2answers
95 views

Identify lines that are out of order

I have a process that generates output mostly in lexicographically sorted order according to a (timestamp) field, but occasionally the lines will be output in the wrong order: 2014-08-14 ...
4
votes
6answers
151 views

grep with a lot of files which contains a lot of lines?

Imagine we have a lot of huge(30000 line)files: a.2014-05-06, a.2014-05-07, a.2014-05-08 and so on. I know that we can grep a line with word using this command: grep "word" a.* First I guess this ...