Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

4
votes
3answers
135 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
2
votes
3answers
73 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
2
votes
3answers
201 views

Text File Renaming Using Script

I have thousands of text files inside a folder and they are named in a certain way. For example: Image_234_Data_7778_n0.txt Image_234_Data_7778_n1.txt Image_234_Data_7778_n2.txt ...
1
vote
2answers
51 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
1
vote
2answers
51 views

Some characters disappear after processing a text file

I've a file named addresses.txt : address1@domain.com address2@domain.com address3@domain.com I'd like to echo "address1@domain.com.html" using for loop. Here's what I did : cat addresses.txt | ...
1
vote
2answers
116 views

Split out data from a column

So at the moment I have some input email addresses and would like to separate the alias and the domain, then copy them to the end of the line. For example, from a line that has ...
0
votes
2answers
54 views

How to match and replace an attribute value inside a tag

I am trying to replace the value of following node using sed command. <KeyValue key="EMPTY_SEARCH_CRITERIA" type="String" value="CS_CODE" comment="Configured empty search criteria."/> I am ...
0
votes
2answers
38 views

Select lines and format

Is there a way to format the following output so that only lines 1 and the respective lines that state the "rule" name print out? Depending on the configuation, the "rule" line could vary. It could be ...
0
votes
2answers
410 views

Formatting quotes from the fortune to use it with conky

I'm using fortune cookie outputs in my conky script. My fortune output is in the following format: Text1 : Some message including( line break tags, e.g. and comma(,),=,semicolon(;) like ...
1
vote
1answer
45 views

Specify the type of field to modify in a file

I have a list of clients, "code", "name", "location", "etc" and I want to make a function that asks the employee to give the client's code and then the field he/she wants to change. For example, ...
1
vote
1answer
86 views

Shell script runs diffrently based on working directory.

I have a script which runs differently based on current working directory. Basically it processes an .ini file and takes parameters from it and runs mysql backup. Script is running as root user. ...
1
vote
1answer
49 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
1
vote
1answer
239 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
1
vote
1answer
81 views

Using “ps -C java -o pid,user,cmd” Can I reduce the output of cmd?

I am trying to get this command down to just the essential information that I need. In the command cmd comes back with this output... /usr/local/jre1.7.0_51/bin/java -Dwd.tag=boxect ...
1
vote
1answer
109 views

How can I compare 2 files and add lines that aren't on the second file?

I have run into a problem while doing some coding, and I need to compare two files, here is an example input file inputfile1.txt: car php module so js node array and another input file: ...
1
vote
1answer
47 views

convert row into column based on particular key

I have file like Below. But header are in different line the header Values are printed in down to the hear. I want all the header in single row & value to the Particular header to be print to ...
1
vote
1answer
104 views

Process Files to create a new file

I have a file1 which contains comma separated values (timein/timeout in military time format eg 0800, 0900, 1300). For each line of File 1 contains timein/timeout for each day. Sample File 1: Name, ...
1
vote
1answer
53 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
1
vote
1answer
248 views

Fastest way of find and replace in large file with replacing filed present in another file

I have two files. File1 is a csv with 60 fields: 111,Check1|^/h1/h2/h3,22062014184500,20,0..... 111,Check2|^/h43/h40/h9,22062014184500,4,..... 111,Check3|^/h1/h3/h4,22062014184500,0,0,..... File2 ...
1
vote
1answer
198 views

Insert a line after a block of text with sed

I have text file: file.conf ### Option: LogFile # Name of log file. # If not set, syslog is used. # # Mandatory: no # Default: # LogFile= ### Option: LogFileSize # Maximum size of ...
0
votes
1answer
58 views

Delete customers domain extension only (in .csv file)

I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command: sed -i '/.br/d' customers.csv But that would also delete a ...
0
votes
1answer
64 views

How can I regex specific links from rss content in yahoo pipes?

From a RSS content I'd like to remove everything except specified href links Example: <div class='text'> <div class="center"> <b><a rel="nofollow" target="_blank" ...
0
votes
1answer
61 views

Search a file for a string in multiple directories

Can someone please help me with a command to search a file for a string in multiple directories? I am searching for the VIP in httpd.conf in multiple httpd instances. I am using: find ./ -name ...
0
votes
1answer
89 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
0
votes
1answer
53 views

Parsing a txt file according to two different tags

I need to parse a txt file like below. The part that needs to be parsed starts with "SASN2010Aber.CallEventRecord.egsnPDPRecord" and text is between two parenthesis {...}. The key text length is not ...
0
votes
1answer
228 views

Loop for's result in a variable for sql query

I have this part of my script for i in `cat ${RELLIST}` do for mr in `cat ${i}/DBA/install.list` do grep -i "'dba_" ${i}/DBA/sql/${mr}/*.tab 2> /dev/null | sed 's/^.*://' #grep -i ...
0
votes
1answer
276 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
0
votes
1answer
91 views

Extracting info from blocks inside text file

I have a text file which has blocks like dir1/dir2/dir3/name_run_number1: line1_run_number1_part1 line2_run_number1_part2 line3_run_number1_part3... Each block is separated with a blank line and ...
0
votes
1answer
47 views

Macbook System Profiler formatted

I want to format the output generated from macbook terminal command system_profiler and transpose the row values to columns under the same Application. Here's a snippet of the file: Firefox: ...
0
votes
1answer
72 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
0
votes
1answer
50 views

How can I write a script which updates a text file?

I have made this script #PACKAGES.TXT array2=( `find . -type f -iname *t?z|awk '{print $1}'|cut -d / -f 3` ) funzioneZ() { echo -e " PACKAGE NAME: $i " } array3=( `find . -type f -iname *t?z|awk ...
0
votes
1answer
423 views

Read text file and find unique records

I have a large text file with columns A, B, C, and D. How do I read the unique values for columns C and D and write those unique rows to a regular text file or in JSON format?
0
votes
1answer
72 views

Remove given parts of a string with sed?

$ echo '<a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a>' <a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a> $ echo '<a ...
0
votes
1answer
46 views

Python: Move lines backward in a textfile

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_FIRST__ 04 text text text 05 text text text 06 text text text 07 text text ...
-1
votes
1answer
763 views

How to print gmail inbox messages

I want to print inbox messages one by one (like cat someFile) so that I can parse the message contents. something like printGmail InboxMesg1 | grep some | sed some Using curl I can access the ...
-2
votes
1answer
59 views

Extract/print every 3rd occurence of pattern

I need help with extracting/printing lines until a stop pattern. This needs to happen after every 3rd occurrence of start pattern, i.e ERROR_FILE_NOT_FOUND and continue until the end of the file. ...
-4
votes
1answer
238 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
2
votes
0answers
144 views

Bash regular expression comparison fail for hex byte \x01

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly. Using GNU bash 4.1.5(1). Is this a bug, or is ...
2
votes
0answers
47 views

Make GNU indent break long comments

Is there a way to convince GNU indent to break long comment and change it to multiline comment? Something like this: // Very long comment, longer than 80 characters. Just imagine that. To this: /* ...
2
votes
0answers
117 views

Shell scripts for the database logs

The actual log file will be like this $ cat file1.log time=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, ...
1
vote
0answers
20 views

Converting a raw print data stream containing PCL commands to PDF

I have a program that sends raw print data through samba to typically PCL compatible printers. The program is written so basic PCL commands control how it prints. When sending it to the printer it ...
1
vote
0answers
61 views

Grepping multiple instances of a string, with context, from a file

I have a MySQL dump file, which is mostly INSERT statements. I want to grep out instances of the string 'media' from a field in a particular table, with +/- ~10 characters of context. The string might ...
1
vote
0answers
85 views

Sed regex printing into a file.txt

I'm connected to a MQTT stream, and I filter out the data by entering : mosquitto_sub -d -t +/# 2>&1 | sed -n "/PUBLISH/{s|.*\('.*',\).*|\1$(date),|;N;s/\n//;p}" into the Ubuntu terminal. ...
1
vote
0answers
911 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
0
votes
0answers
16 views

Searching text data in Hebrew, Arabic or Aramaic

How can I search for a (sub-)string in a database that has several languages, including Hebrew? I am assuming that the text being right-to-left is a question of the presentation, but the logical ...
0
votes
0answers
47 views

Compare specific word counts without variables or expr

I am tasked to write bash command to count number of specific words (given in file) in 2 files and write an error message if number of these words in first file is more than 3 times biggen than in the ...
0
votes
0answers
79 views

Bash - Convert escape character colored text to BBCode

I want to convert a terminal escape-character colored text file to BBCode colored text. For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color ...
0
votes
0answers
36 views

Shell Script for tracking number of times each command executes

How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
0
votes
0answers
222 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
0
votes
0answers
118 views

sybase and iconv

I inherited an old Ubuntu 6.10 server running php 5.1.6 which is using sybase to connect to MSSQL. A user submitted some text containing an en-dash and crashed one of my script. It turns out sybase ...