Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
2answers
69 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
1
vote
2answers
61 views

Some characters disappear after processing a text file

I've a file named addresses.txt : address1@domain.com address2@domain.com address3@domain.com I'd like to echo "address1@domain.com.html" using for loop. Here's what I did : cat addresses.txt | ...
1
vote
2answers
119 views

Split out data from a column

So at the moment I have some input email addresses and would like to separate the alias and the domain, then copy them to the end of the line. For example, from a line that has ...
0
votes
2answers
49 views

Grepping from a parameter file for non matching text

I am new to shell scripting. I have a paramameter file and I'm trying to grip a word and then remove only that word leaving everyting elese. For instance: An Example would be: #Parameter File ...
1
vote
1answer
32 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: ...
1
vote
1answer
51 views

Specify the type of field to modify in a file

I have a list of clients, "code", "name", "location", "etc" and I want to make a function that asks the employee to give the client's code and then the field he/she wants to change. For example, ...
1
vote
1answer
94 views

Shell script runs diffrently based on working directory.

I have a script which runs differently based on current working directory. Basically it processes an .ini file and takes parameters from it and runs mysql backup. Script is running as root user. ...
1
vote
1answer
57 views

delete non important characters from command output

the version command in my Linux machines display the following output version Red.Hat.OS. release..........5.5 APP.Release.............4.2.0.0-02 how to remove the middle "." dot character ...
1
vote
1answer
87 views

Using “ps -C java -o pid,user,cmd” Can I reduce the output of cmd?

I am trying to get this command down to just the essential information that I need. In the command cmd comes back with this output... /usr/local/jre1.7.0_51/bin/java -Dwd.tag=boxect ...
1
vote
1answer
135 views

How can I compare 2 files and add lines that aren't on the second file?

I have run into a problem while doing some coding, and I need to compare two files, here is an example input file inputfile1.txt: car php module so js node array and another input file: ...
1
vote
1answer
47 views

convert row into column based on particular key

I have file like Below. But header are in different line the header Values are printed in down to the hear. I want all the header in single row & value to the Particular header to be print to ...
1
vote
1answer
113 views

Process Files to create a new file

I have a file1 which contains comma separated values (timein/timeout in military time format eg 0800, 0900, 1300). For each line of File 1 contains timein/timeout for each day. Sample File 1: Name, ...
1
vote
1answer
55 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
1
vote
1answer
291 views

Fastest way of find and replace in large file with replacing filed present in another file

I have two files. File1 is a csv with 60 fields: 111,Check1|^/h1/h2/h3,22062014184500,20,0..... 111,Check2|^/h43/h40/h9,22062014184500,4,..... 111,Check3|^/h1/h3/h4,22062014184500,0,0,..... File2 ...
1
vote
1answer
228 views

Insert a line after a block of text with sed

I have text file: file.conf ### Option: LogFile # Name of log file. # If not set, syslog is used. # # Mandatory: no # Default: # LogFile= ### Option: LogFileSize # Maximum size of ...
0
votes
1answer
13 views

different combination of data in a file

I have a very huge file containing numbers only. file - 123212 234234 12324 1243223 5453443 I need output like below 123212,234234 123212,12324 123212,1243223 123212,5453443 234234,123212 ...
0
votes
1answer
72 views

Delete customers domain extension only (in .csv file)

I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command: sed -i '/.br/d' customers.csv But that would also delete a ...
0
votes
1answer
68 views

How can I regex specific links from rss content in yahoo pipes?

From a RSS content I'd like to remove everything except specified href links Example: <div class='text'> <div class="center"> <b><a rel="nofollow" target="_blank" ...
0
votes
1answer
76 views

Search a file for a string in multiple directories

Can someone please help me with a command to search a file for a string in multiple directories? I am searching for the VIP in httpd.conf in multiple httpd instances. I am using: find ./ -name ...
0
votes
1answer
146 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
0
votes
1answer
56 views

Parsing a txt file according to two different tags

I need to parse a txt file like below. The part that needs to be parsed starts with "SASN2010Aber.CallEventRecord.egsnPDPRecord" and text is between two parenthesis {...}. The key text length is not ...
0
votes
1answer
311 views

Loop for's result in a variable for sql query

I have this part of my script for i in `cat ${RELLIST}` do for mr in `cat ${i}/DBA/install.list` do grep -i "'dba_" ${i}/DBA/sql/${mr}/*.tab 2> /dev/null | sed 's/^.*://' #grep -i ...
0
votes
1answer
480 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
0
votes
1answer
104 views

Extracting info from blocks inside text file

I have a text file which has blocks like dir1/dir2/dir3/name_run_number1: line1_run_number1_part1 line2_run_number1_part2 line3_run_number1_part3... Each block is separated with a blank line and ...
0
votes
1answer
47 views

Macbook System Profiler formatted

I want to format the output generated from macbook terminal command system_profiler and transpose the row values to columns under the same Application. Here's a snippet of the file: Firefox: ...
0
votes
1answer
82 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
0
votes
1answer
51 views

How can I write a script which updates a text file?

I have made this script #PACKAGES.TXT array2=( `find . -type f -iname *t?z|awk '{print $1}'|cut -d / -f 3` ) funzioneZ() { echo -e " PACKAGE NAME: $i " } array3=( `find . -type f -iname *t?z|awk ...
0
votes
1answer
525 views

Read text file and find unique records

I have a large text file with columns A, B, C, and D. How do I read the unique values for columns C and D and write those unique rows to a regular text file or in JSON format?
0
votes
1answer
85 views

Remove given parts of a string with sed?

$ echo '<a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a>' <a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a> $ echo '<a ...
0
votes
1answer
47 views

Python: Move lines backward in a textfile

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_FIRST__ 04 text text text 05 text text text 06 text text text 07 text text ...
-1
votes
1answer
824 views

How to print gmail inbox messages

I want to print inbox messages one by one (like cat someFile) so that I can parse the message contents. something like printGmail InboxMesg1 | grep some | sed some Using curl I can access the ...
-2
votes
1answer
61 views

Extract/print every 3rd occurence of pattern

I need help with extracting/printing lines until a stop pattern. This needs to happen after every 3rd occurrence of start pattern, i.e ERROR_FILE_NOT_FOUND and continue until the end of the file. ...
-4
votes
1answer
306 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
2
votes
0answers
35 views

Converting a raw print data stream containing PCL commands to PDF

I have a program that sends raw print data through samba to typically PCL compatible printers. The program is written so basic PCL commands control how it prints. When sending it to the printer it ...
2
votes
0answers
218 views

Bash regular expression comparison fail for hex byte \x01

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly. Using GNU bash 4.1.5(1). Is this a bug, or is ...
2
votes
0answers
123 views

Bash - Convert escape character colored text to BBCode

I want to convert a terminal escape-character colored text file to BBCode colored text. For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color ...
2
votes
0answers
49 views

Make GNU indent break long comments

Is there a way to convince GNU indent to break long comment and change it to multiline comment? Something like this: // Very long comment, longer than 80 characters. Just imagine that. To this: /* ...
2
votes
0answers
138 views

Shell scripts for the database logs

The actual log file will be like this $ cat file1.log time=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, ...
1
vote
0answers
60 views

Convert / import flat file with multiple records, one variable per line

I am searching for an efficient way using (UNIX) command line tools to read a flat file into a(ny) database, perhaps after first converting to an intermediate structured format, for example XML or ...
1
vote
0answers
92 views

Grepping multiple instances of a string, with context, from a file

I have a MySQL dump file, which is mostly INSERT statements. I want to grep out instances of the string 'media' from a field in a particular table, with +/- ~10 characters of context. The string might ...
1
vote
0answers
91 views

Sed regex printing into a file.txt

I'm connected to a MQTT stream, and I filter out the data by entering : mosquitto_sub -d -t +/# 2>&1 | sed -n "/PUBLISH/{s|.*\('.*',\).*|\1$(date),|;N;s/\n//;p}" into the Ubuntu terminal. ...
1
vote
0answers
1k views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
0
votes
0answers
27 views

Excel - Keep data on field that starts with

I have an excel file with the following data example 1 E Eastland Care Ctr pbrotherton@voa.org 2425 Kimberly Pkwy E Columbus OH 43232-4271 2072822617 6148689306 805101 Nursing & ...
0
votes
0answers
39 views

Drawing a histogram from a bash command output

I have the following output from a miRNA Predictor: Read Sequence:hsa-miR-576-3p MIMAT0004796 (22 nt) Read Sequence:gi|182396|gb|M74088.1|HUMFAPAPC Human APC gene mRNA, complete cds (8972 nt) ...
0
votes
0answers
21 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
0
votes
0answers
20 views

Searching text data in Hebrew, Arabic or Aramaic

How can I search for a (sub-)string in a database that has several languages, including Hebrew? I am assuming that the text being right-to-left is a question of the presentation, but the logical ...
0
votes
0answers
54 views

Compare specific word counts without variables or expr

I am tasked to write bash command to count number of specific words (given in file) in 2 files and write an error message if number of these words in first file is more than 3 times biggen than in the ...
0
votes
0answers
41 views

Shell Script for tracking number of times each command executes

How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
0
votes
0answers
256 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
0
votes
0answers
120 views

sybase and iconv

I inherited an old Ubuntu 6.10 server running php 5.1.6 which is using sybase to connect to MSSQL. A user submitted some text containing an en-dash and crashed one of my script. It turns out sybase ...