Tagged Questions

Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
2answers
17 views

Find and delete duplicate txt files

I have a folder with around 300 text files, is there any command that reads every every file indivdually and deletes the duplicate files? I mean the content and not the file names.
3
votes
4answers
87 views

How do I show the numeric character sequence of a string?

Is there a command either standalone or in vi or similar (can be gui) that will show me the numeric sequence of a string? Input: The cat hopped in a box. Output: T h e c a t h o p p e d ...
0
votes
0answers
10 views

Change encoding of a text file in-place: iconv alternative?

Apparently, iconv is unable to use the same file as input and output file. Other tools, e.g. sed, have an -i switch that allows "updating" a file in-place. (Under the hood, temporary files are ...
3
votes
3answers
94 views

Split the lines of a text file into separate files

I have a text file that looks something like this: foo bar zip rar tar I need to use a bash script on OSX to make a new text file after every new line like this: cat text1.txt foo cat text2.txt ...
2
votes
1answer
33 views

Extend grep to find a match after the first match

I have a long list of strings separated by newlines and each section separated by a string starting with ~ (there are no other ~ in the file). I want to extend grep to find the ~ text right below the ...
0
votes
0answers
26 views

The preferred/most-common ways/procedures to programmatically edit configuration files? [on hold]

Well, almost every daemon is configurable. Most distributions have also daemonized system stuff, such as networking. For every installation that I've done in my life, I have always edited these ...
0
votes
1answer
18 views

transpose columns to rows in a spreadsheet [duplicate]

I have a spreadsheet has one column as the following: column A 1 a 2 b 3 c 4 d 5 e I want to ...
3
votes
4answers
321 views

How do I remove every number that's surrounded by <>

I have tried this sed -i '' 's/[0-9]*<>/g' But it didn't work. Example file: <Number1> </Number8> output: <Number> </Number>
5
votes
2answers
326 views

How do i populate end tags using sed, awk or any other command?

I have a txt file that looks like this: <sss>ss< or <firstword>anotherword< I want it to look like this <sss>ss</sss> or ...
1
vote
1answer
45 views

How to grep the logs for a list of patterns and make a report out of it?

I have a C++ executable which scan some of my files and those files has some user id in it. After scanning is completed it generates below log file (abc.log) like this - INFO [0x7faff2b32a00] ...
1
vote
1answer
69 views

Process Files to create a new file

I have a file1 which contains comma separated values (timein/timeout in military time format eg 0800, 0900, 1300). For each line of File 1 contains timein/timeout for each day. Sample File 1: Name, ...
-5
votes
1answer
57 views

Extract multiple lines from text file [on hold]

Source : ! hostname RTR-1 ! boot-start-marker boot-end-marker ! logging buffered 64000 informational no logging console ! interface FastEthernet0/0 description LAN ip address 172.16.29.250 ...
2
votes
2answers
207 views

How can I remove lines matching foo but not matching bar?

Assume I want to remove the 2nd and 3rd, but not the 4th line in the following file (those which match foo but do not match bar), can I easily do this with e.g. sed? text foo some more text even more ...
0
votes
1answer
77 views

How to update files based on another file

I have 4 csv files and I want to update set of three based on values in the other one. The first 2 files (file_1 and file_2) contains names and their numerical ids. The third file (file_3) contains ...
3
votes
2answers
91 views

How to read needle part of the sed command from a file?

I want to replace a long part of a file with something else preferably by sed but want to read that long part from a file. I already found this: sed -e '/<TEXT1>/{r File1' -e 'd}' File2 from ...
1
vote
2answers
47 views

Extract and format data with `cut` and `awk`

I have the below file in txt: "Firstname","Lastname","Company" "Alex","Johnson","Alex Corp." "Baran","Atasoy","Gülerler Ltd." I would like to extract and format data by using awk and/or cut and ...
1
vote
1answer
33 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
1
vote
1answer
48 views

Convert XML to SQL INSERT statements using the command line

Is there a way to convert XML to SQL INSERT using the command line, here is examples: <something> <somthingelse>lol</somethingelse> </something> would be INSERT INTO ...
2
votes
4answers
49 views

Linux log file structure change, abnormal (?) sed behaviour

Ok, so I tried to get this 0x0000: 4500 0044 68f7 4000 4011 25c7 8083 d0bf E..Dh.@.@.%..... 0x0010: 8083 da24 85b1 15b3 0030 1e8b 3132 3033 ...$.....0..1203 0x0020: 3132 2e37 3836 3036 2c20 ...
6
votes
4answers
397 views

combine two text files with adding some separator between?

cat file1 file2 will combine two text files. But if I want to add some separator between, like a line or two of ********************************, do I have to open the first file, and add the line at ...
5
votes
1answer
33 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
4
votes
6answers
877 views

A command to print only last 3 characters of a string

I know that the cut command can print the first n characters of a string but how to select the last n characters? If I have a string with a variable number of characters, how can I print only the ...
0
votes
1answer
32 views

How can I write a script which updates a text file?

I have made this script #PACKAGES.TXT array2=( `find . -type f -iname *t?z|awk '{print $1}'|cut -d / -f 3` ) funzioneZ() { echo -e " PACKAGE NAME: $i " } array3=( `find . -type f -iname *t?z|awk ...
2
votes
2answers
55 views

How to exclude lines that has given columns?

There is an input file that has TAB delimited columns. We need to remove the lines which has "NA" for the fourth AND the eleventh column. Question: how can we do this in awk?
5
votes
4answers
219 views

Replace a string containing newline characters

With the bash shell, in a file with rows like the following ones first "line" <second>line and so on I would like to replace one or more occurrences of "line"\n<second> with other ...
3
votes
1answer
28 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
3
votes
3answers
162 views

Count records matching pattern with Awk

So I was told to do last > lastloggedin which creates a file that shows the classes last login since the last system reboot, and now I am asked to write an Awk script which is named myawk that ...
0
votes
1answer
44 views

how to delete specific characters in a txt file

Please i need some help! I have these lines in a txt file: 1 Q0 /home/nikol123/Downloads/Ergasia_1/Ergasia_1/metadata/13/120411.xml 1 1 Q0 ...
2
votes
2answers
32 views

grep command to display only the two words searched for separated by a comma?

Say I have a file with the following text: Hello there, How are you? How can I grep for Hello and are such that the output of grep looks like: Hello,are
-2
votes
0answers
15 views

Uniqe values in a table [duplicate]

I have asked this question before but later on I realized that the answer provided by @Anthon is not working properly for me How can I find unique values in two a column table? Let me again define ...
0
votes
0answers
21 views

Row merger due to redunt splits of a single row in Unix

Thanks for your explanation and time Avinash. The code is working as expected but my depiction of problem truned much simpler than the actual problem. Problem: I have a file which has ~5Million of ...
0
votes
1answer
42 views

Using ARGV to get user input in Awk script [closed]

I know ARGV[i] can be used for storing user input. However, I want to use it in awk script and get the ARGV[i] to compare the field from another text file. If the ARGV[i] matches the field or the ...
3
votes
2answers
103 views

How can I partition a set of words given pairs that must end up together?

I have a space or comma separated table with two columns, each row representing equivalence of the two words. A B B C B D C E F G What I want is a table with each row listing all mutually ...
0
votes
2answers
14 views

Replicate lines in text fine according to a column value

I have a text file with a column containing text/data followed by an abundance column, e.g. Car 3 Apple 2 In order to subsample these columns I would like to replicate each text column by ...
1
vote
1answer
42 views

How can I maintain line breaks in body text with mailx when sending attachments?

I have a script that emails the text of a log file as the body of the email. That log file has some blank lines between lines of text for readability. When I use the following line in the script, ...
2
votes
2answers
43 views

Sed/awk/perl: modify text preserving parts and align to a column

I have a text like this: A1JOURNEY0TO1 .BYTE 00, 00, 00 A2JOURNEY0TO2 .BYTE 00, 01, 00 A3JOURNEY1TO0 .BYTE 00, 01, 01 I need to have: JOURNEY_01 ; 00 TO ...
1
vote
1answer
16 views

redirect output of awk gsub command to file

I found an answer on how to remove commas between quotes of a comma delimited csv file over here on stackexchange (cannot comment there as I don't have the needed rep). awk -F'"' -v OFS='' '{ for ...
10
votes
6answers
386 views

How to extract a key value pair from ps command

I am trying to extract application name from ps command: ps -af -u sas |grep "java"| grep -v "grep" above command produces the following: sas 24431 1 0 Oct10 ? 00:51:08 ...
0
votes
1answer
12 views

JED in no_mode by default

How do I ensure that JED editor always uses no_mode by default? I've tried this: Default_Mode = &no_mode; But it still opens files with unknown extensions like .rules in text_mode.
7
votes
3answers
231 views

Empty a file without grep subsequently treating it like a binary file

Currently I have netcat piping output to tee which is writing to output.txt with nc -l -k -p 9100 | tee output.txt I want to monitor this output, so I'm watching it with tail -f | egrep -i 'regex' ...
2
votes
3answers
172 views

How do I reorder or rearrange the output below

Here in my INPUT: 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0091:"ODMBS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" 0015:"IDPTS3O" 0034:"ADTSS1I" ...
1
vote
1answer
48 views

Rename file using sed

I have files whose filenames are in format ddmmyyyyTEXT.txt I want to rename all the files to yyyy-mm-dd-TEXT.txt Here, TEXT is some arbitrary text. What sed expression I should use ??
0
votes
3answers
49 views

Filtering a .CSV file based on consecutive numbers in a line

I have a CSV file as follows: 1st 2nd 3rd 4th ID ... Res Res Res Res (other columns) ... RZ_AUTO_1, 1cx0, C118, B, C119, B, A165, ...
3
votes
2answers
91 views

How to add new lines from another file with sed

I'm trying to find the sed command so that I can put filename into filename1. These are my two separate files. INPUT filename has: Cindy 11 22 54 Chester 48 12 84 INPUT filename1 has: Name ...
2
votes
2answers
61 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
-3
votes
0answers
43 views

Copy strings with only 1 word on the line

I’m new to script and I wondering the best way to copy lines with one word to a new file? Would it be best to use parameters?
3
votes
1answer
78 views

Read .desktop files programmatically [closed]

I would like to make a C program (GTK) that reads a .desktop file. The only thing I need from the .desktop file is the exec command. Is there something built into GTK that would allow me to interact ...
1
vote
1answer
78 views

Summing by common strings in different files

I have a file file1 with the amount of times a user shows up in the files, something like this: 4 userC 2 userA 1 userB and I have another file file2 with users and other info like: userC, degree2 ...
0
votes
1answer
59 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
2
votes
4answers
72 views

replace several comma out side one or more sets of curly braces and exception that in one or more sets of curly braces

In the text file I have multi records. Every record has multi column with comma separated, some column have one sets of curly brace and other have more than one curly brace. I need: If a comma is ...