1
vote
3answers
122 views

Joining files on disk

Say I have several file with the following naming: 000001 000002 000003 ... Each of these files is a csv file (may include escape characters). In total the folder has ~20GB of data. How can I ...
1
vote
3answers
83 views

merging all files into one file?

I was trying to merge all the text files in to one file. All the text files has one column and equal number of rows. My data file names are data_1.txt, data_2.txt, data_3.txt, . . . data_n.txt. ...
4
votes
2answers
104 views

Convert file with integers written in ascii to binary file of integers

I have a file that contains a long list of integers written in ascii separated with newlines, like such: -175 2 -19345 345592 -45 2355 etc... I want to convert this file into a "binary" file ...
1
vote
1answer
65 views

Replace a string in a file and overwrite the file

I wrote my script using the the directory ~/deleted (it's a long script). after using ~/deleted for the whole script. I want to let the user choose the directory by putting the path in ~/.rm.cfg. My ...
4
votes
2answers
178 views

Joining two files

I have a 2 files with the following: File1.txt A 1 B 2 C 5 Z 3 File2.txt A 4 B 7 C 10 D 11 What I would like to do is create something like A 1 4 B 2 7 C 5 10 D - 11 Z 3 - Is there a ...
3
votes
2answers
61 views

The reverse of `tree` - reconstruct file and directory structure from text file contents?

We all know we can use tree to get a nicely formatted text "visualization" of the structure of a directory; say: $ tree -spugD /usr/include/boost/accumulators/numeric/ ...
7
votes
6answers
533 views

How can I get a specific line from a file? [duplicate]

I want to extract an exact line from a very big file. For example, line 8000 would be gotten like this: command -line 8000 > output_line_8000.txt
1
vote
2answers
98 views

Create file with specific formatting from directory contents

I am using Solaris and csh. In $home/test there are five files: a.txt, b.txt, c.txt, d.txt and e.txt. If I run % find $home/test/ -type f -print > ../filenames.txt that file will look like: ...
0
votes
0answers
107 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
-1
votes
1answer
60 views

Importing an excerpt from a file into a script

I'm trying to import an excerpt from a file into a script for processing. LIST=$(sed '1,/+++NETWORKLIST+++/d' < /path/to/file | sort -t ";" -k2,2r) My problem is that this script is not deleting ...
12
votes
1answer
3k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
4
votes
3answers
415 views

Parsing string token with bash

I have this output (from a previous command): aaa something bbb someother ccc blabla For each line, I would like to create a file whose name is the first token, and whose ...
2
votes
2answers
133 views

Write In A File at Specific Location

I am trying to create a script that will add a user to the UserDir enabled list or will remove it from UserDir disabled list in userdir.conf. For the sake of my question, let's say I have the ...
3
votes
6answers
849 views

Removing blank lines from a file

I want to remove all empty lines from a file. Even if the line contains spaces or tabs it should also be removed.
3
votes
2answers
572 views

How to find frequency of occurrences of strings contained in a file?

I have a file that contains a list of URLs of the form EDIT http://www.google.com/absd/siidfs/kfd837382$%^$&, www.google.com, google.com yahoo.com/list/page/jhfjkshdjf... I ...
3
votes
3answers
153 views

Transforming text to tabular form

I have a text file with the following structure: aaa bbb ccc ddd eee fff 1 2 3 4 5 6 1.1 1.2 1.3 1.4 1.5 1.6 ggg hhh iii jjj kkk lll 7 8 9 10 11 12 2.1 2.2 2.3 2.4 2.5 2.6 and I want the ...
5
votes
5answers
328 views

Split the content of a file in linux

I have a text file which has content like this: abc.tar^@xxx.tar^@yyy.tar^@ Say for example I have this content in a file named abc.txt and I want to split the content and write the first two ...
2
votes
2answers
106 views

Diverting a file write to process

I have an OSX GUI application which I can't change - it writes new CVS lines to a log file when it completes a job. I'd like to work with these new lines, but they are constantly being added. Is ...
4
votes
3answers
73 views

Replace all the occurences of a term in the files contained in a folder

In a folder on my system, I have a C++ project that I'm trying to compile. This process uses some C++ idioms which are not supported by my compiler version. In particular, often inside the code it is ...
7
votes
3answers
7k views

How do I append text to the beginning and end of multiple text files in Bash?

I have a directory full of text files. My goal is to append text to the beginning and end of all of them. The text that goes at the beginning and end is the same for each file. Based on code I got ...
1
vote
1answer
766 views

Reading the contents of the file and splitting using ksh

We're using a ksh script for installing one product. I've another config file, I'd need to read this configuration file from my main script Content of the Configuration file: ...
20
votes
1answer
899 views

How to implement a horizontal cat? [duplicate]

Standard cat concatenates files line by line (row by row, if you will). I find myself needing a horizontal cat command more and more often recently; i.e. a command that takes a list of files and ...
2
votes
1answer
589 views

Regarding generating intersection and union of two csv files

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C. I would ...
2
votes
2answers
100 views

reformat a text file to have all text on one line using CentOS 5.6

I have a 210-line block of HTML that I have stored in a HTML for debugging purposes. The HTML, as you would expect, is formatted accordingly so is stored in the file with spaces and tabs. Is it ...
1
vote
2answers
452 views

Remove first instance of lines with duplicate field value

I have sha1 summed all the image files on my storage server and put the results in a text file in the form of: sha1sum filename I've sorted the file and removed all all unique sha1sum entries. So ...
3
votes
3answers
285 views

Regarding separate a single file into multiple files according to line separation

Currently, I have a plain text file, A, such as lowest priority very high significance. outstanding very novel In this file, every line contains a sentence. I want to separate this file into ...
37
votes
5answers
19k views

How to remove duplicate lines inside a text file?

A huge (up to 2 GiB) text file of mine contains about 100 exact duplicates of every line in it (useless in my case, as the file is a CSV-like data table). What I need is to remove all the repetitions ...
14
votes
3answers
10k views

How can I find out how many lines a text file contains without viewing it?

How can I find how many lines a text file contains without opening the file in an editor or a viewer application? Is there a handy Unix console command to see the number?
2
votes
4answers
172 views

Lines 1,2,3,4,…,n-1, n to lines n,n-1, …,4,3,2,1

I just realized that I could solve this problem by reversing line numbers from 1, 2, 3,...,n to n, n-1, ..., 3,2,1 and then use the same logic as earlier. So I want to know how can I reverse the order ...
2
votes
2answers
491 views

Comparing files and their properties

I get information of a certain set of files in my mail every day, which looks like this: 58623208 Sep 14 20:08 blbn_blfbe_drv 57904920 Sep 14 19:54 blbn_cycmn 55814208 Sep 14 06:02 clsa_Upd 38912000 ...
1
vote
1answer
920 views

remove carriage return character from large binary file

I have a large binary file ( nearly binary file ! ) which has many \r\n in it [ gets Generated in windows env and doesnt have controle over generating application ]. I want to remove \r from this ...
9
votes
3answers
5k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
7
votes
2answers
1k views

How can I cut a large file in place?

I have a very big text file, about 80GB, and I need to cut a part form it that lies between two given lines. The part I need is not big, and I have not enough space left on the hard drive to do things ...
2
votes
1answer
894 views

How can I add a line to multiple files in a directory tree structure?

Adding a line to multiple files in the same structure is not very hard, but I am stumped on how to do it over a possibly large directory tree structure. So lets say we have your project directory, ...
14
votes
2answers
2k views

How can I count the number of different characters in a file?

I would need a program, that outputs the number of the different characters in a file. Example: > stats testfile ' ': 207 'e': 186 'n': 102 Exists any tool, that do this?
18
votes
3answers
28k views

How to merge all (text) files in a directory into one?

I've got 14 files all being parts of one text. I'd like to merge them into one. How to do that?