These are 2 common delimited, text file transformation problems I'd like to find scripts to do (I dont know if
perl is best for these tasks).
csh is the default shell on my Solaris10 unix server, so solutions that work for either the
sh shells would be preferred.
I have the following 2 delimited ASCII files:
File1.dat (map file):
1|A 2|B 3|C 4|D 5|E 6|F . . . 26|Z 27|Who 28|What 29|Where
Field1|Field2|Field3 1|John|20120101 3|Amy|20120103 6|Larry|20120104 3|Kevin|20120203 8|Fred|20120623 4|Mary|20120728 26|Eric|20120819 28|Larry|20120917
I want to use the map that is represented in
File1.dat to transform
File2.dat into a 3rd file
File3.dat that looks like the following. Essentially I am using the map in File1.dat to create/populate a new column into my file so that at the end, the new transformed file has 4 columns, as follows:
Field1|Field2|Field3|Field4 1|A|John|20120101 3|C|Amy|20120103 6|F|Larry|20120104 3|C|Kevin|20120203 8|H|Fred|20120623 4|D|Mary|20120728 26|Z|Eric|20120819 28|What|Larry|20120917
I want to keep only the 1st record (in a series of sorted records) of a family of records that share the following commonality [Field 2, Field3, Field4 have the same value across records]
I have the following initial delimited file File1.dat (15 records each with 4 Fields). To the left is not part of the file its just a Record# so we can keep track of what is happening.
Field1|Field2|Field3|Field4 1. 20120227|Andy|101|34 2. 20120315|Andy|101|34 3. 20120415|Andy|101|36 4. 20120417|Andy|103|37 5. 20120417|Andy|103|37 6. 20120227|Jane|101|34 7. 20120315|Jane|101|34 8. 20120415|Jane|101|36 9. 20120417|Jane|103|37 10. 20120417|Jane|103|37 11. 20120227|Bob|101|34 12. 20120315|Bob|101|34 13. 20120415|Bob|101|36 14. 20120417|Bob|103|37 15. 20120417|Bob|103|37
I want to convert this 15 record file (
File1.dat) into a newfile (
File2.dat) that looks like the following (keeps only these 9 records).
As you can see it takes out the duplicate records in
File1.dat, where the definition of a duplicate record equals [ Field2 & Field3 & Field4 have equal values across any 2 or more records ].
1. 20120227|Andy|101|34 3. 20120415|Andy|101|36 4. 20120417|Andy|103|37 6. 20120227|Jane|101|34 8. 20120415|Jane|101|36 9. 20120417|Jane|103|37 11. 20120227|Bob|101|34 13. 20120415|Bob|101|36 14. 20120417|Bob|103|37
Please let me know where in a solution code it addresses the following 2 things:
a) The fact the the delimiter in my files is a pipe
b) What part of the code refers to Field1, Field2, Field3, Field4, etc