0

So I have a script that creates a CSV file looking like this:

Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16,
0000 UTC MAY 17,
0600 UTC MAY 17,
1200 UTC MAY 17,
1800 UTC MAY 17,
0000 UTC MAY 18,
0600 UTC MAY 18,
1200 UTC MAY 18,
1800 UTC MAY 18,
0000 UTC MAY 19,
0600 UTC MAY 19,
1200 UTC MAY 19,
1800 UTC MAY 19,
0000 UTC MAY 20,
0600 UTC MAY 20,
1200 UTC MAY 20,
1800 UTC MAY 20,
0000 UTC MAY 21,

What I would like to do is edit my code so that I can add data from another file looking like the second image below and append it to the end of each line so it matches up.

28.0
28.9
29.6
30.3
31.0
31.9
33.1
34.4
35.5
36.2
36.8
37.0
36.9
36.2
35.5
34.6
33.7
32.8

My current code is here:

#! /bin/bash
inputFileString=$1
outputFileString=$2
touch "$2"
chmod 755 "$2"
echo "" > lat.txt
sed -i '1d' lat.txt
rm $2
test=(0,1,2,3,4,5,6,7,8,9)
echo "Converting $inputFileString -> $outputFileString ..."
echo "" >  $2
awk -F '[<>]' '/<lat>/ {print $3}' $1 >> lat.txt
grep -Eo '<dtg>.*</dtg>' $1 | cut -c 6-20  >> $2
sed -i 's/$/,/' $2
#awk '{print $0, "$variable"}' $2 >> $2
#awk -F '[<>]' '/<lon>/ {print $3}' $1 >> $2
sed -i '1d' $2
sed -i '1i Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity' $2
echo Done!

Just for reference my code takes in an input of a kml file and extracts specific data and uses that data to produce a csv file.

3
  • Please don't post images of text. Copy and paste the text itself into your question and format it as code by selecting it and pressing Ctrl-K or by using the editor's {} icon.
    – cas
    Mar 15, 2022 at 4:50
  • BTW, you're probably making this far more complicated than it needs to be. KML is an XML format, so you can use an XML parsing tool (such as xmlstartlet) or XML parsing library for your preferred programming language (e.g. perl, python, whatever) to extract all the data fields you require and output them in CSV format. Also BTW, your question is quite similar to another question from earlier today, also asking about extracting data from a KML file, you may find useful info there: unix.stackexchange.com/questions/694397/…
    – cas
    Mar 15, 2022 at 4:54
  • @cas So unfortunately I have to write this program in one script file using UNIX and not python but I can still look into using an XML parsing tool correct? Mar 15, 2022 at 5:00

3 Answers 3

0

Using Raku (formerly known as Perl_6)

raku -e 'my @a = @*ARGS[0].IO.lines.skip; my @b = @*ARGS[1].IO.lines;
         "Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity".put; 
         .put for [Z] @a, @b;' ethan1.txt ethan2.txt

Sample Output:

Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16, 28.0
0000 UTC MAY 17, 28.9
0600 UTC MAY 17, 29.6
1200 UTC MAY 17, 30.3
1800 UTC MAY 17, 31.0
0000 UTC MAY 18, 31.9
0600 UTC MAY 18, 33.1
1200 UTC MAY 18, 34.4
1800 UTC MAY 18, 35.5
0000 UTC MAY 19, 36.2
0600 UTC MAY 19, 36.8
1200 UTC MAY 19, 37.0
1800 UTC MAY 19, 36.9
0000 UTC MAY 20, 36.2
0600 UTC MAY 20, 35.5
1200 UTC MAY 20, 34.6
1800 UTC MAY 20, 33.7
0000 UTC MAY 21, 32.8

Above is an answer coded in Raku, a member of the Perl-family of programming languages. The Timestamp file is read off the command line as @*ARGS[0] (the Timestamp header line is skip-ped and added back later). The numeric column is read off the command line as @*ARGS[1]. Each file is converted to an IO object, broken into lines and stored as arrays @a and @b. Then @a and @b are [Z] zipped together element-wise and put.

https://docs.raku.org/routine/Z
https://raku.org

0

Don't know what you are expecting but I think paste is what you are looking for:

sed '1s!^!\n!' another.txt | paste -d '\0' input.csv -
Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16,28.0
0000 UTC MAY 17,28.9
0600 UTC MAY 17,29.6
1200 UTC MAY 17,30.3
1800 UTC MAY 17,31.0
0000 UTC MAY 18,31.9
0600 UTC MAY 18,33.1
1200 UTC MAY 18,34.4
1800 UTC MAY 18,35.5
0000 UTC MAY 19,36.2
0600 UTC MAY 19,36.8
1200 UTC MAY 19,37.0
1800 UTC MAY 19,36.9
0000 UTC MAY 20,36.2
0600 UTC MAY 20,35.5
1200 UTC MAY 20,34.6
1800 UTC MAY 20,33.7
0000 UTC MAY 21,32.8

input.csv and another.txt is copied from your question description.

6
  • 1
    Using \n in the replacement pattern is undefined. GNU sed will insert a newline, most other sed will insert an n. If you simply want to insert an empty line at the beginning, better do sed '1H;1x', that's more portable and easier to read.
    – Philippos
    Mar 15, 2022 at 7:16
  • i.imgur.com/CZoMku9.png Tested on BSD, GNU and busybox. I don't think it would be a problem for the majority. BTW, my shell is fish. Mar 15, 2022 at 7:25
  • @Philippos I am curious and would appreciate if you can tell me other sed implementations that don't do to work. Mar 15, 2022 at 7:27
  • 1
    What is the BSD sed version you tested with? The manual freebsd.org/cgi/man.cgi?query=sed&sektion=&n=1 clearly states »To specify a newline character in the replacement string, precede it with a backslash.« Same for OpenBSD and NetBSD. I'm pretty sure that this will not work with BSD sed on MacOS. Neither will it work on Solaris and HP-UX. It's simply no POSIX feature. Also see unix.stackexchange.com/questions/42321/…
    – Philippos
    Mar 15, 2022 at 8:02
  • That's weird because as the image I gave shows that it works on MacOS, isn't the built-in sed on MacOS based on BSD? Mar 15, 2022 at 8:07
0

Using any versions of echo, cat, and paste in any bourne-like shell:

$ { echo ""; cat file2; } | paste -d '' file1 -
Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16,28.0
0000 UTC MAY 17,28.9
0600 UTC MAY 17,29.6
1200 UTC MAY 17,30.3
1800 UTC MAY 17,31.0
0000 UTC MAY 18,31.9
0600 UTC MAY 18,33.1
1200 UTC MAY 18,34.4
1800 UTC MAY 18,35.5
0000 UTC MAY 19,36.2
0600 UTC MAY 19,36.8
1200 UTC MAY 19,37.0
1800 UTC MAY 19,36.9
0000 UTC MAY 20,36.2
0600 UTC MAY 20,35.5
1200 UTC MAY 20,34.6
1800 UTC MAY 20,33.7
0000 UTC MAY 21,32.8

Or using any awk in any shell on every Unix box:

$ awk 'NR==1{print} NR==FNR{a[NR-1]=$0; next} {print a[FNR] $0}' file1 file2
Timestamp,Latitude,Longitude,MinSeaLevelPressure,MaxIntensity
1800 UTC MAY 16,28.0
0000 UTC MAY 17,28.9
0600 UTC MAY 17,29.6
1200 UTC MAY 17,30.3
1800 UTC MAY 17,31.0
0000 UTC MAY 18,31.9
0600 UTC MAY 18,33.1
1200 UTC MAY 18,34.4
1800 UTC MAY 18,35.5
0000 UTC MAY 19,36.2
0600 UTC MAY 19,36.8
1200 UTC MAY 19,37.0
1800 UTC MAY 19,36.9
0000 UTC MAY 20,36.2
0600 UTC MAY 20,35.5
1200 UTC MAY 20,34.6
1800 UTC MAY 20,33.7
0000 UTC MAY 21,32.8

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .