6

I have a file that look like this one:

10-04-2022 00:39:13,36707,1455008753,32
11-05-2022 00:39:13,36708,1555008753,26
21-05-2022 00:39:13,36708,1555408753,15
12-06-2022 00:39:13,36709,1655008753

Because values into last field are related to running time of next line, I would like to shift down them to this way for using with gnuplot.

10-04-2022 00:39:13,36707,1455008753,
11-05-2022 00:39:13,36708,1555008753,32
21-05-2022 00:39:13,36708,1555408753,26
12-06-2022 00:39:13,36709,1655008753,15

4 Answers 4

12

A simple awk script could do this:

awk 'BEGIN {FS=OFS=","} { tmp=$NF; $NF=save; print; save=tmp; }' < input > output

This saves the fourth field into a temporary variable, replaces the fourth field with the previously-saved value, then prints the new line. Once printed, it saves the previous value of that fourth field for the next iteration. On the first line, the "save"d value is empty (the default awk behavior), which achieves the desired result.

The "BEGIN" section sets the Field Separator (used for splitting the input) and the Output Field Separator (used when printing the line/fields back out) to a comma. See your local awk man page, various online references, or the Open Group Base Specifications for awk to learn more.

0
9

Another awk solution, tested on GNU awk 5.1.0.

$ awk -F, -v OFS=, '{print $1,$2,$3,x;x=$NF}' myfile
10-04-2022 00:39:13,36707,1455008753,
11-05-2022 00:39:13,36708,1555008753,32
21-05-2022 00:39:13,36708,1555408753,26
12-06-2022 00:39:13,36709,1655008753,15
$
0
2

Using Raku (formerly known as Perl_6)

raku -e 'my @a = lines>>.split(","); my $m = @a>>.elems.max;  \
         .flat.join(",").put for [Z] @a>>.[0..$m-2], ("", @a>>.[$m-1]).flat;'

OR

raku -e 'my @a = lines.map: *.split(","); my $m = @a.map(*.elems).max;  \
         .flat.join(",").put for [Z] @a.map(*.[0..$m-2]), ("", @a.map: *.[$m-1]).flat;' 

This Raku answer is a little longer but is also more of a general solution, since the number of columns per row isn't hard-coded: a file-wide max is computed.

Briefly (both examples), lines are read-in, each split on , commas, and stored in @a array. A max number of elems (columns) is computed and stored as $m. Then Raku's [Z] "Zip Reduction" operator is used to pull off elements from the two lists that follow it one-by-one: these elements are flattened and joined back together in the output, such that now a "" empty string occupies the end of the first row, and subsequent @a[$m-1] indexed-values are moved down a row. Nota bene: Raku doesn't auto-flatten array elements, so you have to flatten them manually (if desired).

In the two examples I've tried to show how Raku's >> "hyper"-operator is similar to mapping a function/index over individual elements, either .map(*.fn) or (at the end of a method chain) using the colon form .map: *.fn;, which can reduce the number of parens required.

Sample Input:

10-04-2022 00:39:13,36707,1455008753,32
11-05-2022 00:39:13,36708,1555008753,26
21-05-2022 00:39:13,36708,1555408753,15
12-06-2022 00:39:13,36709,1655008753

Sample Output:

10-04-2022 00:39:13,36707,1455008753,
11-05-2022 00:39:13,36708,1555008753,32
21-05-2022 00:39:13,36708,1555408753,26
12-06-2022 00:39:13,36709,1655008753,15

Addendum: Here's another Raku approach, this time unwrapping the row/columns into individual elements, and reconstructing with rotor. It gives the exact same result as above:

raku -e 'my @a = lines>>.split(",").flat; my @b = @a[3,7,11...*];  \
         @a.=rotor(3 => 1); @a.=map(*.join(",")); @b.=unshift(""); \
        .join(",").put for [Z] @a, @b;' 

https://docs.raku.org/language/operators#index-entry-[]_(reduction_metaoperators)
https://docs.raku.org/language/operators#index-entry-hyper_method_call_operator
https://docs.raku.org/routine/flat

2
  • 1
    Nice! Does raku have perl5's -a option for auto splitting?
    – terdon
    Jun 15 at 19:26
  • Thank you @terdon! Not that I am aware of. Raku implements two functions that are supposed to obviate reliance on command-line switches: comb and words. AFAIK both these have method fn()and routine .fn forms. The comb function takes a regex matcher, and breaks textual input around any matches. The words function breaks on \s whitespace, and is conceptually similar to Raku's lines function, breaking on \n newlines. Actually, it's probably better to say that .words is really .comb(/ \S+ /) and .lines is really .comb(/ ^^ \N* /) (copied directly out of the docs). Jun 15 at 19:42
0

Using GNU sed 's extended regex mode to simplify regex composing, our approach is to maintain two lines at any time in the pattern space. Then move previous line's second last field to the current line end.

sed -E '
  1s/$/,/
  $!N
  s/(,[^,]+)(,[^,]*\n.*)/\2\1/
  P;D
' file

Input:-

10-04-2022 00:39:13,36707,1455008753,32
11-05-2022 00:39:13,36708,1555008753,26
21-05-2022 00:39:13,36708,1555408753,15
12-06-2022 00:39:13,36709,1655008753#

Output:-

10-04-2022 00:39:13,36707,1455008753,
11-05-2022 00:39:13,36708,1555008753,32
21-05-2022 00:39:13,36708,1555408753,26
12-06-2022 00:39:13,36709,1655008753#,15

4
  • 1
    "second to last" --> "last"? and is there an extra # in the output, or was that part of your input? (it wasn't in the Question, so I wasn't sure)
    – Jeff Schaller
    Jun 15 at 15:30
  • The # I put at my end for testing the last line.
    – guest_7
    Jun 15 at 15:32
  • But it is the second to last of previous line. You have to look carefully.
    – guest_7
    Jun 15 at 15:35
  • This may just be a language/translation issue, but my clarification is that we're not moving the previous line's "second (to) last field" but rather that line's "last field".
    – Jeff Schaller
    Jun 15 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.