0

I have this input file:

...
https://long-url/something?value1=999&something=xyz?value2=555
https://long-url/something?value1=534&something=xyz?value2=2312


https://long-url/something?value1=534&something=xyz?value2=233
https://long-url/something?value1=534
...

I want to capture the value1 and value2 and output them as csv, and drops any lines that don't match:

...
999,555
534,2312
534,233
...

I have no problem on capturing the values, but I'm having issue removing any unwanted lines:

sed 's/.*categoryid=\([0-9]*\).*courseid=\([0-9]*\).*/\1,\2/g;/^[^0-9]\+/d' input-file

Basically, the /^[^0-9]\+/d is not working as intended. I was thinking that it would remove any lines that don't start with any digit, but instead, it gave me:

999,555
534,2312


534,233

My sed version: sed (GNU sed) 4.7

3 Answers 3

3

You are using a substitute, followed by a delete:

sed -e 's/.../.../g' -e '/.../d'

That's a bit volatile, because your /.../d might accidentally match something else in the file's contents.

Instead, I would only print the lines that were affected by the substitute. That can be done with -n (to avoid printing by default), and then adding p to the end of the line to print matched lines. That avoids the need for a second sed statement.

sed -ne 's/.../.../gp'

I would also remove the trailing g since you don't need more than one match per line:

sed -ne 's/.../.../p'

The result is:

sed -ne 's/.*categoryid=\([0-9]*\).*courseid=\([0-9]*\).*/\1,\2/p' input-file

In action:

$ sed -ne 's/.*value1=\([0-9]*\).*value2=\([0-9]*\).*/\1,\2/p' << EOF
https://long-url/something?value1=999&something=xyz?value2=555
https://long-url/something?value1=534&something=xyz?value2=2312


https://long-url/something?value1=534&something=xyz?value2=233
https://long-url/something?value1=534
EOF

outputs

999,555
534,2312
534,233
0
3

Your expression, ^[^0-9]\+, forces the matching of at least one non-digit, which means empty lines won't match it, as they don't contain non-digits.

Personally, I would do this the following way:

sed -e '/.*categoryid=\([0-9]*\).*courseid=\([0-9]*\).*/!d' \
    -e 's//\1,\2/' input-file

This first deletes all lines that do not match your specific regular expression (for example, empty lines). The second expression performs the substitution. When an empty pattern is used (as in the substitution in the second expression), the most recently used pattern is re-used.

Note that you don't need /g at the end of the substitution command as you only ever expect to match the pattern once per input line.

1
  • I wish I could give more votes.
    – annahri
    Commented Feb 12, 2022 at 12:58
1

To match /^[^0-9]\+/, the line must must contain at least one character, i.e. it can't be empty.

Just don't remove the lines that start with a digit:

/^[0-9]/!d

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .