0

Hi I have a md file containing the below string and I want to write a regular expression for this.

Conditions

  1. The id will be anything.
  2. The type will be youtube,vimeo etc
  3. ID and type are mandatory fields
{% include video.html id="T3q6QcCQZQg" type="youtube" %}

So I want to check the string is in a proper format in bash script otherwise will through an error.

Current code look like this . The below code is working for me without an ID. But I need to add a regex for id as well

IFS=$'\n' read -r -d '' -a VIDEOS < <( grep  "video.html"  "$ROOT_DIR$file" && printf '\0' )

#output => {% include video.html id="T3q6QcCQZQg" type="youtube" %}

for str in "${VIDEOS[@]}"   
 do  
       if [[ "$str" =~ ({%)[[:space:]](include)[[:space:]](video.html)[[:space:]](type="youtube"|type="vimeo")[[:space:]](%})$ ]]; then
            flag="dummy"
            echo "Invalid format::  $second"
        fi
done

Please help

6
  • 1
    What does the rest of the file look like? Do any of the strings in that line appear anywhere else in the file? What have you tried so far? Jul 5 '21 at 12:48
  • @NasirRiley hey I updated the question with code. please check Jul 5 '21 at 13:02
  • You have code stating "Remove possible leading /" etc. Can you show a verbatim example (possibly anonymized) of the initial grep output?
    – AdminBee
    Jul 5 '21 at 13:14
  • @AdminBee I am searching through the md file I have. The first grep output is {% include video.html id="T3q6QcCQZQg" type="youtube" %} {% include video.html id="330853122" type="vimeo" %} Jul 5 '21 at 13:23
  • Would id="T3q6QcCQZQg (no closing quote) be invalid? This file format isn't going to be easy to validate. Since it looks a little bit like XML, a basic XML parser might be the way to go. Jul 5 '21 at 13:51
0

In principle you are almost there. The following is a minimal testable version of a regular expression based on the example content you provided:

#!/bin/bash

VIDEOS=( '{% include video.html id="T3q6QcCQZQg" type="youtube" %}' '{% include video.html id="330853122" type="vimeo" %}' '{% include video.html id="330853122" type="nosuchplatform" %}')

regex='^\{% include video.html id="[^"]+" type="(youtube|vimeo)" %\}$'

for v in "${VIDEOS[@]}"
do
    if [[ "$v" =~ $regex ]]
    then
        echo "$v : valid"
    else
        echo "$v : invalid"
    fi
done

The varying id field can be matched using the "[^"]+" construct, i.e. "a starting ", followed by anything that is not a ", and then one "". You can make it more specific if you know what characters are allowed for the id field, i.e. if you know it can only be alphanumeric characters, try "[[:alnum:]]+" instead.

By storing the regular expression in a shell variable you can avoid several problems in formulating it, just be sure to not quote the variable when using it in your test.

I also assumed that if the regular expression matches you want to output valid (currently you would consider sucess of the =~ test as "invalid" pattern).

3
  • Thank u so much .This worked Jul 6 '21 at 10:06
  • @MeeraSebastian You are welcome. If you found this (or any other) answer useful, please consider accepting it so that others facing a similar problem may find it more easily.
    – AdminBee
    Jul 6 '21 at 10:08
  • AdminBee Seems I dont have enough reputation for accepting an answer Jul 7 '21 at 17:32
0

Since the id and type tags are (probably) not required to be in that order, I'd use a series of regex tests:

for str in "${VIDEOS[@]}"; do
    if [[ $str =~ \{%[[:blank:]]+include[[:blank:]]+.*[[:blank:]]+%\} ]] &&
       [[ $str =~ \<id=\"[^\"]+\" ]] &&
       [[ $str =~ \<type=\"(youtube|vimeo)\" ]]
    then
        echo "valid"
    else
        echo "invalid"
    fi
done
0

bash is great at co-ordinating the execution of other programs, but it's a terrible language for text processing. You should use awk or perl for this. See Why is using a shell loop to process text considered bad practice?.

e.g. with a perl "one-liner":

$ perl -lne 'next unless m/{%.*video\.html.*%}/;
             ($id) = m/\bid\s*=\s*"([^"]+)"/i;
             ($type) = m/\btype\s*=\s*"(youtube|vimeo)"/i;
             print "Invalid format on line $. of $ARGV: $_" unless ($id && $type);' *.md

This allows for id and type to be in any order anywhere on the line, and also allows for optional extra whitespace (\s*) around the = symbols. It expects the entire video include to be on a single line (a more robust version could allow for multi-line strings, but this script doesn't do that). It can process multiple input files at once (e.g. *.md) and will tell you the line number and filename of any invalid lines it finds.

If you want to allow any value for $type (not just youtube or vimeo), replace the third line with:

($type) = m/\btype\s*=\s*"([^"]+)"/i;

or just add more allowed types in the alternation.

Same script as a standalone executable:

#!/usr/bin/perl
use strict;

while(<>) {
  chomp;
  next unless m/{%.*video\.html.*%}/;
  my ($id) = m/\bid\s*=\s*"([^"]+)"/i;

  #my ($type) = m/\btype\s*=\s*"([^"]+)"/i;
  my ($type) = m/\btype\s*=\s*"(youtube|vimeo)"/i;

  print "Invalid format on line $. of $ARGV: $_\n" unless ($id && $type);
}

save as, e.g., verify-videos.pl somewhere in your PATH (e.g. ~/bin/ or /usr/local/bin/) and make executable with chmod +x /path/to/verify-videos.pl.

2
  • Unfortunately we are using bash script only :( Jul 6 '21 at 10:06
  • That's entirely your choice, but you need to know that you will be using the wrong tool for the job. Bash is not suited to text processing, using it for that task will make the programming required more difficult and error-prone, as well as much slower.
    – cas
    Jul 6 '21 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.