0

I want to get all JS <script> tag inside data as well as <script src="path/to/js"></script> or <script src="http(s)://example.com/to.js"></script>. same for the <style> and <link href="path/to/css"> or <link src="http(s)://example.com/to.js">

Here I can run different commands to get the script block, script link, and same for style and link.

I have tried below where I can get the script tag details as well as the script link.

sed -n 's/.*\(<script>.*<\/script>\).*/\1/p' path/to/file.html

But here it will return from the start of the <script> tag to the end of the </script>, here if any other context is present in between the

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>My Document</title>
  <link href="https://cdnjs.cloudflare.com/ajax/libs/basscss/8.1.0/css/basscss.css" rel="stylesheet">
  <script src="https://code.jquery.com/jquery-3.6.0.js"></script>
  <style>
    html, body {
      background: grey;
    }
  </style>
  <link rel="stylesheet" href="assets/css/1.css">
</head>
<body>
  
  <a href="https://google.com">Hi</a>

  <link href="assets/css/2.css" rel="stylesheet">
  <script>
    var a = "Hello I'm inclined!";
    console.log(a);
    console.log("jQuery version: ", $.fn.version);
  </script>
  <script src="assets/js/1.js"></script>
  <script src="assets/js/2.js"></script>
</body>
</html>

I got the below outcome with my script sed command when the HTML is minified, and got nothing when the HTML is not minified

<script>var a="Hello I'm inclined!"; console.log(a); console.log("jQuery version: ", $.fn.version); </script> <script src="assets/js/1.js"></script> <script src="assets/js/2.js"></script>

Similar for <style></style> and <link ... rel="stylesheet">

For this I found below command will only return the CSS link from the <link ... /> tag:

sed -n 's/.*href="\([^"]*\).*/\1/p' path/to/file.html

it returns below output, where the anchor(<a ...></a>) tag href is also present due to the match case with the regex I used.

And the outcome is from non minified version.

https://cdnjs.cloudflare.com/ajax/libs/basscss/8.1.0/css/basscss.css assets/css/1.css https://google.com assets/css/2.css

For minified it returns only the last one.

assets/css/2.css

My expectations of the outcome as given below (for non minified version):

scripts

<script src="https://code.jquery.com/jquery-3.6.0.js"></script><script>
    var a = "Hello I'm inclined!";
    console.log(a);
    console.log("jQuery version: ", $.fn.version);
</script>
<script src="assets/js/1.js"></script>
<script src="assets/js/2.js"></script>

Styles

<style>
    html, body {
      background: grey;
    }
</style>
<link href="https://cdnjs.cloudflare.com/ajax/libs/basscss/8.1.0/css/basscss.css" rel="stylesheet"/>
<link href="assets/css/1.css" rel="stylesheet"/>
<link href="assets/css/2.css" rel="stylesheet"/>

Check out my sample HERE

Edit it from HERE

Any other solution will be highly appreciated. (e.g - awk, Python or any other)

10
  • 1
    First, parsing HTML with regular expressions (regex) is extremely error prone and notoriously difficult to get right--especially with the HTML some authors write. Second, your regex is poor: <script>.*</script> means "take everything as long as possible to sill match", because .* is, by default, greedy/longest match. Make it non-greedy, like this <script>.*?</script>, so it will stop at the shortest possible match.
    – C. M.
    Jul 29 '21 at 15:19
  • Thanks for your suggestion. How to make it global or multiline search! I have tried with regex101 code generator for sed, but that never worked.
    – Subhajit
    Jul 29 '21 at 15:28
  • Style sheets are similar when using (inline) <style>...</style> elements, but more complex when using external style sheets, as not all <link ....> definitions are stylesheets--some may define other link relationships. You will have to process them to find which are stylesheets, and which are not. And I have not mentioned yet the added complexities of matching <script ... /> or <link ... /> instead of <script ....>...</script>, or <link ...> or <link ....>...</link>, or attributes with and without quotation marks, and so on.
    – C. M.
    Jul 29 '21 at 15:35
  • With all the above issues, you are better off using a programming or scripting language, such as C/C++, Perl, Python, etc, and an XML/HTML parser library--preferably one that supports XPath to find any script, style, or link nodes.
    – C. M.
    Jul 29 '21 at 15:38
  • regex is the wrong tool to parse html.
    – Panki
    Jul 29 '21 at 17:13
0

All the advice about using XML-aware tools is generally good if you have them and get an answer showing how to use them, otherwise this will probably be adequate for your needs if your input is a subset of XML like you show in your question and > doesn't appear in any of your links.

Using GNU awk for multi-char RS, RT, and \> word boundary:

$ awk -v tag='script' 'BEGIN{RS="</?"tag"\\>"} !(NR%2){print "<"tag $0 RT">"}' file
<script src="https://code.jquery.com/jquery-3.6.0.js"></script>
<script>
    var a = "Hello I'm inclined!";
    console.log(a);
    console.log("jQuery version: ", $.fn.version);
  </script>
<script src="assets/js/1.js"></script>
<script src="assets/js/2.js"></script>

$ awk -v tag='style' 'BEGIN{RS="</?"tag"\\>"} !(NR%2){print "<"tag $0 RT">"}' file
<style>
    html, body {
      background: grey;
    }
  </style>

$ awk -v tag='link' 'BEGIN{RS="<"tag"\\>[^>]*>"} RT{print RT}' file
<link href="https://cdnjs.cloudflare.com/ajax/libs/basscss/8.1.0/css/basscss.css" rel="stylesheet">
<link rel="stylesheet" href="assets/css/1.css">
<link href="assets/css/2.css" rel="stylesheet">
6
  • Thanks for your input but it returns unexpected results. <script <style> html, body { background: grey; } </style> <link rel="stylesheet" href="assets/css/1.css"> </head> <body> <a href="https://google.com">Hi</a> <link href="assets/css/2.css" rel="stylesheet"> > <script <script src="assets/js/1.js">> <script </body> </html>>
    – Subhajit
    Jul 29 '21 at 18:52
  • Are you sure you're using GNU awk (check awk --version)? All we have to test with is the example you provide in your question. If that's not adequate for us to test a solution with then update your question to fix that.
    – Ed Morton
    Jul 29 '21 at 19:06
  • I'm using mac and my awk version is awk version 20070501
    – Subhajit
    Jul 29 '21 at 19:18
  • I have tried your suggestion here also: replit.com/@DevAtCN/…
    – Subhajit
    Jul 29 '21 at 19:19
  • You can join my repl playground and edit from there: replit.com/join/xdapisqpff-devatcn
    – Subhajit
    Jul 29 '21 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.