0

I have a Linux VPS, There are many HTML forms on it, as follows:

<table>
    <thead>
        <tr>
            <th>ver</th>
            <th>link</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td>1.0.1</td>
            <td><a href="http://speedtest.tokyo2.linode.com/100MB-tokyo2.bin">download</a></td>
        </tr>
        <tr>
            <td>1.0.2</td>
            <td><a href="http://speedtest.singapore.linode.com/100MB-singapore.bin">download</a></td>
        </tr>
        <tr>
            <td>1.0.3</td>
            <td><a href="http://speedtest.fremont.linode.com/100MB-fremont.bin">download</a></td>
        </tr>
        <tr>
            <td>1.0.4</td>
            <td><a href="http://speedtest.dallas.linode.com/100MB-dallas.bin">download</a></td>
        </tr>
        <tr>
            <td>1.0.5</td>
            <td><a href="http://speedtest.atlanta.linode.com/100MB-atlanta.bin">download</a></td>
        </tr>
    </tbody>
</table>

The table is relatively simple, only the version number and file download link.

I want to use a shell script to access the URL in the table, in order to get the remote file date and size, then update this information into the table.

They will eventually look like this:

enter image description here

Before asking the question, I looked at the curl and wget documentation, where the curl test can view the file information in standard output, but I don't know how to write an automated script to accomplish this task. I just got in touch with Linux, hope can get everyone's help, thank you!

14
  • lftp? ....... Sep 28, 2019 at 10:18
  • You want to use ssh to get this information and then comile it to an html file on your local machine? Or are you talking about files hosted on an http server and you are trying to get their modification date without having access to the server?
    – rudib
    Sep 28, 2019 at 10:18
  • read the man page for curl, wget or ssh+ls Sep 28, 2019 at 10:19
  • @rudib I have a Linux vps with a number of table files on it. The content format has two columns, the first column is the version number, and the second column is the file download link. I want to run a shell script in Linux that causes the table to display the modification time and size of the download link file. I am a newcomer to Linux. I am not sure if this is feasible?
    – Matthew
    Sep 28, 2019 at 12:04
  • @user1133275 Before asking the question, I looked at the curl and wget documentation, where the curl test can view the file information in standard output, but I don't know how to write an automated script to accomplish this task. I just got in touch with Linux, can you help me?
    – Matthew
    Sep 28, 2019 at 12:12

1 Answer 1

1

This will return the file information you can get from the header:

curl --head http://speedtest.newark.linode.com/100MB-newark.bin

It will return:

HTTP/1.1 200 OK
Server: nginx
Date: Sat, 28 Sep 2019 12:47:03 GMT
Content-Type: application/octet-stream
Content-Length: 104857600
Last-Modified: Thu, 01 Aug 2019 16:35:25 GMT
Connection: keep-alive
ETag: "5d4314cd-6400000"
Accept-Ranges: bytes

If that is what you need, you can write a bash script to generate a table/html file with that information.

You can use this in a script like that:

#!/bin/sh

cat  << EOF
<table>
    <thead>
        <tr>
            <th>ver</th>
            <th>link</th>
            <th>modified</th>
            <th>size</th>
        </tr>
    </thead>
    <tbody>
EOF

$i=1

cat urls.list | while read url
do
        file_info=$(curl -s --head "$url")
        last_modified=$(echo "$file_info" | grep Last-Modified | cut -c16- | tr -d '\r\n')
        content_length=$(echo "$file_info" | grep Content-Length | cut -c17- | tr -d '\r\n')

cat  << EOF
        <tr>
            <td>1.0.$i</td>
            <td><a href="$url">download</a></td>
            <td>$last_modified</td>
            <td>$content_length</td>
        </tr>
EOF
let "i++"
done

cat << EOF
    </tbody>
</table>
EOF

You need to create a file called urls.list which should conatain one url per line. Like that:

http://speedtest.newark.linode.com/100MB-newark.bin
http://speedtest.tokyo2.linode.com/100MB-tokyo2.bin

Running the script will produce an output like that:

<table>
    <thead>
        <tr>
            <th>ver</th>
            <th>link</th>
            <th>modified</th>
            <th>size</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td>1.0.1</td>
            <td><a href="http://speedtest.newark.linode.com/100MB-newark.bin">download</a></td>
            <td>Thu, 01 Aug 2019 16:35:25 GMT</td>
            <td>104857600</td>
        </tr>
    </tbody>
</table>

If you need a specific version name, you could store that in the list file with a seperator (e.g.: version name|url). And would need to adjust the code a bit. Now, it just follows the order of the url list.

6
  • Yes, before I ask questions, I know that I can get the file information like this. The difficulty is that I don't know how to write a shell script. I just got into Linux, can you help me? I will create a test environment VPS, which is posted here.
    – Matthew
    Sep 28, 2019 at 13:11
  • Yes, I know what you need now. I will update my answer as soon as I get to it.
    – rudib
    Sep 28, 2019 at 13:32
  • If the version name has no logic, it will throw a runtime error. Can I output an HTML table based on this raw data text? I tried to adjust the script and it failed.
    – Matthew
    Sep 29, 2019 at 3:19
  • Parsing YAML won't work to well in pure bash, you would need a parser for that. If you keep the file so that it has the date entry on top and then the list below and nothing else, it's still possible to parse it in bash (not yaml compliant). However, a list with the syntax like suggested above, would be simpler.
    – rudib
    Sep 29, 2019 at 9:19
  • 1
    I found a solution, set the custom delimiter using the IFS variable, and then add the read variable to get the version number text.
    – Matthew
    Sep 29, 2019 at 9:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .