Letting HTML file act as URL with wget

Question

If you want to get straight to the question, please skip all the way down to the bolded and italicized question past the code.

Intro:

I am experiencing problems with wget and html files. I am using Ubuntu 15.10.

I am trying to gather data from a database, but instead of using "get" search, it uses "post". This simply means that I don't have URLs to work with for wget.

I have created html file to search through "post" method to bypass the "get" restriction of the site. This means that it loads the initial html file, and since it is searching using "post", the stuff I want wget to download comes takes a while to load.

I understand that wget works by searching the html file for http links, but is there a way to get wget to let the html page load for a second or two, and continue with downloading the page.

Code:

Here is the html code that acts as my link to the page I want:

<form name="PrimarySearch" id="PrimarySearch" action="http://www.dec.ny.gov/cfmx/extapps/derexternal/spills/details.cfm?pageid=2" method="post" onsubmit="return _CF_checkPrimarySearch(this)">
    <p>
        There are two ways to search the database: by spill number or by other search criteria.&nbsp;&nbsp;
        Entering a spill number will provide the full record for one spill.&nbsp;&nbsp;
        Entering other search criteria will provide a list of spills that meet the criteria,&nbsp;
        with partial information about each.
    </p>
    <div>
        <label for="spill_num" id="spill_num">
            <strong>Search method #1:</strong> Enter a spill number:
        </label>
        <script type="text/javascript">
            <!--
            document.write('<input type="hidden" name="bjavascriptEnabled" value="1">');
            -->         
        </script>
        <input name="spill_num" id="spill_num"  type="text" value="1409515"/>
            <input type="hidden" name="pageID" value="2">
        <input type="Submit" value="Submit">
    </div>
</form>
<script>document.getElementById('PrimarySearch').submit();</script>

So if I run this html file with wget and -i, it looks for a link within the html code, without even letting the html file run.

Question:

Can I let wget use my html file as an url and load it to download it?

A prompt response would be much appreciated as I need this data ASAP.

Thanks!

Howasaur

Answer 1

Big thanks to @JeffSchaller , who mentioned --post-file and --post-data, which solved the whole issue.

I was able to use either with the generic "post" method only page (http://www.dec.ny.gov/cfmx/extapps/derexternal/spills/details.cfm?pageid=2) to get wget to input the value that I wanted to access that I could not through "get" method that the site search blocks. I ditched the html code that I used earlier as wget doesn't process html files.

Code:

Disclaimer: I have not tried wget without --no-parent, --convert-links, and --html-extension, but one, some, or all helped me wget the page as a local html file

wget --no-parent --convert-links --html-extension --post-data 'spill_num=1409515' http://www.dec.ny.gov/cfmx/extapps/derexternal/spills/details.cfm?pageid=2

This downloads the page as a local html file. Solving the issue where I would get the page, but with not formatting, rendering the file as blank.

Howasaur