Sunday, February 25, 2007

Dealing with search forms that hide full queries from user

This is the letter from one of Feed43 users:

I'm trying to make a feed off of a page that requires a search first.
It is a job bank and I want certain items defined first - like the type of job
and the county. However, the search results don't give off a unique
URL. Here is the site:
http://www.illinoiseducationjobbank.org/IASAScripts/IASA.dll/JobSearch

Any idea how I could get this to work?

The problem with this site is that whatever you want to search, it doesn't show you the full query in browser's address bar, because it uses POST method to transmit data to the server.

Feed43 uses only GET method, i.e. passes all parameters in URL string. Luckily, most servers that use POST method, also accept parameters passed in URL. The rest is simple: you need to peek into POST string transmitted to the server in question and append it to server URL being displayed in your browser. Here comes handy the Firefox browser with Live HTTP Headers plugin installed.

Here is step-by-step solution to customer's request:

  1. Open Firefox browser, choose Tools > Live HTTP Headers and make sure that '[x]Capture' checkbox is enabled.
  2. Switch back to Firefox without closing the Live HTTP Headers window.
  3. Open original search form (http://www.illinoiseducationjobbank.org/JobBank/View.asp) in Firefox browser.
  4. Fill out the form and press 'Search' button.
  5. When you get the results, switch back to Live HTTP Headers window again, scroll down to the end of report and look for a block starting with line: POST /IASAScripts/IASA.dll/JobSearch? HTTP/1.1
  6. At the end of this block you will see actual query line. It will look something like:
    PositionType=5&Category=Other+School+Service+Personnel
    &County=&Grades=&EmployType=&EmployType=100&SUBMIT=Search
  7. This is what you need. Copy it to clipboard and then append to the URL you see now in FireFox browser. Your full URL will be:
    http://www.illinoiseducationjobbank.org/IASAScripts/IASA.dll/JobSearch?PositionType=5&Category=Other+School+Service+Personnel
    &County=&Grades=&EmployType=&EmployType=100&SUBMIT=Search
  8. Use this URL in Feed43.

5 comments:

Anonymous said...

Hi-
Thanks for answering my e-mail here. I'm sure many will love this workaround. I tried it and it worked perfectly.

You Guys and Girls rock!

~Kyle~

Flexer said...

Much more easier way is by installing WebDeveloper Toolbar and using Convert form method from POST to GET

rupesh said...

is feed43 not working??

tom sherman said...

Another thing you can do is use Firefox's Web Developer Toolbar and use this option:

Formers > Convert Form Methods > POSTs to GETs.

Makes it a little bit easier to see the URL, as it will actually be in the address bar of the browser rather than the http header.

Anonymous said...

Or use the Web Developer extension which has a tool to automatically convert POST forms to GET forms so no messy hacking.

After installing it, make sure it is visible with View > Toolbars >eb Developer, then on the toolbar itself choose Forms > Convert Form Methods > POSTs to GETs

Now submit the form as usual and you can see whether it worked straight away