[SGVLUG] Polling Web Sites

[Robert Leyva]

I would suggest using something like wget (www.gnu.org/software/wget)

Almost all web page actions end up being a URL request with arguments, say
http://www.google.com/jobs/search?status=latest

Once you know what the button is actually doing, you can then put that URL
into wget, then run a shell script that goes to that URL, fetches the
text, and then you can do processing on it, (grep?), to see if there are
things of interest to you.

Just my two cents.

Me
>
>      Got a little project here that I could use some help on.  El Monte
> City School District uses a program called Aesop to post daily
> openings for substitute teachers.  All I have to do is go to their web
site and click on the search button and I can see who has
> currently called in to be absent.  Trouble is, if someone calls in sick
just after I've checked, I won't find out about it until the next time I
check.  And I have better things to do than sit and click on the search
button all evening.
>      So I began to figure out ways to poll that site automatically.  The
> current approach works like this:  A timing program (written in C) runs
in the background on a virtual terminal and produces a negative pulse on
data line 1 of the parallel port every few minutes.  I 'hot wired' the
left click switch (high, pull-down side) of a USB mouse to that data
line (through a diode to protect the port in case someone physically
clicks the mouse).  By leaving the cursor on the search button, the
background program electronically clicks that button every few minutes. 
All I have to do as I go about my business is glance at the screen every
now and then to see if anything new has come up.
>      But this is over-complicated.  There ought to be a simple way to poll
> that page programatically without messing with the hardware.  Say, by
using the usb event mechanisms?  Like as not somebody somewhere has
already written code to do it.  I'd appreciate anyone who could point me
in the right direction.
>
> John
>
> ***************************************************************************************
If the mind is not constrained by walls and fences, where is the need
for Windows and Gates?
>


-- 
"Knowledge is Power" -- Francis Bacon

Robert Leyva
mrflash818 at geophile.net
AOL IM: mrflash818




-- 
"Knowledge is Power" -- Francis Bacon

Robert Leyva
mrflash818 at geophile.net