[kwlug-disc] wget and variable assignment

[Raul Suarez]

I think it's something like 

saveIFS="$IFS"
IFS=$'\n'
variable = $(cat file.html)
IFS="$saveIFS"

Raul Suarez

Technology consultant
Software, Hardware and Practices
_________________
Twitter: rarsamx
http://rarsa.blogspot.com/ 
An eclectic collection of random thoughts


--- On Thu, 6/3/10, Richard Weait <richard at weait.com> wrote:

> From: Richard Weait <richard at weait.com>
> Subject: Re: [kwlug-disc] wget and variable assignment
> To: "KWLUG discussion" <kwlug-disc at kwlug.org>
> Received: Thursday, June 3, 2010, 1:24 PM
> On Thu, Jun 3, 2010 at 1:09 PM,
> Khalid Baheyeldin <kb at 2bits.com>
> wrote:
> > On Thu, Jun 3, 2010 at 12:58 PM, Richard Weait <richard at weait.com>
> wrote:
> >>
> >> I have a simple screen-scrape to do.
> >>
> >> >From the command line it works fine
> >>
> >> wget -q -O - http://www.openstreetmap.org/stats/data_stats.html|
> grep
> >> "<td>Number of users" | sed -e 's/[:a-zA-Z
> <>/:]//g'
> >>
> >> it returns the plain number
> >>
> >> 262086
> >>
> >> Cool, now to add it to a script
> >>
> >> This works fine
> >> GETTEE=`wget -q -O -
> >> http://www.openstreetmap.org/stats/data_stats.html|
> grep "<td>Number
> >> of users" | sed -e 's/[:a-zA-Z <>/:]//g'`
> >> echo "GETTEE = $GETTEE"
> >>
> >> gives:
> >> GETTEE = 262086
> >>
> >> But.  I want to grab some other data from the
> same page, so I want to
> >> wget once, then grep / sed a couple of times.
>  And I'm breaking it.
> >> The page appears to have been stripped of its \n
> and so grepping the
> >> line I want is failing.
> >>
> >> GETTEE=`wget -q -O - http://www.openstreetmap.org/stats/data_stats.html`
> >> echo "GETTEE = $GETTEE"
> >>
> >> This returns a mess.
> >>
> >> The quick and dirty is to wget four times for four
> numbers, but I
> >> don't want to do that.  How do I assign the wget
> to a variable and
> >> keep \n ?
> >
> > This is not fair for whoever is hosting the server.
> 
> Right. That's why I'm here.
> 
> > Do the wget once, using
> >
> > wget -q -O /tmp/osm.html
> >
> > Then parse that file as many times as you want for
> whatever you want.
> 
> But no way to avoid all those disk calls?  The
> assignment to the
> variable kills the \n or something?
> 
> _______________________________________________
> kwlug-disc_kwlug.org mailing list
> kwlug-disc_kwlug.org at kwlug.org
> http://astoria.ccjclearline.com/mailman/listinfo/kwlug-disc_kwlug.org
>