parse http data from website

mega128
Posts: 11
Joined: Tue Oct 24, 2017 12:54 am

parse http data from website

Postby mega128 » Thu Jan 24, 2019 7:33 pm

Hi,

I am following an example from here:
https://techtutorialsx.com/2017/05/19/e ... -requests/

and trying to parse temperature value from a website such as https://www.wunderground.com.
I understand I should be using API for this project but I am looking to make it work using regular html payload.
Basically, I am trying to extract '43' temperature which is listed after Temperature table cell.
It's not JSON so I am looking for a way to search the payload residing in int httpCode = http.GET(); and only get temperature value.

Code: Select all

<tr><td>Temperature</td>
  <td>
  <span class="nowrap"><b>43</b>&deg;F</span>
 </td>
  </tr>[Codebox=html5 file=Untitled.html][/Codebox]


Thanks all

Here is more of payload data

Code: Select all

<table><tr><td> <center>
<table border="1" width="100%">
<tr><td colspan="2" ><a name="conditions"></a>
 Updated: <b>1:35 PM </b><br />Observed at
<b></b>
 </td></tr>
<tr><td>Temperature</td>
  <td>
  <span class="nowrap"><b>43</b>&deg;F</span>
 </td>
  </tr>
  <tr><td>Windchill</td>
  <td>
  <span class="nowrap"><b>34</b>&deg;F</span>
  </td></tr>
<tr><td>Humidity</td>
<td><b>93%</b></td></tr>
<tr><td>Dew Point</td>
<td>
  <span class="nowrap"><b>41</b>&deg;F</span>
</td></tr>
<tr><td>Wind</td>
<td>
<b>NW</b> at
  <span class="nowrap"><b>23</b>&nbsp;mph</span>
</td></tr>
        <tr>
                <td>Wind Gust</td>
                <td>
  <span class="nowrap"><b>30</b>&nbsp;mph</span>
</td>
        </tr>
<tr><td>Pressure</td>
<td>
  <span class="nowrap"><b>29.52</b>&nbsp;in</span>
  <b>(Falling)</b>
</td></tr>
<tr><td>Conditions</td>
<td><b>Light Rain</b></td></tr>
<tr><td>Visibility</td>
<td>
  <span class="nowrap"><b>10.0</b>&nbsp;miles</span>
</td></tr>
<tr>
        <td>UV</td>
                <td>1 out of 16</td>
</tr>
<tr><td>Clouds</td>
<td>
<b>Mostly Cloudy</b>
(BKN)
:
  <span class="nowrap"><b>1600</b>&nbsp;ft</span>
<br />
<b>Mostly Cloudy</b>
(BKN)
:[Codebox=html4strict file=Untitled.html][/Codebox]

mega128
Posts: 11
Joined: Tue Oct 24, 2017 12:54 am

Re: parse http data from website

Postby mega128 » Wed Feb 27, 2019 11:15 am

Any idea guys?

Thanks all.

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: parse http data from website

Postby ESP_Sprite » Thu Feb 28, 2019 2:08 am

The complex solution to this would be to use an XML parser like expat or so to parse this, but perhaps you can get away by just searching for the string surrounding the actual temperature. I'd suggest using an API anyway, as this is an extremely brittle solution and will fail as soon as wunderground changes their layout.

GeorgeFlorian1
Posts: 160
Joined: Thu Jan 31, 2019 2:32 pm

Re: parse http data from website

Postby GeorgeFlorian1 » Thu Feb 28, 2019 9:02 am

ESP_Sprite wrote:
Thu Feb 28, 2019 2:08 am
The complex solution to this would be to use an XML parser like expat or so to parse this, but perhaps you can get away by just searching for the string surrounding the actual temperature. I'd suggest using an API anyway, as this is an extremely brittle solution and will fail as soon as wunderground changes their layout.
What is the API you're talking about ? Is it one that helps you parse a string from a HTML input ?

gdsports
Posts: 15
Joined: Wed Aug 02, 2017 12:17 am

Re: parse http data from website

Postby gdsports » Sat Mar 02, 2019 1:13 am

Wunderground previously supported a free HTTP API to return weather data in JSON. It was easy to use and many demos/example programs used it to create WiFi weather display stations. This service is no longer free so scraping the weather data from HTML is one option. Another option is to use openweather.com.

Who is online

Users browsing this forum: Baidu [Spider], gfvalvo and 69 guests