Part 1 discusses the Davis Vantage Pro 2 Weather Station and the theory of a possible way to hack it and enhance its functionality.

Hardly an inspiring title, but it gets the point across :-)

Having dogs and horses means we have an almost obsessive interest in the weather. A number of years ago we bought a cheap weather station. It worked really well but eventually broke. One of the limitations of the unit was it's isolation. You couldn't get data out of it and the timeline information it provided was limited. We decided to replaced it with a Davis Wireless Vantage Pro2 (VP2) weather station. They are very expensive, but they are one of the best around, and provide various options for for exporting the data. We went for the WeatherLinkIP - horribly expensive but it does give the VP2 the ability to automatically upload it's weather information to servers on the Internet, specifically the Davis WeatherLink site. From there it can be distributed to other sites and services such as Wunderground, however it only does this at 15 min intervals.

The Vantage Pro2

The Vantage Pro2 is made up of 2 main components.

The VP2 Console/Receiver that lives inside the house

And the ISS (Integrated Sensor Suite) instrument package that contains the sensors and lives in the garden.

The two communicate wirelessly using the 868MHz^[1] band employing a "frequency hopping spread spectrum radio technology".

How we installed … and a couple of disadvantages

Our IIS is mounted in the garden on the fence. The placement is not ideal for measuring wind - it's too many things obstructing it, but you have to work with what you have.

We mounted the VP2 console in the middle of the house so it's close to the IIS without too many walls or metalwork in the way. Reception is excellent, and it's handy if you’re in the house for checking the display. But not so handy if you're about to go out the front door and want to check the weather so you know how many layers you need to put on and whether waterproofs are required.

The other reason for mounting the VP2 console there was it's close to a network point. The WeatherLinkIP dongle does allow the VP2 Console to send data out to the internet, but it can't do it wirelessly, you have to have it hardwired into your router with an Ethernet cable. Another reason we couldn't have mounted it near the front door - no network point.

You can buy extra consoles from Davis and use them to display information at each door, but it's horribly expensive, like an extra console is over £350 (ouch!). So I wondered if there was a better (and cheaper) way to do this.

Why Data Scientists really hate data exhaust

There’s was another weakness of the system I wanted to address anyway. The IIS sends data to the VP2 console every 2.5 seconds or 24 readings per minute, but the console only uploads one reading per minute to the Internet even on it's fastest update setting i.e it's summarising the data and you're losing 23 sets of readings. And the Davis WeatherLink website only sends data out to Wunderground every 15 mins, so you’re losing even more readings, 359 to be precise. This does save you storage, but that’s a whole lot of data to be throwing away. This approach did made sense when the VP2 was designed because internet connectivity was much slower, data rates were expensive and so was storage. Nowadays, it makes far less sense. Data rates are far cheaper, speeds are orders of magnitude faster and storage is dirt cheap. Throwing away data you’ve spent a lot of time collecting makes no sense now. As an aside, Data Scientists refer to this process of discarding readings in favour of saving storage space as data exhaust and they dislike it.

How the Vantage Pro2's handle data sharing

The VP2 Console has an expansion port that you can plug various data logger modules into an expansion port on the back of the console. The data loggers provide an interface to the outside world; either serial, USB or IP, and provide extra storage capacity. Currently you can get Windows Serial; Windows USB; Mac USB; & an IP logger. The expansion port in the back is actually a simple serial port. It's possible to access this directly, although after the infamous version 3 or "green dot" firmware update it's a bit more convoluted to do so. However that aside, the first two dataloggers are fairly simply devices that just summarise the serial data and pass it on via serial or USB interface. The third one is a bit more complex in that it has the option to automatically upload the data to the Davis WeatherLink servers on the Internet. You can use third party software to connect to the dataloggers eg Weather Display. All the dataloggers will store a certain amount of data as a history which can be accessed by software using the right comms protocols. Helpfully, Davis do publish these protocols so my initial thought was to use these to access the WeatherLink IP with my own software. However when I went looking for the protocols for accessing the WeatherLink IP, it wasn't very clear what was going on. As far as I can make out all the dataloggers essential use the same access mechanism irrespective of the transport layer involved, and that is derived from a serial access mechanism. This isn't too much of a problem for the serial or USB devices where only one device at a time can physically talk to the console at once. But for the WeatherLink IP it does become critical. With an IP based datalogger there's nothing to stop multiple devices from accessing it at the same. Except in the case of the WeatherLink IP, as far as I can make out, it's using the original serial based protocols and simply routing them over IP. This means only one device can access the logger at one time.

That didn't fit with how I wanted to use the device at all. I wanted to have the WeatherLink IP send data to the Davis servers, but let me get at the data as well from multiple devices. Davis's answer to this appears to be to have the software that's connecting take over the functions of distributing the data. So using the IP interface on the WeatherLInk IP was out as well as I still want it to send data to the WeatherLink servers.

Thinking outside the box, or the console anyway

This got me thinking - most of the data is collected by the ISS. The console adds barometric pressure and internal temperature & humidity but that's all. Of those only the barometric pressure is really useful. The inside temperature is measured by a sensor on the motherboard of the console. This isn't accurate as it's distorted by the heat that the console generates. It gets especially bad if you accidentally leave the backlight of the LCD on. The rest of the activity on the console is to do with collation and storing of various key values, min/max data etc. Because I have a wireless IIS it's transmitting the data to the console over a standard 868MHz frequency (we're in the UK, other area may use different frequency bands). I figure what is transmitted can be intercepted :). So why not just intercept the data form the ISS directly, add the barometric data myself, and then store it in a database on my own server. That way I can get all the readings, I can store them all, and I can analyse it however I want. Once it's on my own server I can route it to any additional displays I want around the house or send it anywhere on the internet.

There are multiple advantages to this approach.

• No warranty issues as you're not making any physical changes to the console or the ISS, in fact you're not even physically touching them at all.
• You can collect and store all the data the IIS sends. No more data exhaust.
• The limitations of the amount of storage in the data loggers go away.
• You can still use the data loggers as a backup.
• Once you have the data in your own database, analysing it becomes child's play and you're free to send the data wherever you want.

Storing the data

The Davis Archive records are 52 bytes in length. A data logger can store 2560 of them, meaning they have a capacity of approximately 130KB. (source: Davis WeatherLink Data Sheet). I figured storing this data would be an ideal job for a Raspberry Pi. Assuming the database on the Raspberry Pi stores at roughly the same rate i.e. 52 bytes per records, and that records come in at rate of one ever 2.5 seconds you'll need to store 34,560 records per day. So a day's worth of records would take 1.7MB. That means a 16GB USB memory stick would hold about 25 years worth :D

The design of the Davis Wireless Vantage Pro2 Weather Station

By modern standards the design of the VP2 is very antiquated, especially the console.

The Integrated Sensor Suite (ISS)

The ISS is not something that particularly needs to change. The design of weather instruments has been evolving for centuries, and the requirements and how best to meet them is well understood. You want something that is robust, durable as it has to live out in all weathers (pun intended :) ), and above all has to be accurate. You also want it to be repairable, especially if you spent the amount of money that we spent on the ISS :). The ISS fulfils all of these requirements.

Having said that, there is one upgrade I would add to the ISS if I was Davis. I'd add a GPS unit to it. That way the consoles could auto-configure their Lat/Long coordinates, and could update their clocks periodically. Yes, a GPS is an expensive add-on but the Vantage Pro 2 is an expensive unit.

The VP2 Console

This is the part that really feels outdated. The VP2 was originally launched in 2004, but it was heavily based on the earlier Davis Vantage Pro (VP). I'm not entirely sure when the VP was launched by there is a copyright date of 2001 in it's console manual and of 2000 on the ISS documentation. which makes it more than seventeen years old. There's a great description of the internal contents of the VP2 console in this blog entry Davis Weather Station Hacking. The console itself is based on a Atmel processor, specifically on an Atmel ATMEGA 128L running at 1.8432MHz, with 128Kbytes of FLASH, 4Kbytes EEPROM, and 4Kbytes SRAM. The console does use a completely custom LCD for it's display. 15 years ago this stuff would have been expensive, and so would have extra memory for data logging. This is what I think influenced the design. Technology has moved on hugely in the last 17 years.

A word about I2C and the dataloggers

You'll also see some references on the web that the VP2 console uses I2C to talk to the dataloggers. This is true, but it's only so it can store data in the dataloggers onboard memory. Basically when you talk to the "dataloggers" you actually talk to the VP2 console via a serial interface. The console may then talk to the memory chip in the datalogger via I2C, pull the data back into the console before it sends it to you via the serial interface.

It's all indicative of a system that was designed a very long time again when memory, protocol convertors and microprocessors were a lot more expensive than they are now so you didn't just randomly shove memory & processors in where you didn't need them. One the plus side of this, because the Davis architecture hasn't changed for a very long time it's very easy to get spare parts for even quite old weather stations from Davis. On the negative side I do wonder if Davis are actually doing any real product development.

Davis Product Development

As I just said I wonder if Davis are doing any product development on their weather stations. I'm surprised that they haven't produced an updated console for it's systems, it's well over 15 years old. Or at least reduced the price of the existing console. The production costs must have fallen over the years. Of course reducing the cost will annoy some of your existing customer, especially the ones who just bought one :), but it will hopefully bring in new customers. Given that the console and ISS are effectively independent of each other it would be easy for them to introduce an updated console with greater storage, colour display and Internet connectivity etc. Potentially they could sell that to their existing customers as well. My worry would be that if they don't do this they will be left behind. Other companies have fallen into this trap believing that they are a market leader and don't need to update their products. Eventually this philosophy will bite you and you'll lose your market leader status.

The IP Data Logger

Another area that could be easily upgraded. This unit is currently costs £349. At that kind of cost it would be perfectly possible to build a micro controller into it to host a web server, as well as sending data out to the internet. That would let people access their Vantage Pro's from their smart phones and tablets.

A home brew console

So getting back to the point of all this, a while ago I mentioned the possibility of using a Raspberry Pi to record the data from the ISS. For those who haven't met it before, the Raspberry Pi is a small, powerful, all-in-one computer about the size of a credit card that costs $35. Of course there's still the small problem of how you connect the Raspberry Pi to the ISS. As I mentioned the ISS uses a standard 868MHz packet radio system. This was proving a bit of an issue until I came across this little gem on the Adafruit website. The Adafruit Feather 32u4 RFM69HCW Packet Radio - 868 / 915 MHz is an 868MHz packet radio transceiver with an 8MHz Atmel ATmega32u4 chip in a single package. I figured I could have this eavesdrop on the transmissions from the IIS to the Console, and then dump the data it collects out on the I2C/SPI/USB interfaces to the Raspberry Pi. True, it doesn't have as much storage as the ATMEGA 128L, used in the VP2 Console, but it it doesn't need as the Raspberry Pi will be providing the storage.

However to decoded the transmissions from the ISS you need to understand the protocols that it's using, and unsurprisingly this bit Davis doesn't publish. Fortunately the Internet once again provides :). There's a great series of posts on a blog called It's science, but works like magic that have a lot of the details you need, including code.

• Davis Weather Station Wireless Sniffing: A Start (21 Feb 2001) - the initial look at the Davis Weather Station frequency hopping using a scope to snoop the SPI bus.
• Davis Frequency Hopping Sequence Revealed! (20 Mar 2011) - the basics of finding the Davis frequency hopping spectrum using a logic analyser.
• IM-ME Proof of Life (29th Jan 2012) - initial scans with a hacked IM-ME pager.
• First You Get The Sugar.. (11th Mar 2012) - talks about the CC1021, the RF chip in the VP2 console and how you might use a CC1110 as an alternative in a home brew project.
•  Achievement Unlocked: IM-ME Weather Station Receiver!  (8th Apr 2012) - details how to build a receiver for the ISS transmissions using an "I'm Me Pretty Pink Pager" which uses the CC1110 chip. Awesome!
• More Than One Way to Skin a Cat (3rd January 2014) - introduction to the Moteino with an integrate RFM69 - with code.
• Build Your Own Davis Weather Station Console!!! (17th Feb 2014) - using a a Moteino with an RFM69 module to connect to the ISS - with code.
• GitHub: dekay/DavisRFM69 (2nd May 2015) - An Arduino library on for the RFM69 transceiver module that enables communications with Davis Instruments weather station hardware.

I mentioned an advantage of doing this type of hack was that it couldn't affect the warranty. Another advantage of intercepting the transmissions from the ISS, is that it's going to be very difficult for Davis to block it. They can update the firmware on the VP2 console as we've seen with the version 3 update, but as far as I know they can't update the firmware on the ISS. Certainly there is no mention anywhere on the Davis website of firmware for the ISS. UPDATE: I've since come across a letter on the Weather Display forum that does talk about new firmware for the ISS. However the only upgrade path is to physically replace the transmitter board. That means Davis can't block easily block this kind of hack until they release a new version of the ISS itself. Woo hoo! :)

An update

I originally wrote this back in April 2016, but didn’t get around to publishing it or following up on it at the time. In June 2016 Adafruit added an additional Feather to their lineup. This one also has a RFM69HCW Packet Radio - 868 / 915 MHz in the package, but rather than pairing it with a 32u4 professor, it pairs it with a Cortex M0 processor running at 48Mhz with 256K of FLASH and 32K of RAM. The Adafruit Feather M0 RFM69HCW Packet Radio - 868 or 915 MHz - RadioFruit. This is twice as much flash as the original ATMEGA 128L in the VP2 Console, and 8 times as much RAM. It’s a far more capable than the original processor in the VP2 Console which raises some interesting questions - could it be used to build a system that completely replaces the VP2 Console.

Stay tuned for Part 2 . . .

“If there is no struggle, there is no progress.”
— Frederick Douglass