geoffrey gauchet

Building Your Own Tweeting Weather Station

I've been a big weather nerd for a while. It started with my grandfather and his cork board hurricane tracking map, and then my father and I tracking on paper and then a DOS program, and then me developing my own hurricane intensity scale. I had a few RaspberryPi's lying around and decided to look into building my own weather station for my house. A RaspberryPi, if you're unaware, is a small computer that packs some decent specs (and a lot of extra input/output pins) in a circuit board about the size of a deck of cards. And it's only like $35. A great tool for this job!

I did some googling and came across this guide for building a weather station with a Pi. So, I ordered all the instruments they suggested, ordered some perf-board and terminal connections, and resistors from DigiKey, and bought some outdoor junction boxes from Lowe's and waited to get started. I followed the guide pretty much exactly, so I'm not gonna go into great detail on it, but I learned a few things and made some modifications and additions, so I'm gonna share those. I also documented the build on my Instagram story, so the only photos I have are in portrait and have colorful text printed on them, but whatever. That's what posting in 2020 is.

Getting Started

The first thing you're gonna do is prep your Pi with Raspbian (the Linux OS for it) and there's half a million guides on this everywhere already, so you figure it out. Instead of using a breadboard for prototyping and then using a HAT to put everything together, I got a little 6in perf board to just solder everything onto. If you're not willing to test in production, you're not willing to engineer! (Please do not tell my job this). My original plan was to put an old IDE connector on my board and use a computer IDE cable to connect it to the Pi. So, I found an old broken disc drive for the original XBOX lying around and stole the IDE connector off of it. I hoard electronics, broken or working, for just such occasions. Turns out IDE cables are weird and my pins were getting crossed, so I eventually scrapped that idea and hardwired a floppy disk ribbon cable to my board and i use that to plug into the Pi.


Hooking Up the Sensors

Ambient Temperature, Air Pressure, and Humidity

One big mistake I made was with the BME combination temperature/humidity/pressure sensor. I stupidly soldered it directly to the board I'm building. This meant it would be going into the same weatherproof box as the Pi itself, which doesn't seem bad on the surface, but since it's a temperature sensor, the computer is going to be putting off heat as they tend to do, which would skew the temperature readings a bit. What's the good of hyperlocal weather data if it's not accurate? So, eventually, I removed it from the board, put some terminal connectors in its place, and put the BME in its only smaller weatherproof box (with a hole drilled in the bottom to allow airflow), and connected it to my board via some scrap ethernet cable I had lying around from an old remote bottle rocket launcher I built years ago (I do a variety of projects from time to time). I'm TERRIBLE at desoldering so I nearly fucked up the solder pads on the sensor, so I ordered a replacement just in case, but turns out it was fine. Might use the replacement for indoor readings?


Ground Temperature

In addition to air temperature, I also have a ground temperature sensor. Air temperature tends to fluctuate a lot, and ground temperature tends to be steady, but is significantly cooler. It's nice to see the data, but I've been taking the mean of the air and ground temperatures to derive a "true" temperature. I've found that the more consistent ground temperature kinda normalizes the wild air temperature and gives me pretty accurate values when compared to both DarkSky and NOAA's weather apps. The ground temp probe connects via terminal blocks and you just shove it into the ground outside.


Wind Speed, Wind Direction, and Rainfall

The wind sensors and rain meter all connect via standard RJ11 plugs -- the very same used on landline telephones, for those of you born in the 1900s. With a global pandemic happening and RadioShack all but not existing anymore, buying some RJ11 jacks ain't exactly easy. I could've ordered some from DigiKey, but I didn't wanna wait 4 or 5 days for them. I considered just hacking the plugs off the sensors' wires, but decided against it so that they'll be easy to disconnect if I ever move the weather station. Then I remembered my stockpile of obsolete and broken electronics. I found an old dial-up modem card for a PC in my collection and it just so happened to have two RJ11 jacks on it. Since I'm terrible at desoldering, I used my Dremel and cut them out of the circuit board because who gives a shit? I only needed two -- the wind sensors plug into each other and connect to the board with a single RJ11 plug.


So, how does the anemometer (wind speed sensor) work? It's pretty cool! So, it has three arms with little cups on them that catch the wind and make the top of the device spin, kinda like when they made pinwheels out of soda cans for the sensors in Twister. Inside the sensor is a reed switch. A reed switch is basically a small cylinder with some metal in it. When a magnet is placed near it, the metal moves and completes the circuit, like a switch. So, as the cups spin, they're moving a little magnet around. Every time the magnet passes the reed switch, it "presses" a "button" basically. So, we make the Pi listen for a button press from the anemometer (which happens on every half turn) and then we take the number of "presses" in a given time period and divide by 2 to get the number of full rotations. Then, knowing the radius of our sensor (9cm, according to the data sheet), we can calculate how fast the sensor moved those 9cm in the specified time period (5mins is what I have mine set to). Then I do some basic conversation to put that in miles per hour since our country refuses to anything the normal way. We take all the readings in that 5min period and get the mean value to get our sustained wind speed. Then we take the maximum value from those readings and that gives us our wind gust value. Note: you do need to calibrate the sensor. Luckily, other people have figured this out and you can just multiply your values by 1.18 to get accurate readings.


So, now for the wind direction, we need to get a little deeper. The inputs on the Pi are digital inputs meaning they can only have 1 of 2 values: on (1) or off (0). This is fine for things like the anemometer that just need to "press" a button on or off, but what about things like the wind vane that needs to tell us which direction it's pointing in? Well, it will give us that information in an analog signal. But, since the Pi gets input in a digital way, how can we read that? Well, you need an analog to digital convertor. I got my ADC from DigiKey. 


So, how exactly does the wind vane work anyway? Glad you asked! Remember the reed switch I mentioned in the anemometer? This fella has 8 reed switches in it. Each reed switch is connected to its own resistor and each resistor has a different value. So, by taking our known voltage we're feeding it (5v) and then calculating what the voltage is that the sensor is outputting (which will be lower depending on how many and which resistors are turned on by their respective reed switches), we can determine which resistors are active and by that, determine what direction the wind vane is pointing in since the wind vane's got a little magnet that spins around when the wind blows and if it's on one or between two reed switches, it'll close them, activating those resistors, setting the voltage to a specific amount. Then, we document what each of those possible voltage values are, and use those to convert to degrees on a circle.


And just to make things look nicer and more human readable, I convert those degrees into compass directions. I had to determine where 0º and 180º were pointing when I mounted the system on my house. I happened to put 180 pointing north, so then I knew 0 was south. Then I spun the vane to point west, got the value for that and with those three degrees, I could derive the remaining directions.


The rain meter is a lot easier. It also has a single reed switch in it. Inside the meter is a little bucket on a little hinge. When the bucket fills up with water, it tips over dumping out the rain and trips the reed switch, acting as a button press like on the anemometer. According to the data sheet for the rain meter, the bucket dumps when it has 0.2794mm of rain in it, so knowing that, we just multiply the number of bucket dumps by that and bingo bango we know how much rain has fallen in a given time period (again, 5mins for my system). Yes, that does mean if you receive less than 0.2794mm of rain in that 5min period, your system will not know because the bucket hasn't tipped yet. That sucks if you're like me and crave decimal places, but them's the breaks.

Putting It All Together

So now we've got ambient temperature, ground temperature, relative humidity, air pressure, wind speed and direction, and rainfall being measured. Time to mount the sucker in a weatherproof box and install it outside! I bought an 8x8in junction box from Lowe's to house the Pi and my board. I used velcro to attach it,  but screws would probably be best, but I just used what I had around. I also used a little 4in junction box to house the BME combination sensor. The wind instruments come with a cool T-shaped pole that you can mount to your house or a fence or even a larger pole. Whatever works for you. I suggest putting it somewhere where it won't be obstructed by another house or your roof or the fence. Somewhere high up. Rain meter obviously should have a very clear view of the sky and not have runoff from your roof affect it. I mounted mine up high on the same thing as the wind meters.


Modifying the Python Code A Little

Follow the guide to get your Python code going to collect data and store it in the SQL database. I modified the code to store a Epoch timestamp with each record to make sorting and queries easier than working with date strings. I added a column named TIMESTAMP (stupid to use that... call it something that doesn't conflict with other SQL stuff) and made it a simple FLOAT value column. Then, in the database.py file that the guide gives you, modify the insert_template on line 115 to accept the timestamp:

self.insert_template = "INSERT INTO WEATHER_MEASUREMENT (AMBIENT_TEMPERATURE, GROUND_TEMPERATURE, AIR_QUALITY, AIR_PRESSURE, HUMIDITY, WIND_DIRECTION, WIND_SPEED, WIND_GUST_SPEED, RAINFALL, TIMESTAMP, CREATED) VALUES(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s);"

Then you need to tell it to store that data. In database.py on line 130, you'll have to change the insert function to look like this:

    def insert(self, ambient_temperature, ground_temperature, air_quality, air_pressure, humidity, wind_direction, wind_speed, wind_gust_speed, rainfall, timestamp, created = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")):
        timestamp = round(timestamp, 6)
        params = ( ambient_temperature,
            ground_temperature,
            air_quality,
            air_pressure,
            humidity,
            wind_direction,
            wind_speed,
            wind_gust_speed,
            rainfall,
            timestamp,
            created )
        print(self.insert_template % params)
        self.db.execute(self.insert_template, params)

Then, in your main loop in your weather_station_byo.py file, set the timestamp variable to the current time with timestamp = time.time() and then pass the timestamp variable into the db.insert() call in the loop after rainfall and before created. And now you'll have a simple integer to deal with for your timestamps to make querying a lot easier. I'm not going into detail about making this change or modifying the database schema because I'm assuming if you've made it this far, you know what you're doing.

Building a Frontend Dashboard and Deriving New Data

This means we've got our weather station up and running, bringing in data! Cool! At this point though, the only way to see the current weather conditions is to do a database query manually in a database tool or something, which... isn't very user-friendly. The good thing is, all of our data is in a MySQL (well, MariaDB technically) database, which means we can access the data from pretty much anything these days. I decided to build a simple little PHP web app to display, visualize, and query the weather data. I used a free Bootstrap "admin dashboard" template for the frontend and customized it because I love dark mode, baby!


So, you'll notice that I've got some additional data points here: estimated temperature, feels like temperature, and dew point. These values are all derived from existing data, so we don't have to store them in the database. The "Estimated Temperature" is something I came up with. As I mentioned early on, the ambient temperature fluctuates a lot and can sometimes give high readings if the sun is directly on it, etc. the ground temperature doesn't rollercoaster as much so I get the estimated temperature by adding the ambient temperature and the ground temperature and then dividing by 2, also known as the average or mean. This seems to give a more realistic temperature in my experience.

The "Feels Like" temperature is a lot more complicated. This uses the temperature, humidity, wind speed, AND constants like the rate at which moisture evaporates off the human body at 6ft above ground and a bunch of other stuff. I dug around online to find a formula for Heat Index (temps above 70) and wind chill (temps below 70) and the feels like just uses the appropriate formula depending on the ambient temperature. Here's the PHP code for the Feels Like value:

function calculateFeelsLike($temp, $rh, $wind){
  $temp_sq = $temp * $temp;
  $rh_sq = $rh * $rh;
  if($temp >= 70){
    $p_feels_like = round(-42.379 + (2.04901523 * $temp) + (10.14333127 * $rh) + (-0.22475541 * $temp * $rh) 
      + (-6.83783E-3 * $temp_sq) + (-5.481717E-2 * $rh_sq) + (1.22874E-3 * $temp_sq * $rh) + (8.5282E-4 * $temp * $rh_sq) 
      + (-1.99E-6 * $temp_sq * $rh_sq), 2);
  }else{
    $p_feels_like = round(35.74 + (0.6215 * $temp) - (35.75 * $wind) + (0.4275 * $temp * $wind), 2);
  }

  return $p_feels_like;
}

Simple, right?! This expects the temp to be in Fahrenheit, btw. The Pi gets the temp as Celsius, so you have to convert it. That's easy:

((TEMP) x (9/5)) + 32

The Dew Point is also calculated, and kind of gives you a better idea of how muggy and gross it's gonna be than the humidity alone. Calculating dew point also takes into account a lot of stuff that we just don't have the data for like moisture readings, etc. But luckily, there's a shortcut that's a little less accurate but accurate for our purposes. This just takes the temperature and relative humidity and a bunch of constants way smarter people figured out for us:

243.04 x (ln(HUMIDITY / 100) + ((17.625 x TEMP) / (243.04 + TEMP))) / (17.625 - ln(HUMIDITY / 100) - ((17.625 x TEMP) / (243.04 + TEMP)))

And boom, you've got your dew point in Fahrenheit. Yes, it's also ridiculous. I haven't used natural log in real life maybe ever.

The rain value in the Right Now section on my dashboard is the sum of all the rain values since 12:00:00am of the current day. That's a simple SQL query:

SELECT SUM(RAINFALL) AS TOTAL FROM WEATHER_MEASUREMENT WHERE `TIMESTAMP` >= [integer timestamp for midnight this morning]

Depending on the language you're using, getting the midnight stamp varies. In PHP is just this little dude: strtotime('00:00:00 today') which is cool.

I'm a web developer by trade, so I went further with this dashboard, creating graphs and stuff for each data point, that can be viewed by predetermined time frames like "Last 12 hours" or "Yesterday", or a custom date range. Here's a few:




So yeah, once you have the data in the database, you can do pretty much anything with it, limited only by your imagination and coding skills! I might release this frontend as an open source project at some point. I installed Apache and PHP on the Raspberry Pi running the weather station Python code and I run this frontend directly off the Raspberry Pi, so it's only accessible when I'm on my WiFi, though I did do some port forwarding in my router and setup a dynamic DNS via DuckDNS so I can access it when I'm away from home. There's guides on setting that up and is a tad out of the scope here.

Now that we have our cool dashboard, it'd be fun to let other people see your weather data! I'm not comfortable with giving people the link to view the dashboard since it's on my internal home network, so I wanted a way to put data out there without giving people access to my network, which aside from being dangerous from a hacking or DDoS standpoint, but also it's against almost every ISP's terms of service to run a public webserver from home, so I'd prefer them not shut off my internet or fine me. So, to make this happen, I decided to make the weather station tweet!

Making It Tweet

First thing you'll have to do is go to the Twitter Developer portal and register for a developer account. They don't just instantly give you one anymore -- you have to request an account and it takes a few days for them to review and get back to you which sucks ass. But, they granted me one by explaining exactly the type of account, that it's a bot that tweets the weather from my custom-built weather station for personal use, etc. Once you have an account, create an app in the portal, and then generate Access tokens for your account via the portal. It's good to create a new account for all of this so your personal account isn't tweeting the weather out. Copy the access key and secret and save them on your computer -- they won't show them to you again.

On the Pi, you'll wanna install a Twitter library for Python to make this easier for us. So, on the command line, run pip install python-twitter and we're ready to get coding. If you don't have pip installed, here's a quick guide on installing it.

I abstracted out all the twitter stuff into its own file to keep things clean. So make a new file called tweet.py (do not name it twitter.py! It will confuse things when you try to import python-twitter. I found this out the hard way). And make your twitter.py look like this:

import twitter
import datetime
import decimal
import math

api = twitter.Api(consumer_key="YOUR TWITTER APP ID",
                  consumer_secret="YOUR TWITTER APP SECRET",
                  access_token_key="YOUR TWITTER ACCOUNT ACCESS KEY",
                  access_token_secret="YOUR TWITTER ACCOUNT SECRET")

def cToF(temp):
    return round((float(temp) * float(9/5)) + float(32), 2)

def kphToMph(speed):
    return round(float(speed) / float(1.609), 2)

def degToDirection(deg):
    if deg >= 0 and deg < 22.5:
        return "S"
    elif deg >= 22.5 and deg < 45:
        return "SSE"
    elif deg >= 45 and deg < 67.5:
        return "SE"
    elif deg >= 67.5 and deg < 90:
        return "ESE"
    elif deg >= 90 and deg < 112.5:
        return "E"
    elif deg >= 112.5 and deg < 135:
        return "ENE"
    elif deg >= 135 and deg < 157.5:
        return "NE"
    elif deg >= 157.5 and deg < 180:
        return "NNE"
    elif deg >= 180 and deg < 202.5:
        return "N"
    elif deg >= 202.5 and deg < 225:
        return "NNW"
    elif deg >= 225 and deg < 247.5:
        return "NW"
    elif deg >= 247.5 and deg < 270:
        return "WNW"
    elif deg >= 270 and deg < 292.5:
        return "W"
    elif deg >= 292.5 and deg < 315:
        return "WSW"
    elif deg >= 315 and deg < 337.5:
        return "SW"
    elif deg >= 337.5:
        return "SSW"

def calculateFeelsLike(temp, rh, wind):
    temp_sq = float(temp * temp)
    rh_sq = float(rh * rh)
    temp = float(temp)
    rh = float(rh)
    wind = float(wind)

    if temp >= 70:
        feels_like = round(float(-42.379) + (float(2.04901523) * temp) + (float(10.14333127) * rh) + (float(-0.22475541) * temp * rh) + (float(-6.83783E-3) * temp_sq) + (float(-5.481717E-2) * rh_sq) + (float(1.22874E-3) * temp_sq * rh) + (float(8.5282E-4) * temp * rh_sq) + (float(-1.99E-6) * temp_sq * rh_sq), 2)
    else:
        feels_like = round(float(35.74) + (float(0.6215) * temp) - (float(35.75) * wind) + (float(0.4275) * temp * wind), 2)

    return feels_like

def calculateDewPoint(t, rh):
    t = float(t)
    rh = float(rh)
    return round(float(243.04) * (float(math.log(rh / float(100))) + ((float(17.625) * t) / (float(243.04) + t))) / (float(17.625) - float(math.log(rh / float(100))) - ((float(17.625) * t) / (float(243.04) + t))), 2)

def postTweet(temp, ground_temp, humidity, pressure, wind_speed, wind_direction, rain_data):
    now = datetime.datetime.now()
    rain = round(float(rain_data["TOTAL"]) / float(25.4), 2)
    est_temp = cToF((temp + ground_temp) / 2)
    
    tweet = now.strftime("%m/%d/%Y %I:%M%p") + "nn"
    tweet = tweet + "Temp: " + str(est_temp) + "°n"
    tweet = tweet + "Feels Like: " + str(calculateFeelsLike(est_temp, humidity, wind_speed)) + "°n"
    tweet = tweet + "Humidity: " + str(round(humidity, 2)) + "%n"
    tweet = tweet + "Pressure: " + str(round(pressure, 2)) + "mbn"
    tweet = tweet + "Wind: " + str(kphToMph(wind_speed)) + "mph " + degToDirection(wind_direction) + "n"
    tweet = tweet + "Dew Point: " + str(calculateDewPoint(est_temp, humidity)) + "°n"
    tweet = tweet + "Rain Today: " + str(rain) + "in"

    api.PostUpdate(tweet)
    print(tweet)

You can change the format of the Tweet to however you like. You'll also have to adjust the wind direction function based off of how your wind vane is positioned. Unless you pointed yours the exact same way I did mine. Now that we have our tweet.py file ready, let's modify weather_station_byo.py to make it tweet.

Le's make our station use our tweet module. Add these lines to he top of the file by the other import statements:

import tweet
from datetime import date
from datetime import datetime

Now, the main loop stores data in the database every 5 minutes, but we don't want our bot to tweet that often. I wanted mine to tweet every 30mins. So at the top of the file where we defined some other variables, add:

tweet_interval = (30 * 60) / interval
interval_count = 0

What this does is figures out, based on our main loops interval time (5 mins) how many intervals would happen in 30 minutes. That would give us 6, meaning the loop will happen 6 times in the span of 30 minutes. We need to keep track of how many times the loop has fired, so we create a counter and initialize it to 0. Since we know we'll get 6 loops in our desired 30 minutes, we'll make the station post a tweet when we've done the loop 6 times, reset the counter back to 0, and wait for the 6th go to come around again before tweeting again, giving us a single tweet every 30mins (or 6 loops) instead of tweeting every 5mins! Here's where we do that in the loop:

    if interval_count >= tweet_interval:
        now = time.time()
        today = date.today()
        midnight = datetime.combine(today, datetime.min.time()).timestamp()
        rain_results = rawdb.query("SELECT SUM(RAINFALL) AS TOTAL FROM WEATHER_MEASUREMENT WHERE `TIMESTAMP` >= " + str(midnight) + " AND `TIMESTAMP` <= " + str(now))
        rain_data = rain_results[0]
        interval_count = 0
        tweet.postTweet(ambient_temp, ground_temp, humidity, pressure, wind_speed, wind_average, rain_data)

So, what happens here is at the end of every cycle of the loop, after the database has been updated, we check to see if the number of cycles we've completed is equal to or greater than the tweet_interval. Why also great then? In some cases, your tweet interval might no be a whole number. For instance, if you want to tweet every 30mins, but your main loop interval is 7 minutes, your tweet_interval would be roughly 4.28, so what would happen here is we would tweet on the 5th loop since 5 is greater than 4 and our interval_count will go from 4 to 5 and never be equal to 4.28. You could also round the tweet_interval value to a whole number up or down, but this works if you're not looking for precise tweet times.

We're also just piping in the variables we've set all the weather values to to store in the database, but, for the rainfall, we want total rain since midnight, so we do a little DB query here to get the total rain for today rather than just showing the rain accumulated in the last 5 minutes, which doesn't tell you much other than that it is currently raining.

So there you have it! That's how I customized my weather station based off the Oracle guide. I've ordered a lightning strike detector that I'll be adding soon and I'll update this post with that. It tells you when a lightning strike happens within 40km, how far away it was, and how intense it was. looking forward to playing with that! i'm also adding a USB webcam to it so I can look at the weather when I'm away, and I'd like to record snapshots from when big wind gusts occur, or the rain intensifies, or lightning strikes happen, etc.

I'm happy to help if you run into any walls or have questions! Follow me on Twitter at @animatedGeoff and follow my weather bot at @70122weather.


« 13 Ways to Prepare for Hurricanes
May 31, 2019
View or Post Comments...