At the top of this page, you may have seen the weather station link, showing up-to-date temperature, pressure and more. The page updates automatically every 10 minutes, 24/7. The weather station was installed 31. Jan 2010 taking readings every 10 minutes. Beginning 04 Jan 2012, the frequency of readings was increased to every 5 minutes, and it has been doing that since then, almost without any pause. 31 January 2015 will mark the 5th anniversary of the weather station and the associated web page, pumping out graphs like this:
On 03 Sep 2014, my new Raspberry PI based weather camera became operational, it has been capturing images every minute 24/7 since then, adding visual context to the other data. It is easier to understand the causes of the temperature fluctuations when you see what the weather actually looked like at the time. The weather camera construction and setup will be subject of later post(s) here.
The database for the weather station has collected only 24MB of data since 2010, but capturing images is something completely different, even though the weather camera is set up to save images at half resolution and as JPEG with high compression. Keeping all those images as individual files in the file system, eventually causes trouble. It is also cumbersome to review old images that way. Therefore, I have started developing a database & viewer for the images and other data produced by the weather camera. Eventually, this will be merged with the weather station data, but first we need to handle the substantial volume of image data coming from the camera.
As a first test of some ideas, I created a prototype database and viewer application to see how responsive it would be. I took all images from October 2014 – more than 44 000 individual images! – and put them all in the same database, resulting in a file size of 2.95GB. Here is a small glimpse of the responsiveness, which I find quite adequate.
Even though this appears to be working fine, I would like to find ways of compressing the data more, without compromising on the final image quality, so there will be more tests before I settle down on the “final system”. I am thinking of presenting temperature or other graphs alongside the images, to give better overview. The image database is based on SQLite, but I have added my own C++ layer on top of it, which makes it more suitable for object orientation. This layer may become open source at some stage, if it turns out the way I want it.
The next video below, isn’t really about the database as such, but it illustrates with better image quality what the weather camera can do. Of course, producing such time lapse videos from the database is a possible goal.
If you are interested in any of this, and want to know more detail, tell me about it!