Index of /rawdata

Name	Last modified	Size

Parent Directory		-
shiplake-20210801.xml.gz	2021-08-01 00:30	3.1M
shiplake-20211001.xml.gz	2021-10-01 00:30	3.0M
shiplake-20210701.xml.gz	2021-07-01 00:30	3.0M
shiplake-20210901.xml.gz	2021-09-01 00:30	3.0M
shiplake-20210601.xml.gz	2021-06-01 00:30	3.0M
shiplake-20210501.xml.gz	2021-05-01 00:30	3.0M
shiplake-20210401.xml.gz	2021-04-01 00:30	3.0M
shiplake-20210301.xml.gz	2021-03-01 00:30	3.0M
shiplake-20210201.xml.gz	2021-02-01 00:30	3.0M
shiplake-20200901.xml.gz	2020-09-01 00:30	2.9M
shiplake-20200801.xml.gz	2020-08-01 00:30	2.9M
shiplake-20201001.xml.gz	2020-10-01 00:30	2.9M
shiplake-20200601.xml.gz	2020-06-01 00:30	2.9M
shiplake-20201201.xml.gz	2020-12-01 00:30	2.9M
shiplake-20200701.xml.gz	2020-07-01 00:30	2.9M
shiplake-20200401.xml.gz	2020-04-01 00:30	2.9M
shiplake-20210101.xml.gz	2021-01-01 00:30	2.9M
shiplake-20201101.xml.gz	2020-11-01 00:30	2.9M
shiplake-20200501.xml.gz	2020-05-01 00:30	2.9M
shiplake-20200301.xml.gz	2020-03-01 00:30	2.9M
shiplake-20190601.xml.gz	2019-06-01 00:30	2.8M
shiplake-20200101.xml.gz	2020-01-01 00:30	2.8M
shiplake-20190801.xml.gz	2019-08-01 00:30	2.8M
shiplake-20190701.xml.gz	2019-07-01 00:30	2.8M
shiplake-20190901.xml.gz	2019-09-01 00:30	2.8M
shiplake-20191201.xml.gz	2019-12-01 00:30	2.8M
shiplake-20191001.xml.gz	2019-10-01 00:30	2.8M
shiplake-20200201.xml.gz	2020-02-01 00:30	2.8M
shiplake-20190501.xml.gz	2019-05-01 00:30	2.8M
shiplake-20191101.xml.gz	2019-11-01 00:30	2.8M
shiplake-20190401.xml.gz	2019-04-01 00:30	2.8M
shiplake-20190301.xml.gz	2019-03-01 00:30	2.7M
shiplake-20181001.xml.gz	2018-10-01 00:30	2.7M
shiplake-20180801.xml.gz	2018-08-01 00:30	2.7M
shiplake-20180901.xml.gz	2018-09-01 00:30	2.7M
shiplake-20181101.xml.gz	2018-11-01 00:30	2.7M
shiplake-20180701.xml.gz	2018-07-01 00:30	2.7M
shiplake-20190201.xml.gz	2019-02-01 00:30	2.7M
shiplake-20181201.xml.gz	2018-12-01 00:30	2.6M
shiplake-20180601.xml.gz	2018-06-01 00:30	2.6M
shiplake-20190101.xml.gz	2019-01-01 00:30	2.6M
shiplake-20170701.xml.gz	2017-07-01 00:30	2.6M
shiplake-20180501.xml.gz	2018-05-01 00:30	2.6M
shiplake-20180401.xml.gz	2018-04-01 00:30	2.5M
shiplake-20170801.xml.gz	2017-08-01 00:30	2.5M
shiplake-20180301.xml.gz	2018-03-01 00:30	2.5M
shiplake-20170901.xml.gz	2017-09-01 00:30	2.5M
shiplake-20180201.xml.gz	2018-02-01 00:30	2.5M
shiplake-20171001.xml.gz	2017-10-01 00:30	2.5M
shiplake-20180101.xml.gz	2018-01-01 00:30	2.5M
shiplake-20171201.xml.gz	2017-12-01 00:30	2.5M
shiplake-20171101.xml.gz	2017-11-01 00:30	2.5M
shiplake-20170322.xml.gz	2017-03-22 18:24	2.4M
shiplake-20170601.xml.gz	2017-06-01 00:30	2.4M
shiplake-20170511.xml.gz	2017-05-11 16:25	2.4M
shiplake-20160119.xml.gz	2017-03-22 18:28	2.4M
shiplake-6h-averages-20211001.csv	2021-10-01 00:30	1.1M
shiplake-6h-averages-20210901.csv	2021-09-01 00:30	1.1M
shiplake-6h-averages-20210801.csv	2021-08-01 00:30	1.1M
shiplake-6h-averages-20210701.csv	2021-07-01 00:30	1.0M
shiplake-6h-averages-20210601.csv	2021-06-01 00:30	1.0M
shiplake-6h-averages-20210501.csv	2021-05-01 00:30	1.0M
shiplake-6h-averages-20210401.csv	2021-04-01 00:30	1.0M
shiplake-6h-averages-20210301.csv	2021-03-01 00:30	1.0M
shiplake-6h-averages-20210201.csv	2021-02-01 00:30	1.0M
shiplake-6h-averages-20210101.csv	2021-01-01 00:30	1.0M
shiplake-6h-averages-20201201.csv	2020-12-01 00:30	1.0M
shiplake-6h-averages-20201101.csv	2020-11-01 00:30	1.0M
shiplake-6h-averages-20201001.csv	2020-10-01 00:30	1.0M
shiplake-6h-averages-20200901.csv	2020-09-01 00:30	1.0M
shiplake-6h-averages-20200401.csv	2020-04-01 00:30	1.0M
shiplake-6h-averages-20200801.csv	2020-08-01 00:30	1.0M
shiplake-6h-averages-20200701.csv	2020-07-01 00:30	1.0M
shiplake-6h-averages-20200601.csv	2020-06-01 00:30	1.0M
shiplake-6h-averages-20200501.csv	2020-05-01 00:30	1.0M
shiplake-6h-averages-20200301.csv	2020-03-01 00:30	1.0M
shiplake-6h-averages-20200201.csv	2020-02-01 00:30	1.0M
shiplake-6h-averages-20200101.csv	2020-01-01 00:30	1.0M
shiplake-6h-averages-20191201.csv	2019-12-01 00:30	1.0M
shiplake-6h-averages-20191102.csv	2019-11-02 18:54	1.0M
README.html	2019-11-02 19:06	3.6K

Raw data

The River Thames data logger above Shiplake Lock records data in an RRD database, which is periodically dumped to an XML file and archived here.

The data is open source and you are welcome to use it, but there are a few things that you need to be aware of:

The sensors are MCP9701A chips, which have a typical accuracy of +/- 1C and max error of -4/+2C. I have not attempted to check the calibration. The analogue-to-digital conversion is done by an ATMega328P.
The original river sensor has been erratic at times. See the bottom graph at https://dl1.findlays.net/bin/display/rivertemp - 2014, 2015, and 2016 all included some time reading below 0C! The sensor was embedded in silicone but I suspect the water got in anyway.
I replaced the river sensor in early October 2016 and it appears to be OK now. This one is in a sealed copper pipe.
The river sensor is some way below the surface - in the silt of the river bed near the bank. It is probably 1m below the surface in normal conditions - rather more in floods!
River temperature seems to follow air temperature more closely than I had originally expected. Where data is missing or dubious it will probably be OK to use a time-smoothed version of air temp.

The data is stored in RRDtool - a constant-size time-series database format. The datalogger collects one measurement every minute and averages these to produce 5-minute intervals in storage. Further averaging produces 1-hour and 6-hour intervals which are held for longer:

# Archive periods based on 300s primary data points:
#       samples/entry   time/entry      entries         archive length
#       1               5m              10000           34 days
#       12              1h              10000           416 days
#       72              6h              15000           10 years

Data is dumped from the database to XML on the first day of every month. There are a couple of older files, allowing access to more frequent readings back to about January 2015. All files contain 6-hour readings back to October 2012.
Each XML file has a header and three sections containing 5-minute, 1-hour, and 6-hour data. Each row has these items:
- Air temperature (C)
- River temperature (C)
- River level (cm)
- Min river level in the interval (cm)
- Max river level in the interval (cm)
6-hour averages are also dumped each month, and these are stored in CSV format which is much easier to handle. This output started in November 2019 but the data goes back to about Noveber 2012.