Data reflect the world and the systems that generate it. A lot of the patterns that can be identified using various analytic methods are not surprising, but quite obvious if looking at the practices producing the data. In many ways, it makes more sense to talk about rhythms and routines when considering patterns in data. The patterns are weak or strong signals, or residue or traces of the people and practices, and not isolated patterns only visible in the data. This note is part about me learning to use D3 to visualize data and part of me trying to figure out if our electricity consumption follows the annual changes in daylight.
Data on electricity consumption
In Denmark, anyone can download data on their electricity consumption from El Overblik. You can download data for your household meter with a resolution from the year, over month and day, to an hour and even every 15 minutes in some cases. We start with a dataset of my household’s electricity consumption since November 2017 per month. Some wrangling is required to reduce the object and format the data.
Show the code
// Loading our data filedata =FileAttachment("/public/data/el-consumption-days-all.csv").csv({typed:true}).then(raw => {var lat =56.16, long=10.20//We want to simplify our objects and translate the values into appropriate formats.//We keep the date and the value raw = raw.filter(o => {var str = o["Fra_dato"].substring(6,10)return o["Type"] ==="Tidsserie"&& str !="2017" })var days = {} raw = raw.map(o => {var dstr = o["Fra_dato"].replace(" 00:00:00","").split("-")//Gonna remove the leap year. if(dstr[1] ==="02"&& dstr[0] ==="29"){return }var date =newDate(`${2022}-${dstr[1]}-${dstr[0]}`)if(!days[date]){//Create object for date//Including data on the hours of the dayvar sun = SunCalc.getTimes(newDate(date), lat,long)var day_hs = sun.sunset.getHours() - sun.sunrise.getHours()//day object to return days[date] = {date: date,kwhs_min:100,kwhs_max:0,kwhs_values:[],kwhs_median:0,kwhs_q_lower:0,kwhs_q_upper:0,day_hs:day_hs} }var v =typeof o["Mængde"] =="string"?parseInt(o["Mængde"]) : o["Mængde"]//adding kwhs consumption to the value array and calculate min/max days[date].kwhs_values.push(v) days[date].kwhs_min= v < days[date].kwhs_min? v : days[date].kwhs_min days[date].kwhs_max= v > days[date].kwhs_max? v : days[date].kwhs_max })//Convert days Object to Array raw =Object.values(days)//Sorting dates because the order is messy after conversion from object to array raw.sort((a,b) => {return a["date"] > b["date"] })//Calculating medians raw = raw.map(o => { o.kwhs_median= d3.median(o.kwhs_values) o.kwhs_q_lower= d3.quantile(o.kwhs_values,0.25) o.kwhs_q_upper= d3.quantile(o.kwhs_values,0.75)return o })return raw})
Annual electricity consumption
I wanted to plot our annual consumption in a way that followed the calendar year and carried across into the new year. The annual radial visualization in figure Figure 1 below captures our energy consumption from 2018 to 2022 and daylight hours from the calendar year 2022.
Show the code
//Chart appropriated from https://observablehq.com/@d3/radial-area-chartradial_chart = {const width =900const height =900const innerRadius = width /5const outerRadius = width /2const xAxis = g => g.attr("font-family","sans-serif").attr("font-size",12).call(g => g.selectAll("g").data(x.ticks()).join("g").each((d, i) => d.id= DOM.uid("month")).call(g => g.append("path").attr("stroke","#000").attr("stroke-opacity",0.2).attr("d", d =>` M${d3.pointRadial(x(d), innerRadius)} L${d3.pointRadial(x(d), outerRadius)} `)).call(g => g.append("path").attr("id", d => d.id.id).datum(d => [d, d3.timeMonth.offset(d,1)]).attr("fill","none").attr("d", ([a, b]) =>` M${d3.pointRadial(x(a), innerRadius-15)} A${innerRadius},${innerRadius} 0,0,1 ${d3.pointRadial(x(b), innerRadius-15)} `)).call(g => g.append("text").append("textPath").attr("startOffset",2).attr("xlink:href", d => d.id.href).text(d3.timeFormat("%B"))))const yAxis = g => g.attr("text-anchor","middle").attr("font-family","sans-serif").attr("font-size",12).call(g => g.append("circle").attr("stroke","#000").attr("fill","none").attr("stroke-opacity",0.2).attr("r",y(0))).call(g => g.selectAll("g").data(y.ticks().reverse()).join("g").attr("fill","none").call(g => g.append("circle").attr("stroke","#000").attr("stroke-opacity",0.2).attr("r", y)).call(g => g.append("text").attr("y", d =>-y(d)).attr("dy","0.5em").attr("stroke","#fff").attr("stroke-width",5).text((x, i) =>`${x.toFixed(0)}${i ?"":" kwhs"}`).clone(true).attr("y", d =>y(d)).selectAll(function() {return [this,this.previousSibling]; }).clone(true).attr("fill","currentColor").attr("stroke","none")))const x = d3.scaleTime().domain([newDate(2022,0,1),newDate(2022,11,31)]).range([0,2*Math.PI])const y = d3.scaleLinear().domain([d3.min(data, d => d.kwhs_min), d3.max(data, d => d.kwhs_max)+2]).range([innerRadius, outerRadius])const area = d3.areaRadial().curve(d3.curveBasis).angle(d =>x(d.date))const line = d3.lineRadial().curve(d3.curveBasis).angle(d =>x(d.date))const svg = d3.create("svg").attr("viewBox", [-width /2,-height /2, width, height]).attr("stroke-linejoin","round").attr("stroke-linecap","round"); svg.append("path").attr("fill","lightyellow").attr("d", area.innerRadius(d =>y(0)).outerRadius(d =>y(d.day_hs)) (data)) svg.append("path").attr("fill","lightsteelblue").attr("fill-opacity",1).attr("d", area.innerRadius(d =>y(d.kwhs_min)).outerRadius(d =>y(d.kwhs_max)) (data)); svg.append("path").attr("fill","steelblue").attr("fill-opacity",1).attr("d", area.innerRadius(d =>y(d.kwhs_q_lower)).outerRadius(d =>y(d.kwhs_q_upper)) (data)); svg.append("path").attr("fill","none").attr("class","median").attr("stroke","darkblue").attr("stroke-width",1).attr("d", line.radius(d =>y(d.kwhs_median)) (data)); svg.append("g").call(xAxis); svg.append("g").call(yAxis);var legend = svg.append("g").attr("transform","translate("+(-width/2)+","+ (-height/2) +")") legend.attr("font-family","sans-serif").attr("font-size",16) legend.append("rect").attr("x",2).attr("y",0).attr("width",20).attr("height",20).attr("fill","lightsteelblue") legend.append("text").attr("x",27).attr("y",0).attr("dy","1em").text("Minimum and maximum consumption 2018 - 2022") legend.append("rect").attr("x",2).attr("y",25).attr("width",20).attr("height",20).attr("fill","steelblue") legend.append("text").attr("x",27).attr("y",25).attr("dy","1em").text("Upper and lower quartiles consumtion (kwhs)") legend.append("rect").attr("x",2).attr("y",50).attr("width",20).attr("height",20).attr("fill","darkblue") legend.append("text").attr("x",27).attr("y",50).attr("dy","1em").text("Median consumption (kwhs)") legend.append("rect").attr("x",2.5).attr("y",75).attr("width",19).attr("height",19).attr("stroke","grey").attr("fill","lightyellow") legend.append("text").attr("x",27).attr("y",75).attr("dy","1em").text("Daylight hours")return svg.node()}
Figure 1: Annual household electricity consumption and daylight hours
I did the visualisation with the hypothesis that our electricity consumption would correlate with the available daylight. With the bright summer nights, we need less light and as the days get shorter towards the winter we need more. This assumes that a significant amount of our electricity is consumed by sources that somehow map to this rhythm. Lights are one source, but indoor entertainment (gaming, streaming, television, etc.) would also increase during the winter.
It is easy to observe the rhythm of the daylight changes in Figure 1, but the correlation with energy consumption is less obvious. Or rather, the erratic nature of the energy data obscures whatever might be there. It looks like the summer months are slightly below the 10 kwhs line, with the spring and fall months sitting on the line, and the winter months moving slightly above the 10 kwhs line. Turns out that when computing the correlation coefficient (r=0.243, see below), then there is hardly any correlation between daylight and electricity consumption data.
While dissapointed that the visualisation does not show a clear pattern, this also hightligt the benefits of visualising the data. While the correlation coefficient tell me that there is no correlation between daylight and electricity consumption, the visualisation (and making it) has a lot of additional benefits:
I am learning useful information about the particular data from crafting the visualisation. Information that is useful for future experiments and visualisations.
The visualisation and data forces me to reflect on my hypothesis and look for alternative explanations
The visualisation provide hints and information useful for speculating about explanations on the relationship between consumption and annual patterns.
So, why does our annual electricity consumption not correlate with available daylight?
The hypothesis assumes that a significant amount of our electricity consumption goes to sources that decrease with an increase in available daylight and increase when it gets darker outside during the winther. That would mostly be electrical lights and indoor entertainment. First, I do not think that lights account for a significant amount of our consumption. This would go to refriguation, laundry and cooking. On top of that, most of our indoor lighting is LED based which consume only a few watts per unit per hour and our use of entertainment likely only changes marginally from summer to winther. I still play computer games and we still stream during the summer.
This pose an interesting question. We are often told that private consumers should change habits and implement various micro-initiatives as part of solutions toward the climate and energy crisis. While some initatives do work, the savings seem minimal. My guestimate is that we can properly optimize our consumption to save a couple of kilowatts per day at most. A lot of our consumption is not something we can change at the moment. We still need a freezer and refrigerator, and we need to cook meals. A lot of larger initiatives do not make economic sense. So, is it even possible to optimize household consumption in a meaningful way so it contributes to lowering the energy demand? I do not think so. Not without heavy economic incentives. I will try to explore everyday habits, theoretical savings and impact in a future post.
Show the code
simplestats =require("simple-statistics@7")// Loading our data filedays =FileAttachment("/public/data/el-consumption-days-all.csv").csv({typed:true}).then(raw => {var lat =56.16, long=10.20//We want to simplify our objects and translate the values into appropriate formats.//We keep the date and the value raw = raw.filter(o => {var str = o["Fra_dato"].substring(6,10)return o["Type"] ==="Tidsserie"&& str !="2017" }) raw = raw.map(o => {// Create a data objectvar dstr = o["Fra_dato"].replace(" 00:00:00","").split("-")var date =newDate(`${dstr[2]}-${dstr[1]}-${dstr[0]}`)//Create object for date//Including data on the hours of the dayvar sun = SunCalc.getTimes(newDate(date), lat,long)var daylight = (sun.sunset- sun.sunrise) /1000/60var v =typeof o["Mængde"] =="string"?parseInt(o["Mængde"]) : o["Mængde"]return {date: date,kwhs:v,daylight:daylight} })//Sorting dates because the order is messy after conversion from object to array raw.sort((a,b) => {return a["date"] > b["date"] })return raw})