Divable conditions are mainly a factor of swell, wind and wind direction. With Python and d3, we can visualise the week ahead.
Vis.Report is a website that allows divers to share visibility reports - a measure of underwater site conditions.
In a previous blog post, I have described how I plan to use this data to predict site visibility. Over the last two years, divers have contributed over 400 reports. We are making good progress on the amount of data needed to resolve the fine details!
In the meantime (and to encourage more reports) I have added a weather forecast function. One site I use often is Seabreeze, which shows a simple wind and swell forecast with colour coding for wind speed. I have always wanted a version fine-tuned by dive site, and that can also take into account wind direction - which is often more important than wind speed (more on that in a moment).
This post describes the mechanics of that system, which is used by Vis.Report to produce interactive graphs for the week ahead. If wind (arrows) and swell (area) are both green - you should have a pleasant dive.
A popular snorkel site in Perth is the Mettam’s Pool in Marmion (https://vis.report/MET). It is a relatively well-protected shallow reef, with a huge variety of marine life.
library(leaflet)
leaflet(options = leafletOptions(minZoom = 10, maxZoom = 13)) %>%
setView(lng=115.747833, lat=-31.963271, zoom = 10) %>%
addProviderTiles(providers$CartoDB.Voyager) %>%
addCircleMarkers(opacity=0, fillOpacity=0,
lng=115.747833, lat=-31.863271,
label="Mettam's Pool",
labelOptions = labelOptions(noHide = T, textsize = "15px"))
Access to the site, and underwater conditions are very weather dependent.
Ideally you are looking for swell less than 1m, so that you can get into the water without being knocked over by a wave. You are also hoping for light easterly winds, which come off the land. On the other hand, a strong westerly (say above 10 knots) will create wind waves on the ocean’s surface, making it choppy and reducing underwater visibility. This can be represented using a wind rose, where green shows ideal conditions, and red shows poor conditions.
This wind information can be represented formally as a table, using a column for wind angle (degrees), wind speed (knots), and a score (0 = ideal, 1 = marginal, 2 = poor).
The Javascript code block below shows how this table can be turned into the wind rose above using the d3 visualisation library.1
var width = 330
= 330
height = 5
margin
var radius = Math.min(width, height) / 2 - margin
//select windrose div and add graph spaces
var svg = d3.select("#windrose")
.append("svg")
.attr("width", width)
.attr("height", height)
.append("g")
.attr("transform", "translate(" + width / 2 + "," + height / 2 + ")");
//read in the conditions table
var wind_data = d3.csv("wind_csv.csv", d3.autoType).then((data) => {
//define colours
var color = d3.scaleOrdinal()
.domain([0, 1, 2])
.range(["#5cb85c", "#f0ad4e", "#d9534f"]);
//build the arcs for wind direction
var arc = d3.arc()
.innerRadius(function (d) { return d.wind * radius / 20 })
.outerRadius(function (d) { return (d.wind + 5) * radius / 20 })
.startAngle(function (d) { return (d.angle - 22.5) * (Math.PI / 180) })
.endAngle(function (d) { return (d.angle + 22.5) * (Math.PI / 180) })
//define update score function for click event
var updatescore = function (d, i) {
.score = (i.score == 2 ? 0 : i.score + 1)
i
.select(this)
d3.style("fill", function (d) { return color(i.score); });
;
}
//colour each arc by score
.append('g')
svg.selectAll("path")
.data(data)
.enter()
.append("path")
.attr("d", function (d) { return arc(d); })
.style("fill", function (d) { return color(d.score) })
.style("stroke", '#EAECEE')
.on("click", updatescore)
//add in the labels
svg.selectAll('labels')
.data(data)
.enter()
.append('text')
.attr("font-size", 14)
.attr("pointer-events", "none")
.text(function (d) { return d.label })
.attr('fill', 'white')
.attr('transform', function (d) {
var pos = arc.centroid(d)
return 'translate(' + pos + ')';
})
.catch((error) => {
})console.error("Error loading the data");
; })
Vis.Report is built in Django, which is a Python web framework. Most of this post is written in Python, which means it can be run directly in a Django views (this is how it is implemented in Vis.Report). For the full code, visit the GitHub repository for this post or the repository for Vis.Report.2
# needs virtual environment
# python3 -m venv .venv
# source .venv/bin/activate
# pip install -r requirements.txt
Sys.setenv(RETICULATE_PYTHON = paste0(getwd(), "/.venv/bin/python"))
library(reticulate)
WILLYWEATHER_API = Sys.getenv("WILLYWEATHER_API")
In Australia, the best API for coastal weather is WillyWeather. For just over a 100 requests per day it works out to cost about a dollar a year. Once you have an API key, querying it with the Python requests library is easy:
Get the nearest location in their database (docs)
Get the forecast (docs)
Check this has swell as well as wind, and if it doesn’t choose the closest that does docs
Pull the swell and wind data out of the json responses using
pd.json_normalize
Combine them into a table with pd.merge
(remembering
the fill the NAs produced by the 5-7 day swell forecast being
2-hourly)
import pandas as pd
import numpy as np
import arrow
import requests
= r.WILLYWEATHER_API
api_key # or os.getenv('WILLYWEATHER_API')
# coordinates for Mettam's Pool
= -31.963271
lat = 115.747833
lon
# Willyweather search api endpoint
= f'https://api.willyweather.com.au/v2/{api_key}/search.json'
search_url
# Find closest point
= requests.get(
search
search_url,={
params'lat': lat,
'lng': lon,
"range": 20,
"distance": "km"
},
)
# Get the station id for next step
id = search.json()['location']['id']
# Willyweather forecast api endpoint
= f'https://api.willyweather.com.au/v2/{api_key}/locations/{id}/weather.json'
forecast_url
# Get forecast
= requests.get(
forecast
forecast_url,={
params'forecasts': 'wind,swell',
'days': 7,
},
)
# Read the forecast response
= forecast.json()
weather_json
# Check for swell data (not all sites has it)
# if no swell data, find the closest location that has it
if not weather_json['forecasts']['swell']:
= f'https://api.willyweather.com.au/v2/{api_key}/search/closest.json'
close_url = requests.get(
close
close_url,={
params'id':id,
'weatherTypes':"swell",
"distance": "km"
},
)
id = close.json()['swell'][0]['id']
= f'https://api.willyweather.com.au/v2/{api_key}/locations/{id}/weather.json'
forecast_url = arrow.now().floor('day').format('YYYY-MM-DD')
startDate
= requests.get(
forecast
forecast_url,={
params'forecasts': 'wind,swell',
'days': 7,
'startDate': startDate, # Convert to UTC timestamp
},
)= forecast.json()
weather_json
# Convert wind json to pandas dataframe
= pd.json_normalize(weather_json['forecasts']['wind']['days'],
wind_data ='entries'
record_path
).rename(={"speed": "wind", "direction": "wind_dir",
columns"directionText": "wind_dir_text"})
# Convert swell json to pandas dataframe
= pd.json_normalize(
swell_data 'forecasts']['swell']['days'],
weather_json[='entries'
record_path'period', 'direction'], axis=1
).drop([={"height": "swell", "directionText": "swell_dir_text"})
).rename(columns
# Merge together and fill
= pd.merge(
weather
wind_data,="left", on=['dateTime']
swell_data, how="ffill")
).fillna(method
# Show table
weather.head()
dateTime wind wind_dir wind_dir_text swell_dir_text swell
0 2022-04-16 00:00:00 28.8 169 S WSW 1.3
1 2022-04-16 01:00:00 29.5 168 SSE WSW 1.3
2 2022-04-16 02:00:00 31.3 165 SSE WSW 1.3
3 2022-04-16 03:00:00 30.6 168 SSE WSW 1.3
4 2022-04-16 04:00:00 31.7 169 S WSW 1.3
Scoring the forecast involves a join to the wind table, and a function for swell. On the forecast table, wind speed is rounded down to the nearest 5 knots, and wind angle is rounded to the nearest 45 degrees. This table is then left-joined to the wind condition table. A function for swell defining marginal (1.2m for Mettam’s) and maximum (1.8m) is applied to the swell height. These are both combined with the original forecast and saved as a csv, for passing to the graphing function.3
# Read the conditions table to pandas
= pd.read_csv('wind_csv.csv').drop('label', axis=1)
con
# function to limit rounded wind speed to 15 knots
def clamp(n, maxn=15):
return max(min(maxn, n), 0)
# round wind speed to nearest 5kn
# round wind angle to nearest 45 degrees
# this is so it can be joined to the score table
= pd.DataFrame({
rounded 'wind': pd.to_numeric(
5*((weather['wind']*0.54/5
apply(np.floor)), downcast='integer'
).apply(clamp),
).'angle': pd.to_numeric(
45*((weather['wind_dir']/45).apply(np.round)),
='integer').replace([360],0),
downcast
})
# join rounded wind values to score table
= pd.merge(
wind
rounded,="left", on=["angle", "wind"]
con, how'score': 'wind_score'}, axis=1)
).rename({
# function to score swell
# if it is over marginal, it gets a score of 1
# if it is over bad, it gets a score of 2
def swell_calc(swell, marginal, bad):
if swell <= marginal:
= 0
score elif swell <= bad:
= 1
score else:
= 2
score return score
# score swell
= weather['swell'].apply(
swell =1.2, bad = 1.8)
swell_calc, marginal
# function to cap total score (wind + swell) to 2
def cap(n):
return min(n, 2)
# construct table of conditions and scores
= pd.DataFrame({
scores 'time': weather['dateTime'],
'swell': weather['swell'],
'swell_dir_text' : weather['swell_dir_text'],
'wind': (weather['wind']*0.54).round(decimals = 1),
'wind_dir': weather['wind_dir'],
'wind_dir_text' : weather['wind_dir_text'],
'swell_score': swell,
'wind_score': wind['wind_score'],
'total_score': (swell+wind['wind_score']).apply(cap),
})
# csv to be passed to d3
'scores.csv')
scores.to_csv(
scores.head()
time swell ... wind_score total_score
0 2022-04-16 00:00:00 1.3 ... 2 2
1 2022-04-16 01:00:00 1.3 ... 2 2
2 2022-04-16 02:00:00 1.3 ... 2 2
3 2022-04-16 03:00:00 1.3 ... 2 2
4 2022-04-16 04:00:00 1.3 ... 2 2
[5 rows x 9 columns]
This table of conditions and their scores can be passed back to d3 for plotting. This plot shows wind as arrows, with y = speed in knots and pointing in the direction of wind flow. It shows swell as an area with y = height in metres. Both of these are coloured by their scores. This summarises the week, with ideal diving conditions shown in green, marginal days in yellow, and poor days in red.
The CSS:
.grid line {
lightgrey;
stroke: 0.7;
stroke-opacity: ;
shape-rendering: crispEdges
}
.grid path {
0;
stroke-width:
}
.line {
url(#swell_gradient);
stroke: 1.5;
stroke-width: fill : url(#swell_gradient);
opacity : 0.4;
}
And the Javascript:
// set the dimensions and margins of the graph
var margin_weather = { top: 10, right: 40, bottom: 30, left: 60 },
= 700 - margin_weather.left - margin_weather.right,
width_weather = 200 - margin_weather.top - margin_weather.bottom;
height_weather
// append the svg object to the body of the page
var svg_weather = d3.select("#weather")
.append("svg")
.attr("width", width_weather + margin_weather.left + margin_weather.right)
.attr("height", height_weather + margin_weather.top + margin_weather.bottom)
.append("g")
.attr("transform",
"translate(" + margin_weather.left + "," + margin_weather.top + ")");
//read in csv
var wind_data = d3.csv("scores.csv").then((data) => {
//parse the time
var parseTime = d3.timeParse("%Y-%m-%d %H:%M:%S");
var dates = [];
for (let obj of data) {
.push(parseTime(obj.time));
dates
}
//setup x axis
var domain = d3.extent(dates);
function make_x_gridlines() {
return d3.axisBottom(x)
.ticks(10)
}
var x = d3.scaleTime()
.domain(domain)
.range([0, width_weather]);
.append("g")
svg_weather.attr("transform", "translate(0," + height_weather + ")")
.call(d3.axisBottom(x));
.append("g")
svg_weather.attr("class", "grid")
.attr("transform", "translate(0," + height_weather + ")")
.call(make_x_gridlines()
.tickSize(-height_weather)
.tickFormat("")
)
.append("text")
svg_weather.attr("class", "y label")
.attr("text-anchor", "end")
.style("font-size", "14px")
.attr("y", 6)
.attr("dy", "-2.2em")
.attr("transform", "rotate(-90)")
.text("wind (knots)");
//setup dual y axis
function make_y_gridlines() {
return d3.axisLeft(y)
.ticks(5)
}
var y = d3.scaleLinear()
.domain([0, 30])
.range([height_weather, 0]);
.append("g")
svg_weather.call(d3.axisLeft(y)
.ticks(5));
var y1 = d3.scaleLinear()
.domain([0, 6])
.range([height_weather, 0]);
.append("g")
svg_weather.attr("class", "grid")
.call(make_y_gridlines()
.tickSize(-width_weather)
.tickFormat("")
)
.append("g")
svg_weather.attr("transform", "translate(600,0)")
.call(d3.axisRight(y1).ticks(5));
.append("text")
svg_weather.attr("class", "y label")
.attr("text-anchor", "end")
.style("font-size", "14px")
.attr("y", 6)
.attr("dy", "590")
.attr("transform", "rotate(-90)")
.text("swell (m)");
//define score colors
var wind_colours = d3.scaleOrdinal()
.domain([0, 1, 2])
.range(['green', 'orange', 'red']);
//setup swell color gradient (see also css)
.append("linearGradient")
svg_weather.attr("id", "swell_gradient")
.attr("gradientUnits", "userSpaceOnUse")
.attr("x1", 0)
.attr("x2", width_weather)
.selectAll("stop")
.data(data)
.join("stop")
.attr("offset", d => x(parseTime(d.time)) / width_weather)
.attr("stop-color", d => wind_colours(parseInt(d.swell_score)));
.append("path")
svg_weather.datum(data)
.attr("class", "line")
.attr("stroke-width", 1.5)
.attr("d", d3.area()
.x(function (d) { return x(parseTime(d.time)) })
.y0(y(0))
.y1(function (d) { return y(d.swell * 5) })
)
//add in arrows
.select("#weather")
d3.select('g')
.selectAll('path')
.data(data)
.enter()
.append('path')
.attr('transform', function (d) {
return 'translate(' + x(parseTime(d.time)) + ',' + y(d.wind) + ' ) rotate(' + d.wind_dir + ')';
}).attr('d', "M2.2 0 0 6.6 0 6.6 0 6.6-2.2 0-5.5-6.6 0-4.4 5.5-6.6Z")
.style("fill", function (d) { return wind_colours(parseInt(d.wind_score)); })
.style("opacity", 0.7)
//add tooltip div
var Tooltip = d3.select("#weather")
.append("div")
.style("opacity", 0)
.style("position", 'absolute')
.attr("class", "tooltip text-center")
.style("background-color", "white")
.style("border", "solid")
.style("height", "0px")
.style("border-width", "1px")
.style("border-color", "#343a40")
.style("border-radius", "5px")
.style("padding", "6px")
.style("top", "0px")
.style("left", "0px")
const formatTime = d3.timeFormat("%a %I:00 %p");
//define mouseover functions to create tooltip
var mouseover = function (event, a) {
Tooltip.style("opacity", .90)
.style("min-width", "130px")
.style("max-width", "160px")
.style("height", "80px")
.html("<b>" + formatTime(parseTime(a.time)) +
'</b><span style="color:' + wind_colours(parseInt(a.wind_score)) + '">' +
'<br>Wind: ' + a.wind + 'kn ' + a.wind_dir_text +
'</span><span style="color:' + wind_colours(parseInt(a.swell_score)) + '">' +
'<br>Swell: ' + a.swell + 'm ' + a.swell_dir_text + '</span>')
.style("top", event.pageY - 100 + "px")
.style("left", event.pageX - 60 + "px")
.select(event.currentTarget)
d3.style("stroke", wind_colours(parseInt(a.total_score)))
.style("fill", wind_colours(parseInt(a.total_score)))
.style("opacity", .25)
}var mouseleave = function (d) {
Tooltip.style("opacity", 0)
.select(this)
d3.style("stroke", "none")
.style("fill", "none")
.style("opacity", 0.8)
}
//create boxes for each hour to catch tooltip
.append("g")
svg_weather.attr("fill", "None")
.attr("pointer-events", "all")
.selectAll("rect")
.data(d3.pairs(data))
.join("rect")
.attr("x", ([a, b]) => x(parseTime(a.time)))
.attr("height", height)
.attr("width", ([a, b]) => x(parseTime(b.time)) - x(parseTime(a.time)))
.on("mouseover", (event, [a]) => mouseover(event, a))
.on("mouseout", mouseleave)
})
On Vis.Report, this is saved into the database for each site. Notice how you can click the wind rose to change its colour. You are able to post this modified table back to to the database using a button with the id=“wind_button”, and a custom onsubmit event for the form: document.getElementById(‘wind_button’).value = d3.csvFormat(d3.selectAll(‘path’).data());↩︎
The back-end python code is mostly in this views file. This front-end graphing is in the templates.↩︎
On Vis.Report, this is a csv endpoint for each site. Each time it is called, it calculates the scores against the weather forecast. If the weather forecast has been refreshed in the last 3 hours it just pulls it from the database. Otherwise, it updates before scoring. See for Mettam’s Pool.↩︎
If you see mistakes or want to suggest changes, please create an issue on the source repository.
For attribution, please cite this work as
Morrison (2022, April 16). Patrick Morrison: Visualising weather forecasts for diving and snorkelling. Retrieved from https://padmorrison.com/posts/2022-04-16-visualising-weather-forecasts-for-diving-and-snorkelling/
BibTeX citation
@misc{morrison2022visualising, author = {Morrison, Patrick}, title = {Patrick Morrison: Visualising weather forecasts for diving and snorkelling}, url = {https://padmorrison.com/posts/2022-04-16-visualising-weather-forecasts-for-diving-and-snorkelling/}, year = {2022} }