Tutorial: Fetch and Load

Note

You can download an executable version (Jupyter Notebook) of this tutorial here.

Fetching and Loading Environmental Data¶

In this tutorial, we will use Kadlu to retrieve environmental data from online sources and load the data into numpy arrays for further processing.

The first step is to import kadlu. Note that the datetime package is used to specify dates and times

In [3]:

from datetime import datetime

import kadlu

Quick start guide¶

With Kadlu, environmental data can be downloaded and stored in one step. Here, we demonstrate how to obtain modeled surface salinity data from HYCOM for the geographic region $47^{\circ}$N to $49^{\circ}$N and $-63^{\circ}$W to $-61^{\circ}$W for the first week of January 2013.

In [4]:

# fetch and load salinity (g/kg salt in water)
salinity, lat, lon, epoch, depth = kadlu.load(
        source='hycom', var='salinity',
        south=47, west=-63, 
        north=49, east=-61, 
        bottom=0, top=0,
        start=datetime(2013, 1, 1), end=datetime(2013, 1, 7))

Note how the arguments bottom and top are both set to 0, thereby selecting only data at a depth of 0 m, i.e., at the surface.

The load function produced flattened numpy arrays, the length of which corresponds to the number of data points in the selected geographic region, depth range, and temporal window.

In [5]:

# print the first 10 values of each array
print(lat[0:10])      # latitude (degrees north)
print(lon[0:10])      # longitude (degrees west)
print(depth[0:10])    # depth (meters)
print(epoch[0:10])    # time (hours since 00:00:00 on 1 January 2000)
print(salinity[0:10]) # ocean salt content (g/kg)

[47. 47. 47. 47. 47. 47. 47. 47. 47. 47.]
[-62.96002197 -62.88000488 -62.79998779 -62.7199707  -62.64001465
 -62.55999756 -62.47998047 -62.40002441 -62.32000732 -62.23999023]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[113991. 113991. 113991. 113991. 113991. 113991. 113991. 113991. 113991.
 113991.]
[30.791 30.886 30.92  30.903 30.874 30.853 30.817 30.747 30.714 30.636]

We can use the epoch_2_dt function to convert the time values into a more human-friendly date-time format,

In [6]:

print(kadlu.epoch_2_dt(epoch[0]))

2013-01-01 15:00:00

Data sources¶

Kadlu includes functionality to load data from a variety of different data sources. For a high level overview, print the source_map:

In [7]:

print(kadlu.source_map)

    CHS   (Canadian Hydrography Service)
          load_bathymetry:          bathymetric data in Canada's waterways. metres, variable resolution 

    GEBCO (General Bathymetric Chart of the Oceans)
          load_bathymetry:          global bathymetric and topographic data. metres below sea level 

    ERA5  (Global environmental dataset from Copernicus Climate Data Store)
          load_windwaveswellheight: combined height of wind, waves, and swell. metres
          load_wavedirection:       mean wave direction, degrees
          load_waveperiod:          mean wave period, seconds
          load_wind_uv:             wind speed computed as sqrt(u^2 + v^2) / 2, where u, v are direction vectors
          load_wind_u:              wind speed coordinate U-vector, m/s
          load_wind_v:              wind speed coordinate V-vector, m/s 

    HYCOM (Hybrid Coordinate Ocean Model)
          load_salinity:            g/kg salt in water
          load_temp:                degrees celsius
          load_water_uv:            ocean current computed as sqrt(u^2 + v^2) / 2, where u, v are direction vectors
          load_water_u:             ocean current coordinate U-vector, m/s
          load_water_v:             ocean current coordinate V-vector, m/s 

    WWIII (WaveWatch Ocean Model Gen 3)
          load_wavedirection:       primary wave direction, degrees
          load_waveperiod:          primary mean wave period, seconds
          load_windwaveheight:      combined height of wind and waves, metres
          load_wind_uv:             wind speed computed as sqrt(u^2 + v^2) / 2, where u, v are direction vectors
          load_wind_u:              wind speed coordinate U-vector, m/s
          load_wind_v:              wind speed coordinate V-vector, m/s
    

for more information on a specific source, print the class:

In [11]:

print(kadlu.hycom())

Native hycom .[ab] data converted to NetCDF at the Naval
Research Laboratory, interpolated to 0.08° grid between
40°S-40°N (0.04° poleward) containing 40 z-levels.
Availability: 1994 to 2015
	https://www.hycom.org/data/glbv0pt08

function input arguments:
	(south, north, west, east, start, end, top, bottom)

class functions:
	load_salinity
	load_temp
	load_water_u
	load_water_uv
	load_water_v

keyword arguments can be passed as a dictionary when using the same load arguments for multiple datatypes

In [13]:

kwargs = dict(
        south=47, west=-63, 
        north=49, east=-61, 
        bottom=0, top=0,
        start=datetime(2013, 1, 1), end=datetime(2013, 1, 7))

bathy1, lat1, lon1 = kadlu.load(source='gebco', var='bathymetry', **kwargs)

waveheight2, lat2, lon2, epoch2 = kadlu.load(source='era5', var='waveheight', **kwargs)

2020-06-17 03:01:05  loading elevation from The GEBCO_2020 Grid - a continuous terrain model for oceans and land at 15 arc-second intervals

Manual loading from netcdf and geotiff¶

Kadlu can load from arbitrary netcdf- and geotiff-formatted data using the functions 'load_netcdf_2D' and 'load_geotiff_2D'. In the case of netcdf databases, the data must contain three variables, two of which are 'lat' and 'lon'. Kadlu will make an assumption that the X and Y axis are specified in coordinate degrees.

In [14]:

kwargs = dict(south=47, west=-63, north=49, east=-61)

bathy3, lat3, lon3 = kadlu.load_netcdf_2D(filename='/storage/gebco_bathy.nc', **kwargs)

2020-06-17 03:02:08  loading elevation from The GEBCO_2020 Grid - a continuous terrain model for oceans and land at 15 arc-second intervals

When loading from an arbitrary netcdf database, data transformation must be done by the user. For example, when loading GEBCO netcdf data directly from the file instead of the gebco().load_bathymetry function, bathymetric values will be returned as a measure of elevation (equal to depth * -1)

In [15]:

# returns a 2D array of elevation values
bathy3

Out[15]:

array([[-4339, -4339, -4340, ..., -4902, -4902, -4901],
       [-4345, -4343, -4343, ..., -4905, -4903, -4903],
       [-4345, -4349, -4349, ..., -4905, -4905, -4906],
       ...,
       [-5253, -5253, -5254, ..., -5252, -5252, -5252],
       [-5248, -5250, -5250, ..., -5252, -5252, -5252],
       [-5248, -5240, -5246, ..., -5254, -5253, -5253]], dtype=int16)