¶ Switching between space and time: Spatio-temporal analysis with
cubble ¶

H. Sherry Zhang

Monash University, Australia

ECSS Miniconference 2022

2022 Nov 17

Hi!

• A third year PhD student at Monash University, Melbourne, Australia

• My research centers on exploring multivariate spatio-temporal data with data wrangling and visualisation tool.

• Find me on

  • Twitter: huizezhangsh,
  • GitHub: huizezhang-sherry, and
  • https://huizezhangsh.netlify.app/

• Thanks for the invitation to speak

• Today I will be talking about …

• First, a little bit about myself

• I’m Sherry Zhang, …

• …

• Here are the details to find me on Twitter, GitHub, and my website

Storage of spatial and temporal data is often split into different tables

• Often, spatial and temporal data are stored separately

• This is convenient for data collection, not for the analysis

A long table?

Inefficient memory use,

repeated information,

especially when large geometry objects are combined with frequent temporal data (daily or weekly).

• Joining spatial and temporal data into a long table is not necessarily the best way to structure the data.

• This is because of the inefficient memory use, due to repeated information, especially when large geometry objects are combined with frequent temporal data.

Still use all my data analysis toolkits built from tibble (data frame)

• This motivates me to think about how to arrange spatio-temporal data for analysis.

• Ideally, we would like a data structure that can easily pivoting between spatial and temporal form

• It would be a plus if the interface is a tibble or data frame, so that we can use functionalities from existing packages for spatial and temporal analysis

Cubble: a spatio-temporal vector data structure

• Today I will introduce a new data structure, called cubble, for vector spatio-temporal data

• A cubble has two forms which you will see shortly that they can be pivoted back and forth.

• In the nested form, spatial variables are in columns out and temporal variables are nested into a list column called ts

• In the long form, the time series data are shown in columns and each row is cross identified by the site and date in a long table

Cubble: a spatio-temporal vector data structure

Cubble is a nested object built on tibble that allow easy pivoting between spatial and temporal form.

• With cubble, you can easily pivoting between these two forms for the analysis.

• In general, spatial operations will be performed in the nested form, on the left, and temporal operations are carried out in the long form.

• The pair face_temporal() and face_spatial() to switch the cubble between the two forms.

• With face_temporal(), the focus of the data is now on the temporal face of the spatio-temporal cube and this corresponds to switch the data to the long form.

• With face_spatial(), the long cubble is switched back to the nested form, the spatial face of the datacube.

Pipeline with cubble

spatial <- stations %>% 
  {{ Your spatial analysis }} 

##############################
# more subsetting step if temporal analysis
# depends on spatial results
sp_id <- spatial %>% pull(id)
ts_subset <- ts %>% filter(id %in% sp_id)
##############################

temporal <- ts_subset %>% 
  {{ Your temporal analysis }} 

##############################
# more subsetting step if spatial analysis 
# depends on temporal results
ts_id <- temporal %>% pull(id)
sp_subset <- spatial %>% filter(id %in% ts_id)
##############################

sp_subset %>% 
  {{ Your spatial analysis }} 

cb_obj %>% 
  {{ Your spatial analysis }} %>% 
  face_temporal() %>% 
  {{ Your temporal analysis }} %>% 
  face_spatial() %>% 
  {{ Your spatial analysis }} 

• Here is a syntax comparison with and without cubble

• On the right hand side with cubble, you can do some spatial analysis in the nested form, pivot it to the long form for some temporal analysis, and then pivot it back to the nested form for some additional spatial analysis.

• Sometimes, the spatial analysis include extracting some interesting sites.

• I have highlighted the code you will need to do to pull out some interesting stations and then filter out those station in the temporal data

• In cubble, these updates are automatically handled by face_temporal() and face_spatial(), so you don’t need to manually updates your spatial or temporal data to match with the other.

• Also the cubble pipeline chains all the operations together and there is no intermediate objects created in the workflow.

Australian weather station data:

stations

# A tibble: 30 × 6
  id            lat  long  elev name                       wmo_id
  <chr>       <dbl> <dbl> <dbl> <chr>                       <dbl>
1 ASN00060139 -31.4  153.   4.2 port macquarie airport aws  94786
2 ASN00068228 -34.4  151.  10   bellambi aws                94749
3 ASN00017123 -28.1  140.  37.8 moomba airport              95481
4 ASN00081049 -36.4  145. 114   tatura inst sustainable ag  95836
5 ASN00018201 -32.5  138.  14   port augusta aero           95666
# … with 25 more rows

ts

# A tibble: 10,632 × 5
  id          date        prcp  tmax  tmin
  <chr>       <date>     <dbl> <dbl> <dbl>
1 ASN00003057 2020-01-01     0  36.7  26.9
2 ASN00003057 2020-01-02    41  34.2  24  
3 ASN00003057 2020-01-03     0  35    25.4
4 ASN00003057 2020-01-04    40  29.1  25.4
5 ASN00003057 2020-01-05  1640  27.3  24.3
# … with 10,627 more rows

• Now, let’s see cubble in a dataset of Australian weather stations

• The stations data records 30 weather stations, along with their longitude, latitude, elevation, and name

[breath]

• On the temporal side, we have precipitation, maximum and minimum temperature collected daily for each station in 2020.

Cast your data into a cubble

(weather <- as_cubble(
  list(spatial = stations, temporal = ts),
  key = id, index = date, coords = c(long, lat)
))

# cubble:   id [30]: nested form
# bbox:     [114.09, -41.88, 152.87, -11.65]
# temporal: date [date], prcp [dbl], tmax [dbl], tmin [dbl]
  id            lat  long  elev name              wmo_id ts                
  <chr>       <dbl> <dbl> <dbl> <chr>              <dbl> <list>            
1 ASN00003057 -16.5  123.     7 cygnet bay         94201 <tibble [316 × 4]>
2 ASN00005007 -22.2  114.     5 learmonth airport  94302 <tibble [363 × 4]>
3 ASN00005084 -21.5  115.     5 thevenard island   94303 <tibble [366 × 4]>
4 ASN00010515 -32.1  117.   199 beverley           95615 <tibble [354 × 4]>
5 ASN00012314 -27.8  121.   497 leinster aero      95448 <tibble [366 × 4]>
# … with 25 more rows

• the spatial data (stations) can be an sf object and temporal data (ts) can be a tsibble object.

• To cast the two separate tables into a cubble, you can supply them in a named list.

• You also need to tell cubble some identifiers it looks for

• The key argument is the spatial identifier that connects the two tables.

• The index argument is the temporal identifier that prescribes the timestamp.

• The coords argument is to used to specify the coordinate

[breath]

• From the cubble header, you can read that the key is id, there are 30 stations and it is in the nested form.

• The third line here shows you the available temporal variables and their types.

• Also, if the spatial and temporal data is an sf or tsibble object, they will be indicated in the header as well.

Switch between the two forms

long form

(weather_long <- weather %>% 
  face_temporal())

# cubble:  date, id [30]: long form
# bbox:    [114.09, -41.88, 152.87, -11.65]
# spatial: lat [dbl], long [dbl], elev [dbl],
#   name [chr], wmo_id [dbl]
  id          date        prcp  tmax  tmin
  <chr>       <date>     <dbl> <dbl> <dbl>
1 ASN00003057 2020-01-01     0  36.7  26.9
2 ASN00003057 2020-01-02    41  34.2  24  
3 ASN00003057 2020-01-03     0  35    25.4
4 ASN00003057 2020-01-04    40  29.1  25.4
5 ASN00003057 2020-01-05  1640  27.3  24.3
# … with 10,627 more rows

back to the nested form:

(weather_back <- weather_long %>% 
   face_spatial())

# cubble:   id [30]: nested form
# bbox:     [114.09, -41.88, 152.87, -11.65]
# temporal: date [date], prcp [dbl], tmax [dbl],
#   tmin [dbl]
  id         lat  long  elev name  wmo_id ts      
  <chr>    <dbl> <dbl> <dbl> <chr>  <dbl> <list>  
1 ASN0000… -16.5  123.     7 cygn…  94201 <tibble>
2 ASN0000… -22.2  114.     5 lear…  94302 <tibble>
3 ASN0000… -21.5  115.     5 thev…  94303 <tibble>
4 ASN0001… -32.1  117.   199 beve…  95615 <tibble>
5 ASN0001… -27.8  121.   497 lein…  95448 <tibble>
# … with 25 more rows

identical(weather_back, weather)

[1] TRUE

• Here is what a cubble look like when being switched between the long and the nested form.

  • With the weather object we just created, we turn it into the long form with the function face_temporal()

• Notice that the third line in the header now changes to see the available spatial variables

[breath]

• On the right, weather_long is switched back the nested form with the function face_spatial()

• As you can see from the last line of code, face_temporal() and face_spatial() are the exact inverse.

• Hence weather_back and weather are identical

Access variables in the other form

Reference temporal variables with $

weather %>% 
  mutate(avg_tmax = mean(ts$tmax, na.rm = TRUE))

# cubble:   id [30]: nested form
# bbox:     [114.09, -41.88, 152.87, -11.65]
# temporal: date [date], prcp [dbl], tmax [dbl], tmin [dbl]
  id            lat  long  elev name              wmo_id ts                 avg_tmax
  <chr>       <dbl> <dbl> <dbl> <chr>              <dbl> <list>                <dbl>
1 ASN00003057 -16.5  123.     7 cygnet bay         94201 <tibble [316 × 4]>     32.4
2 ASN00005007 -22.2  114.     5 learmonth airport  94302 <tibble [363 × 4]>     33.2
3 ASN00005084 -21.5  115.     5 thevenard island   94303 <tibble [366 × 4]>     30.7
4 ASN00010515 -32.1  117.   199 beverley           95615 <tibble [354 × 4]>     26.4
5 ASN00012314 -27.8  121.   497 leinster aero      95448 <tibble [366 × 4]>     29.6
# … with 25 more rows

Move spatial variables into the long form

weather_long %>% unfold(long, lat)

# cubble:  date, id [30]: long form
# bbox:    [114.09, -41.88, 152.87, -11.65]
# spatial: lat [dbl], long [dbl], elev [dbl], name [chr], wmo_id [dbl]
  id          date        prcp  tmax  tmin  long   lat
  <chr>       <date>     <dbl> <dbl> <dbl> <dbl> <dbl>
1 ASN00003057 2020-01-01     0  36.7  26.9  123. -16.5
2 ASN00003057 2020-01-02    41  34.2  24    123. -16.5
3 ASN00003057 2020-01-03     0  35    25.4  123. -16.5
4 ASN00003057 2020-01-04    40  29.1  25.4  123. -16.5
5 ASN00003057 2020-01-05  1640  27.3  24.3  123. -16.5
# … with 10,627 more rows

• Sometimes, you may need to access variables from the other form for your analysis.

• For example, we may want to calculate some per station summary of the time series data.

• We can refer to the temporal variables from the nested form with the $ sign.

• Here I’m calculating the average maximum temperature across the whole year for each station and I need to get access to tmax from the list-column ts.

• In the long form, you need the cubble verb unfold() to move the spatial variables into the long form.

• Here I move the two coordinate columns into the long form and later we will see how it can help us to create a glyph map.

Explore temporal pattern across space

Sometimes, although we technically have spatio-temporal data, we may choose to fix a few stations to explore their temporal patterns, or select a few timestamps to explore their spatial distributions.

A more holistic approach is to explore the space and time simultaneously and now we will see an example of how to use cubble to explore temporal pattern across space in a glyph map.

Glyph map transformation

DATA %>%
  ggplot() +
  geom_glyph(
    aes(x_major = X_MAJOR, x_minor = X_MINOR,
        y_major = Y_MAJOR, y_minor = Y_MINOR)) +
  ...

• The glyphmap is essentially a transformation of temporal variable into space.

• I have a short illustration to show you how the transformation works

• Here (1) shows a single station on the map with its long and lat coordinate and (2) is its associated time series.

• In step (3), the temporal axes is transformed into the spatial scale with linear algebra, and

• Once we have the time series in the transformed axes, they can be placed onto the map as in (4)

• To make a glyph map, you can use the geom_glyph function from the cubble package.

• It requires a pair of major and a pair of minor variable as required aesthetics

• The major variable are the spatial coordinates, long and lat here and the minor variable are the temporal coordinates, date and tmax here.

Avg. max. temperature on the map

cb <- as_cubble(
  list(spatial = stations, temporal = ts),
  key = id, index = date, coords = c(long, lat)
)

set.seed(0927)
cb_glyph <- cb %>%
  slice_sample(n = 20) %>%
  face_temporal() %>%
  mutate(month = lubridate::month(date)) %>%
  group_by(month) %>% 
  summarise(tmax = mean(tmax, na.rm = TRUE)) %>%
  unfold(long, lat)

ggplot() +
  geom_sf(data = oz_simp, 
          fill = "grey95", 
          color = "white") +
  geom_glyph(
    data = cb_glyph,
    aes(x_major = long, x_minor = month,
        y_major = lat, y_minor = tmax),
    width = 2, height = 0.7) + 
  ggthemes::theme_map()

• This is a full example of using glyph map to explore Australian weather pattern.

• The code has three blocks.

• In the first block we first create a cubble object form the stations and ts data.

• The second block involves wrangling the data using the nested and long form.

• Sampling 20 stations is a spatial operations, so it is performed in the nested form.

• Then we need to do a summary of average maximum tmeperature by month. It is a temporal operation, so the cubble is then switched to the long form with face_temporal().

• The glyph map requires both the spatial and temporal axes variables as aesthetics, so we move the column long and lat with unfold() into the long form.

• The last chunk shows the ggplot2 code to make the glyph map with geom_glyph()

[breath]

• On the map, you can see that the temperature curve in the north and south (the Tasmania Island) are relative constant throughout the year.

• Those inland stations, for example in the eastern Australia, have a much visible variation in the year, as compared to the coastline ones.

• And remember Australia is in the southern hemisphere, so winter is in the June, July, and August and the temperature is in the U-shape.

Acknowledgements

• The slides are made with Quarto

• All the materials used to prepare the slides are available at sherryzhang-ecssmini2022.netlify.app

Reference

• cubble: https://huizezhang-sherry.github.io/cubble/

• Wickham, H., Hofmann, H., Wickham, C., & Cook, D. (2012). Glyph‐maps for visually exploring temporal patterns in climate data and models. Environmetrics, 23(5), 382-393: https://vita.had.co.nz/papers/glyph-maps.pdf

• This wraps up my talk today.

• Cubble has already made its way to CRAN

• There will be a version update on cubble in the next two weeks, so stay tuned!

• Thanks for listening