## Custom install & load function
install.load.package <- function(x) {
  if (!require(x, character.only = TRUE)) {
    install.packages(x, repos = "http://cran.us.r-project.org")
  }
  require(x, character.only = TRUE)
}
## names of packages we want installed (if not installed yet) and loaded
package_vec <- c(
  "rgbif",
  "knitr", # for rmarkdown table visualisations
  "rio" # for dwc import
)
## executing install & load for each package
sapply(package_vec, install.load.package)

## rgbif knitr   rio 
##  TRUE  TRUE  TRUE

options(gbif_user = "my gbif username")
options(gbif_email = "my registred gbif e-mail")
options(gbif_pwd = "my gbif password")

First, we need to register the correct usageKey for Lagopus muta:

sp_name <- "Lagopus muta"
sp_backbone <- name_backbone(name = sp_name)
sp_key <- sp_backbone$usageKey

Second, this is the specific data we are interested in obtaining:

occ_final <- occ_search(
  taxonKey = sp_key,
  country = "NO",
  year = "2000,2020",
  basisOfRecord = "HUMAN_OBSERVATION",
  occurrenceStatus = "PRESENT"
)

Finally, we are ready to obtain GBIF mediated data we may want to use in downstream analyses and applications. This can be done in one of two ways - either via:

1. Near-Instant Download - via occ_data(...)
2. Asynchronous Download - via occ_download(...)

Let’s explore both of these in turn.

Near-Instant Download

Near-instant downloads do not require a GBIF account to be set-up and are the quickest way to obtain GBIF mediated data, but also the more limited functionality since near-instant downloads allow the user to only obtain 100,000 records per query and do not create citable DOIs for easy reference and accreditation of GBIF mediated data.

Let me demonstrate how these can be executed nevertheless.

As a matter of fact, we have already executed one of these through our call to occ_search(...) at the top of this page. While this function does download data via the ...$data output it provides, let’s showcase instead how we could use the occ_data(...) function to arrive at the same result:

NI_occ <- occ_data(
  taxonKey = sp_key,
  country = "NO",
  year = "2000,2020",
  basisOfRecord = "HUMAN_OBSERVATION",
  occurrenceStatus = "PRESENT"
)
knitr::kable(head(NI_occ$data))

The near-instantaneous nature of this type of download is largely due to the hard limit on how much data you may obtain. For our example, the 100,000 record limit is not an issue as we are dealing with only 10606 records being available to us, but queries to GBIF mediated records can easily and quickly exceed this limit.

Asynchronous Download

To avoid the reproducibility and data richness limitations of near-instant downloads, we must make a more formal download query to the GBIF API and await processing of our data request - an asynchronous download. To do so, we need to do three things in order:

1. Specify the data we want and request processing and download from GBIF
2. Download the data once it is processed
3. Load the downloaded data into R

Let’s tackle these step-by-step.

Data Request at GBIF

To start this process of obtaining GBIF-mediated occurrence records, we first need to make a request to GBIF to start processing of the data we require. This is done with the occ_download(...) function. Making a data request at GBIF via the occ_download(...) function comes with two important considerations:

1. occ_download(...) specific syntax
2. data query metadata

Syntax and Query through occ_download()

The occ_download(...) function - while powerful - requires us to confront a new set of syntax which translates the download request as we have seen it so far into a form which the GBIF API can understand. To do so, we use a host of rgbif functions built around the GBIF predicate DSL (domain specific language). These functions (with a few exceptions which you can see by calling the documentation - ?download_predicate_dsl) all take two arguments:

• key - this is a core term which we want to target for our request
• value - this is the value for the core term which we are interested in

Finally, the relevant functions and how the relate key to value are:

• pred(...): equals
• pred_lt(...): lessThan
• pred_lte(...): lessThanOrEquals
• pred_gt(...): greaterThan
• pred_gte(...): greaterThanOrEquals
• pred_like(...): like
• pred_within(...): within (only for geospatial queries, and only accepts a WKT string)
• pred_notnull(...): isNotNull (only for geospatial queries, and only accepts a WKT string)
• pred_isnull(...): isNull (only for stating that you want a key to be null)
• pred_and(...): and (accepts multiple individual predicates, separating them by either “and” or “or”)
• pred_or(...): or (accepts multiple individual predicates, separating them by either “and” or “or”)
• pred_not(...): not (negates whatever predicate is passed in)
• pred_in(...): in (accepts a single key but many values; stating that you want to search for all the values)

Let us use these to query the data we are interested in:

res <- occ_download(
  pred("taxonKey", sp_key),
  pred("basisOfRecord", "HUMAN_OBSERVATION"),
  pred("country", "NO"),
  pred("hasCoordinate", TRUE),
  pred_gte("year", 2000),
  pred_lte("year", 2020)
)

Now that the download request has been made with GBIF, you need to wait for GBIF to process your query and return the desired data to you. You will find an overview of all your requested and processed downloads in your personal download tab on the GBIF webportal.

Data Query Metadata

To keep track of all your data queries, each call to occ_download(...) returns a set of metadata for your download request as part of its object structure. This can be assessed as follows:

occ_download_meta(res)

## <<gbif download metadata>>
##   Status: SUCCEEDED
##   DOI: 10.15468/dl.vjazpv
##   Format: DWCA
##   Download key: 0010612-241024112534372
##   Created: 2024-10-29T13:25:46.576+00:00
##   Modified: 2024-10-29T13:27:15.574+00:00
##   Download link: https://api.gbif.org/v1/occurrence/download/request/0010612-241024112534372.zip
##   Total records: 10609

It can even be subset for relevant parts of it like so:

occ_download_meta(res)$downloadLink

## [1] "https://api.gbif.org/v1/occurrence/download/request/0010612-241024112534372.zip"

This is done according to the underlying structure of the metadata:

str(occ_download_meta(res), max.level = 1)

## List of 12
##  $ key           : chr "0010612-241024112534372"
##  $ doi           : chr "10.15468/dl.vjazpv"
##  $ license       : chr "http://creativecommons.org/licenses/by-nc/4.0/legalcode"
##  $ request       :List of 5
##  $ created       : chr "2024-10-29T13:25:46.576+00:00"
##  $ modified      : chr "2024-10-29T13:27:15.574+00:00"
##  $ eraseAfter    : chr "2025-04-29T13:25:46.526+00:00"
##  $ status        : chr "SUCCEEDED"
##  $ downloadLink  : chr "https://api.gbif.org/v1/occurrence/download/request/0010612-241024112534372.zip"
##  $ size          : int 2842371
##  $ totalRecords  : int 10609
##  $ numberDatasets: int 18
##  - attr(*, "class")= chr "occ_download_meta"
##  - attr(*, "format")= chr "DWCA"

Note: You will find that the number of records in our query made via occ_download(...) here and occ_search(...) above are different. This is due to different ways data ranges are handled.

occ_download_meta(res)$totalRecords

## [1] 10609

Data Download

Now that the download request is registered with GBIF, it is time to actually obtain the data you have queried. To facilitate this, there are two distinct ways to obtain the data itself:

1. Webportal
2. rgbif

Personally, I can not recommend any of these methods over the rgbif-internal download method. However, in case you might prefer other options, I will quickly show how these work, too.

GBIF Portal Download Method

At the most basic level, you may obtain data through the GBIF portal where you will find an overview of your download requests. Any individual download request can be investigated more deeply by clicking the “SHOW” button:

There, you will see a breakdown of your download request and also whether it is still being processed or already finished:

Note that the DOI differs here from the other images and the data product we are working with ultimately. This is because I had to requeue the download to get this screenshot.

Upon completion of your request on the GBIF end, a “DOWNLOAD” button will appear on the specific download request overview page:

For requests of few data points you will usually have to be quite quick to see this in action as GBIF is often pretty fast and getting data ready for you.

rgbif Download Method

To download our data programmatically, we need to execute roughly the same steps as we did above:

1. Wait for data to be ready for download
2. Download the data
3. Load the data into R (we didn’t do this in the above)

This is done with the following three corresponding functions:

## 1. Check GBIF whether data is ready, this function will finish running when done and return metadata
res_meta <- occ_download_wait(res, status_ping = 5, curlopts = list(), quiet = FALSE)
## 2. Download the data as .zip (can specify a path)
res_get <- occ_download_get(res)
## 3. Load the data into R
res_data <- occ_download_import(res_get)

Let’s have a look at the data we just loaded:

dim(res_data)

## [1] 10609   223

knitr::kable(head(res_data))

Note that the structure of this data set is different to that obtained with occ_search(...) hence enforcing that you should always stage downloads whose data are used in publications with occ_download(...)!

Session Info

## R version 4.4.0 (2024-04-24 ucrt)
## Platform: x86_64-w64-mingw32/x64
## Running under: Windows 11 x64 (build 22631)
## 
## Matrix products: default
## 
## 
## locale:
## [1] LC_COLLATE=Norwegian Bokmål_Norway.utf8  LC_CTYPE=Norwegian Bokmål_Norway.utf8   
## [3] LC_MONETARY=Norwegian Bokmål_Norway.utf8 LC_NUMERIC=C                            
## [5] LC_TIME=Norwegian Bokmål_Norway.utf8    
## 
## time zone: Europe/Oslo
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] rio_1.2.3   knitr_1.48  rgbif_3.8.1
## 
## loaded via a namespace (and not attached):
##  [1] styler_1.10.3     sass_0.4.9        utf8_1.2.4        generics_0.1.3   
##  [5] xml2_1.3.6        blogdown_1.19     stringi_1.8.4     httpcode_0.3.0   
##  [9] digest_0.6.37     magrittr_2.0.3    evaluate_0.24.0   grid_4.4.0       
## [13] bookdown_0.40     fastmap_1.2.0     R.oo_1.26.0       R.cache_0.16.0   
## [17] plyr_1.8.9        jsonlite_1.8.8    R.utils_2.12.3    whisker_0.4.1    
## [21] crul_1.5.0        urltools_1.7.3    httr_1.4.7        purrr_1.0.2      
## [25] fansi_1.0.6       scales_1.3.0      oai_0.4.0         lazyeval_0.2.2   
## [29] jquerylib_0.1.4   cli_3.6.3         rlang_1.1.4       triebeard_0.4.1  
## [33] R.methodsS3_1.8.2 bit64_4.0.5       munsell_0.5.1     cachem_1.1.0     
## [37] yaml_2.3.10       tools_4.4.0       dplyr_1.1.4       colorspace_2.1-1 
## [41] ggplot2_3.5.1     curl_5.2.2        vctrs_0.6.5       R6_2.5.1         
## [45] lifecycle_1.0.4   stringr_1.5.1     bit_4.0.5         pkgconfig_2.0.3  
## [49] pillar_1.9.0      bslib_0.8.0       gtable_0.3.6      data.table_1.16.0
## [53] glue_1.7.0        Rcpp_1.0.13       xfun_0.47         tibble_3.2.1     
## [57] tidyselect_1.2.1  htmltools_0.5.8.1 rmarkdown_2.28    compiler_4.4.0