Section : 2. Data Providers

Module : 2.4. GDAL Remote layers

Accessing remote layers with GDAL

“GDAL library provides a way stream data from external resources and access them as if they were on your local machine.”

GDAL is used to read and write various vector/raster data formats. GDAL can access files located on “standard” file systems, i.e. in the / hierarchy on Unix-like systems or in C: D: etc… drives on Windows. GDAL raster and vector drivers can access less standard types of files, such as in-memory files, compressed files (.zip, .gz, .tar, .tar.gz archives), encrypted files, files stored on the network (either publicly-accessible or in private buckets of commercial cloud storage services), etc.

You try:

Goal: To explore fetching data from remote sources using GDAL

  • Navigate to https://openaerialmap.org/
  • Select an image from the gallery provided.
  • Right-click on the download link and copy link address
  • Navigate to QGIS Download manager.
  • Choose add raster layer and specify the source type as Protocol.
  • Select HTTP/HTTPS/FTP and add the URL you copied i.e https://oin-hotosm.s3.amazonaws.com/6066c52fd8a0ef00061b9503/0/6066c52fd8a0ef00061b9504.tif
  • Load the Layer into QGIS.
  • Investigate the properties of the layer and any other options available
    Can you experiment with loading remote vector layer, remote zipped shapefiles. Are you able to load them and do any operation on them ?

More about loading remote layers

Gdal drivers allow the reading/writing of various data types. The less common data types that can be read by GDAL are the remote layers. For raster data stored in remote servers as Cloud Optimised Tiff(COG) GDAL streams the data in bytes allowing faster access to a certain portion of the raster layers. This has a significant advantage especially for visualising these files in web browsers. GDAL has many drivers notably

  • /vsizip/ (.zip archives) - Allows reading ZIP archives on-the-fly without decompressing them beforehand.
  • /vsigzip/ (gzipped file) - Allows on-the-fly reading of GZip (.gz) files without decompressing them in advance.
  • /vsitar/ (.tar, .tgz archives) - Allows on-the-fly reading in regular uncompressed .tar or compressed .tgz or .tar.gz archives, without decompressing them in advance.
  • /vsicurl - Allows access to online resources i.e S3buckets, Minio store

Check your knowledge:

  1. Can you use QGIS to access zipped shapefiles:

    1. Yes, but editing capabilities are not available
    2. Yes, it comes with full edit capabilities
    3. No, It is not possible

  2. Which of the following statements is true:

    1. Cloud optimised tiff can also be natively loaded as a file based raster
    2. Cloud optimised provides faster access to raster layers than normal file based rasters

Further reading: