Warning "Unknown GDAL driver" on Aurora PostgreSQL with PostGIS

0

When trying to import raster data into Aurora PostgreSQL, we get the error WARNING: Unknown GDAL driver: .

Sample import that produces the error:

$ raster2pgsql -d -f raster_data -Y 1000 -s 54009 -I -C -M -t auto /data/ghsl/GHS_SMOD_E2030_GLOBE_R2023A_54009_1000_V2_0.tif imports.ghs_smod | psql service=udc
Processing 1/1: /data/ghsl/GHS_SMOD_E2030_GLOBE_R2023A_54009_1000_V2_0.tif
INFO: Using computed tile size: 256x256
BEGIN
NOTICE:  table "ghs_smod_test_to_delete" does not exist, skipping
DROP TABLE
CREATE TABLE
WARNING:  Unknown GDAL driver: 
COPY 1000
[...]

The driver that the raster2pgsql command is using the GTiff/GeoTIFF driver:

$ gdalinfo /data/ghsl/GHS_SMOD_E2030_GLOBE_R2023A_54009_1000_V2_0.tif | grep "Driver:"
Driver: GTiff/GeoTIFF

The PostGIS extension installed as per the documentation and postgis.gdal_enabled_drivers is set to ENABLE_ALL via an AWS parameter group – as confirmed below – including the GTiff/GeoTIFF driver.

udc=> SHOW postgis.gdal_enabled_drivers;
 postgis.gdal_enabled_drivers 
------------------------------
 ENABLE_ALL
(1 row)

udc=> SELECT short_name, long_name FROM ST_GDALDrivers(); 
   short_name    |                        long_name                         
-----------------+----------------------------------------------------------
 VRT             | Virtual Raster
 DERIVED         | Derived datasets using VRT pixel functions
 GTiff           | GeoTIFF
 COG             | Cloud optimized GeoTIFF generator
 NITF            | National Imagery Transmission Format
 RPFTOC          | Raster Product Format TOC format
 ECRGTOC         | ECRG TOC format
 HFA             | Erdas Imagine Images (.img)
 SAR_CEOS        | CEOS SAR Image
 CEOS            | CEOS Image
 JAXAPALSAR      | JAXA PALSAR Product Reader (Level 1.1/1.5)
 GFF             | Ground-based SAR Applications Testbed File Format (.gff)
 ELAS            | ELAS
 ESRIC           | Esri Compact Cache
 AIG             | Arc/Info Binary Grid
 AAIGrid         | Arc/Info ASCII Grid
 GRASSASCIIGrid  | GRASS ASCII Grid
 ISG             | International Service for the Geoid
 SDTS            | SDTS Raster
 DTED            | DTED Elevation Raster
 PNG             | Portable Network Graphics
 JPEG            | JPEG JFIF
 MEM             | In Memory Raster
 JDEM            | Japanese DEM (.mem)
 GIF             | Graphics Interchange Format (.gif)
 BIGGIF          | Graphics Interchange Format (.gif)
 ESAT            | Envisat Image Format
 BSB             | Maptech BSB Nautical Charts
 XPM             | X11 PixMap Format
 BMP             | MS Windows Device Independent Bitmap
 DIMAP           | SPOT DIMAP
 AirSAR          | AirSAR Polarimetric Image
 RS2             | RadarSat 2 XML Product
 SAFE            | Sentinel-1 SAR SAFE Product
 PCIDSK          | PCIDSK Database File
 PCRaster        | PCRaster Raster File
 ILWIS           | ILWIS Raster Map
 SGI             | SGI Image File Format 1.0
 SRTMHGT         | SRTMHGT File Format
 Leveller        | Leveller heightfield
 Terragen        | Terragen heightfield
 ISIS3           | USGS Astrogeology ISIS cube (Version 3)
 ISIS2           | USGS Astrogeology ISIS cube (Version 2)
 PDS             | NASA Planetary Data System
 PDS4            | NASA Planetary Data System 4
 VICAR           | MIPL VICAR file
 TIL             | EarthWatch .TIL
 ERS             | ERMapper .ers Labelled
 L1B             | NOAA Polar Orbiter Level 1b Data Set
 FIT             | FIT Image
 GRIB            | GRIdded Binary (.grb, .grb2)
 RMF             | Raster Matrix Format
 MSGN            | EUMETSAT Archive native (.nat)
 RST             | Idrisi Raster A.1
 GSAG            | Golden Software ASCII Grid (.grd)
 GSBG            | Golden Software Binary Grid (.grd)
 GS7BG           | Golden Software 7 Binary Grid (.grd)
 COSAR           | COSAR Annotated Binary Matrix (TerraSAR-X)
 TSX             | TerraSAR-X Product
 COASP           | DRDC COASP SAR Processor Raster
 R               | R Object Data Store
 MAP             | OziExplorer .MAP
 KMLSUPEROVERLAY | Kml Super Overlay
 PDF             | Geospatial PDF
 Rasterlite      | Rasterlite
 MBTiles         | MBTiles
 CALS            | CALS (Type 1)
 WMTS            | OGC Web Map Tile Service
 SENTINEL2       | Sentinel 2
 MRF             | Meta Raster Format
 PNM             | Portable Pixmap Format (netpbm)
 DOQ1            | USGS DOQ (Old Style)
 DOQ2            | USGS DOQ (New Style)
 PAux            | PCI .aux Labelled
 MFF             | Vexcel MFF Raster
 MFF2            | Vexcel MFF2 (HKV) Raster
 GSC             | GSC Geogrid
 FAST            | EOSAT FAST Format
 BT              | VTP .bt (Binary Terrain) 1.3 Format
 LAN             | Erdas .LAN/.GIS
 CPG             | Convair PolGASP
 NDF             | NLAPS Data Format
 EIR             | Erdas Imagine Raw
 DIPEx           | DIPEx
 LCP             | FARSITE v.4 Landscape File (.lcp)
 GTX             | NOAA Vertical Datum .GTX
 LOSLAS          | NADCON .los/.las Datum Grid Shift
 NTv2            | NTv2 Datum Grid Shift
 CTable2         | CTable2 Datum Grid Shift
 ACE2            | ACE2
 SNODAS          | Snow Data Assimilation System
 KRO             | KOLOR Raw
 ROI_PAC         | ROI_PAC raster
 RRASTER         | R Raster
 BYN             | Natural Resources Canada's Geoid
 ARG             | Azavea Raster Grid format
 RIK             | Swedish Grid RIK (.rik)
 USGSDEM         | USGS Optional ASCII DEM (and CDED)
 GXF             | GeoSoft Grid Exchange Format
 NWT_GRD         | Northwood Numeric Grid Format .grd/.tab
 NWT_GRC         | Northwood Classified Grid Format .grc/.tab
 ADRG            | ARC Digitized Raster Graphics
 SRP             | Standard Raster Product (ASRP/USRP)
 BLX             | Magellan topo (.blx)
 PostGISRaster   | PostGIS Raster driver
 SAGA            | SAGA GIS Binary Grid (.sdat, .sg-grd-z)
 XYZ             | ASCII Gridded XYZ
 HF2             | HF2/HFZ heightfield raster
 OZI             | OziExplorer Image File
 CTG             | USGS LULC Composite Theme Grid
 ZMap            | ZMap Plus Grid
 NGSGEOID        | NOAA NGS Geoid Height Grids
 IRIS            | IRIS data (.PPI, .CAPPi etc)
 PRF             | Racurs PHOTOMOD PRF
 SIGDEM          | Scaled Integer Gridded DEM .sigdem
 TGA             | TGA/TARGA Image File Format
 OGCAPI          | OGCAPI
 STACTA          | Spatio-Temporal Asset Catalog Tiled Assets
 STACIT          | Spatio-Temporal Asset Catalog Items
 GPKG            | GeoPackage
 CAD             | AutoCAD Driver
 GenBin          | Generic Binary (.hdr Labelled)
 ENVI            | ENVI .hdr Labelled
 EHdr            | ESRI .hdr Labelled
 ISCE            | ISCE raster
 Zarr            | Zarr
(126 rows)

udc=> SELECT PostGIS_GDAL_Version();
      postgis_gdal_version       
---------------------------------
 GDAL 3.5.3, released 2022/10/21
(1 row)

udc=> \dx postgis*
                                       List of installed extensions
          Name          | Version |  Schema  |                        Description                         
------------------------+---------+----------+------------------------------------------------------------
 postgis                | 3.4.0   | public   | PostGIS geometry and geography spatial types and functions
 postgis_raster         | 3.4.0   | public   | PostGIS raster types and functions
 postgis_tiger_geocoder | 3.4.0   | tiger    | PostGIS tiger geocoder and reverse geocoder
 postgis_topology       | 3.4.0   | topology | PostGIS topology spatial types and functions
(4 rows)

The warning is produced by the PostGIS extension (see PostGIS source code), which lets me anticipate that this could be something that needs fixing by AWS. Alternatively, it could be that raster2pgsql is not handing over the driver requirement properly. Usually, the warning message would state the specific driver that is unknown. In this case, it seems like enabled_drivers_array[i] that is defined in the PostGIS source code is empty.

Many thanks for any comments in advance!

asked 15 days ago27 views
1 Answer
0

The issue you're experiencing with the "Unknown GDAL driver" warning on Aurora PostgreSQL with PostGIS is unusual, especially given that you've properly installed the PostGIS extension and configured the GDAL drivers.

There are a few potential reasons for this warning:

  1. Version mismatch: Ensure that the version of PostGIS installed on your Aurora PostgreSQL instance is compatible with the GDAL version being used. You're using PostGIS 3.4.0 with GDAL 3.5.3, which should be compatible, but it's worth double-checking for any known issues with this specific combination.

  2. Configuration issue: Although you've set postgis.gdal_enabled_drivers to ENABLE_ALL, there might be an issue with how this setting is being applied or recognized by the PostGIS extension.

  3. Raster functionality: Starting with PostGIS 3.0, the raster functionality is separated into its own extension called postgis_raster. Make sure this extension is properly installed and configured.

  4. AWS-specific limitation: There might be an AWS-specific limitation or configuration issue that's preventing the proper recognition of GDAL drivers.

To troubleshoot this issue:

  1. Verify that the postgis_raster extension is properly installed and up to date.

  2. Try explicitly enabling the GTiff driver by setting postgis.gdal_enabled_drivers to 'GTiff' instead of 'ENABLE_ALL'.

  3. Check if the issue persists with different raster file formats.

  4. Review the Aurora PostgreSQL and PostGIS logs for any additional error messages or warnings that might provide more context.

  5. Consider reaching out to AWS support, as this could be related to how PostGIS and GDAL are implemented in the Aurora PostgreSQL environment.

Despite the warning, if your data is being imported correctly, it might not affect your workflow. However, if you're experiencing data import issues or unexpected results, it's crucial to resolve this warning to ensure proper functionality of PostGIS raster capabilities in your Aurora PostgreSQL environment.
Sources
Managing spatial data with the PostGIS extension - Amazon Aurora
Unsupported PostgreSQL features in Aurora DSQL - Amazon Aurora DSQL

profile picture
answered 15 days ago
  • I updated the PostGIS extension via the commands below. That did not resolve the issue. Regarding changing postgis.gdal_enabled_drivers to GTiff, this is not an option with AWS Aurora (you should know as the official AWS generative AI). Only two parameters are allowed in the parameter groups (Allowed values ENABLE_ALL,DISABLE_ALL) for postgis.gdal_enabled_drivers in AWS.

    udc=> SELECT * FROM pg_available_extensions WHERE name = 'postgis';
      name   | default_version | installed_version |                          comment                           
    ---------+-----------------+-------------------+------------------------------------------------------------
     postgis | 3.4.2           | 3.4.0             | PostGIS geometry and geography spatial types and functions
    (1 row)
    
    udc=> SELECT PostGIS_Extensions_Upgrade();
    NOTICE:  Updating extension postgis 3.4.0
    NOTICE:  Updating extension postgis_raster 3.4.0
    WARNING:  Unknown GDAL driver: 
    NOTICE:  Updating extension postgis_topology 3.4.0
    NOTICE:  Updating extension postgis_tiger_geocoder 3.4.0
                                 postgis_extensions_upgrade                             
    ------------------------------------------------------------------------------------
     Upgrade to version 3.4.2 completed, run SELECT postgis_full_version(); for details
    (1 row)
    

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions