ahi-data-analysis-gdal.md

+# Manipulating AGLS CF-netCDF with GDAL Comamnd Line Interface
+
+The GDAL command line utilities may be to resize the netCDF data, reproject the netCDF data or convert to a different format (GeoTIFF, JPEG, PNG) for use with other software applications (such as GIS tools):
+Environment
+
+Note that some versions of the GDAL module will require the Proj4 module to be loaded (module load proj).
+
+```
+module load gdal/1.11.1-python
+data_path=/g/data/rr5/satellite/obs/himawari8/FLDK/2015/07/12/0000
+short_file_name=20150712000000-P1S-ABOM_BRF_B01-PRJ_GEOS141_1000-HIMAWARI8-AHI.nc
+
+# we can give gdal a hint that we are using a netCDF file
+# and that we want to access the variable named channel_0001_brf
+gdal_file_name=NETCDF:"$data_path/$short_file_name":channel_0001_brf
+```
+
+## Basic Information
+
+```
+# find out about the input image
+gdalinfo $gdal_file_name
+
+# find out about the coordinate system
+gdalsrsinfo $gdal_file_name
+
+#PROJ.4 : '+proj=geos +lon_0=140.7 +h=35785863 +x_0=0 +y_0=0 +a=6378137 +b=6356752.3 +units=m +no_defs '
+#
+#OGC WKT :
+#PROJCS["unnamed",
+#    GEOGCS["unnamed ellipse",
+#        DATUM["unknown",
+#            SPHEROID["unnamed",6378137,298.2570248822722]],
+#        PRIMEM["Greenwich",0],
+#        UNIT["degree",0.0174532925199433]],
+#    PROJECTION["Geostationary_Satellite"],
+#    PARAMETER["central_meridian",140.7],
+#    PARAMETER["satellite_height",35785863],
+#    PARAMETER["false_easting",0],
+#    PARAMETER["false_northing",0]]
+```
+
+## Simple Image Creation
+
+Often we want a quick look at the data we are working with.
+```
+# generate a thumbnail image (8-bit uint) scaled from 0 to 1 (output as 0-255)
+gdal_translate -scale 0 1 $gdal_file_name thumbnail.jpg \
+-of JPEG -ot Byte -outsize 256 256
+
+gdal_translate -scale 0 1 $gdal_file_name small.jpg \
+-of JPEG -ot Byte -outsize 10% 10%
+
+gdal_translate -scale 0 1 $gdal_file_name full_size.jpg \
+-of JPEG -ot Byte
+```
+
+## Reprojection
+
+Many other applications require simple projections (e.g. cylindrical equidistant). Note that the extent and resolution need to be specified in the output projection coordinates. In the case of "+proj=latlong", this is degrees. For most other projections, the coordinates are in "m" and must be specified accordingly.
+```
+# warp to a simple lat/lon projection with 0.01 degree resolution, use cubic interpolation. 
+# store the data as compressed (deflate/zlib) 32-bit floats
+# resulting tif can be viewed in GIS tools
+gdalwarp -overwrite -r cubic \
+-t_srs '+proj=latlong +a=6378137 +b=6356752' \
+-te 108 -44 156 -8 -tr 0.01 0.01 \
+$gdal_file_name map_float32.tif \
+-co "COMPRESS=DEFLATE" -co "TILED=YES"
+```
+
+## Export for GIS
+
+Convert the netCDF to GeoTIFF for display in GIS systems. Resulting tif can be viewed in GIS tools
+```
+gdal_translate $gdal_file_name slot_float32.tif \
+-co "COMPRESS=DEFLATE" -co "ZLEVEL=6" -co "TILED=YES"
+```
+Create image pyramids within the output image. Image pyramids are low resolution copies or overviews of the data. Overviews may be used by visualisation tools when the image is "zoomed out".
+```
+gdaladdo --config COMPRESS_OVERVIEW DEFLATE slot_float32.tif 2 4 8 16
+```
+Convert our map GeoTIFF for display in GIS systems. Resulting tif can be viewed in GIS tools. Here the scale keywork is used to map 0-1 in data space to 0-100 in map space (so that data can be stored as a byte, but remain interperable). We can use lossy JPEG compression for uint8 data type.
+```
+gdal_translate -scale 0 1 0 100 map_float32.tif map_uint8.tif \
+-ot Byte -co "COMPRESS=JPEG" -co "TILED=YES"
+gdaladdo --config COMPRESS_OVERVIEW JPEG map_uint8.tif 2 4 8 16
+```
+
+## Export for Display
+
+Convert 32-bit float to an image (8-bit uint) scaled from 0 to 1 (output as 0-255). The resulting tif can be viewed in GIS tools, but also within most image viewers. Lossy JPEG compression can significantly reduce the file size.
+```
+gdal_translate -scale 0 1 map_float32.tif map_uint8.tif \
+-ot Byte -co "COMPRESS=JPEG" -co "TILED=YES"
+```
+
+## Export to netCDF (CF 1.5)
+
+Depending on the projection support, GDAL can convert GeoTIFF to CF compliant netCDF.
+```
+# convert 32-bit float to a CF compliant netcdf
+# resulting netCDF should be CF compliant and viewable 
+# in most GIS tools and scientific data analysis tool
+gdal_translate map_float32.tif map_float32.nc \
+-of NETCDF -co "COMPRESS=DEFLATE"
+```
\ No newline at end of file