[MacPorts] #47056: nco/ncks and netcdf

[MacPorts]

#47056: nco/ncks and netcdf
-------------------------------+--------------------------------
 Reporter:  barry.j.mcinnes@…  |      Owner:  macports-tickets@…
     Type:  defect             |     Status:  new
 Priority:  Normal             |  Milestone:
Component:  ports              |    Version:  2.3.3
 Keywords:  nco netcdf         |       Port:  nco netcdf
-------------------------------+--------------------------------
 The current nco and netcdf on macports, currently take more than 2 hours
 (killed), whereas the versions on linux for the
 same files and command take 13 seconds. A similar nccopy command takes 13
 seconds on the same mac for the same files.

 The linux setup is
 /usr/local/bin/ncks -r
 NCO netCDF Operators version "4.4.5" last modified 2014/07/15 built Aug 29
 2014 on linux64.psd.esrl.noaa.gov by root
 ncks version 4.4.5
 Linked to netCDF library version 4.3.3, compiled Feb 24 2015 07:21:57
 Copyright (C) 1995--2014 Charlie Zender
 This program is part of NCO, the netCDF Operators.
 NCO is free software and comes with a BIG FAT KISS and ABOLUTELY NO
 WARRANTY
 You can redistribute and/or modify NCO under the terms of the
 GNU General Public License (GPL) Version 3 with exceptions described in
 the LICENSE file
 GPL: http://www.gnu.org/copyleft/gpl.html
 LICENSE: http://nco.cvs.sf.net/nco/nco/LICENSE
 Homepage: http://nco.sf.net
 User Guide: http://nco.sf.net/nco.html
 Configuration Option:   Active? Meaning or Reference:
 Check _FillValue        Yes     http://nco.sf.net/nco.html#mss_val
 Check missing_value     No      http://nco.sf.net/nco.html#mss_val
 DAP clients             Yes     http://nco.sf.net/nco.html#dap
 Debugging: Custom       No      Pedantic, bounds checking (slowest
 execution)
 Debugging: Symbols      No      Produce symbols for debuggers (e.g., dbx,
 gdb)
 GNU Scientific Library  Yes     http://nco.sf.net/nco.html#gsl
 HDF4 support            Unknown http://nco.sf.net/nco.html#hdf4
 Internationalization    No      http://nco.sf.net/nco.html#i18n (pre-
 alpha)
 MPI parallelization     No      http://nco.sf.net/nco.html#mpi (beta)
 netCDF3 64-bit files    Yes     http://nco.sf.net/nco.html#lfs
 netCDF4/HDF5 available  Yes     http://nco.sf.net/nco.html#nco4
 netCDF4/HDF5 enabled    Yes     http://nco.sf.net/nco.html#nco4
 OpenMP SMP threading    No      http://nco.sf.net/nco.html#omp
 Optimization: run-time  No      Fastest execution possible (slowest
 compilation)
 Parallel netCDF3        No      http://nco.sf.net/nco.html#pnetcdf (pre-
 alpha)
 Regular Expressions     Yes     http://nco.sf.net/nco.html#rx
 Shared libraries built  No      Small, dynamically linked executables
 Static libraries built  Yes     Large executables with private namespaces
 UDUnits conversions     Yes     http://nco.sf.net/nco.html#udunits
 UDUnits2 conversions    Yes     http://nco.sf.net/nco.html#udunits

 The macports is
 ncks -r
 NCO netCDF Operators version "4.4.7" last modified 2014/11/05 built Jan  6
 2015 on mac73.psd.esrl.noaa.gov by root
 ncks version 4.4.7
 Linked to netCDF library version 4.3.2, compiled Feb  3 2015 08:18:38
 Copyright (C) 1995--2014 Charlie Zender
 This program is part of NCO, the netCDF Operators.
 NCO is free software and comes with a BIG FAT KISS and ABOLUTELY NO
 WARRANTY
 You can redistribute and/or modify NCO under the terms of the
 GNU General Public License (GPL) Version 3 with exceptions described in
 the LICENSE file
 GPL: http://www.gnu.org/copyleft/gpl.html
 LICENSE: http://nco.cvs.sf.net/nco/nco/LICENSE
 Homepage: http://nco.sf.net
 User Guide: http://nco.sf.net/nco.html
 Configuration Option:   Active? Meaning or Reference:
 Check _FillValue        Yes     http://nco.sf.net/nco.html#mss_val
 Check missing_value     No      http://nco.sf.net/nco.html#mss_val
 DAP clients             Yes     http://nco.sf.net/nco.html#dap
 Debugging: Custom       No      Pedantic, bounds checking (slowest
 execution)
 Debugging: Symbols      No      Produce symbols for debuggers (e.g., dbx,
 gdb)
 GNU Scientific Library  Yes     http://nco.sf.net/nco.html#gsl
 HDF4 support            Unknown http://nco.sf.net/nco.html#hdf4
 Internationalization    No      http://nco.sf.net/nco.html#i18n (pre-
 alpha)
 MPI parallelization     No      http://nco.sf.net/nco.html#mpi (beta)
 netCDF3 64-bit files    Yes     http://nco.sf.net/nco.html#lfs
 netCDF4/HDF5 available  Yes     http://nco.sf.net/nco.html#nco4
 netCDF4/HDF5 enabled    Yes     http://nco.sf.net/nco.html#nco4
 OpenMP SMP threading    No      http://nco.sf.net/nco.html#omp
 Optimization: run-time  No      Fastest execution possible (slowest
 compilation)
 Parallel netCDF3        No      http://nco.sf.net/nco.html#pnetcdf (pre-
 alpha)
 Regular Expressions     Yes     http://nco.sf.net/nco.html#rx
 Shared libraries built  No      Small, dynamically linked executables
 Static libraries built  Yes     Large executables with private namespaces
 UDUnits conversions     Yes     http://nco.sf.net/nco.html#udunits
 UDUnits2 conversions    Yes     http://nco.sf.net/nco.html#udunits

 The command is
 ncks -D 3 --fl_fmt=netcdf4_classic -L 5 --overwrite
 huss_Amon_ACCESS1-0_historical_r1i1p1_185001-200512.nc
 /Projects/CMIP5A//Monthly/historical/ACCESS1-0//r1i1p1//huss_Amon_ACCESS1-0_historical_r1i1p1_185001-200512.nc

 The file size is 200MB.
 Maybe an update to netcdf 4.3.3 will speed things up ?
 Maybe its the chunking size the macports cannot handle ?
 One thing I note is that the chunck size for the ncks for the huss
 variable on Mac ends up being:
 1872, 145, 192
 but using nccopy, it's 1, 145, 192 which is more in line with what I would
 expect.

-- 
Ticket URL: <https://trac.macports.org/ticket/47056>
MacPorts <https://www.macports.org/>
Ports system for OS X