========================================================================== = = = This files conatins a history of the HDF4.* releases. = = = = To find information about a particular release search = = for %%%4.#r# string, for example 4.1r2. = = List of all releases is on the top of the file. = = = = Documents in this file refer to several *.txt files that were = = originally stored in the release_notes directory of the HDF4 = = source tree. Those file are now combined into one misc_docs.txt = = file in the same directory. = = = ========================================================================== List of the HDF4 releases 4.2r0 December 2003 4.2r0-Beta September 2003 4.1r5 November 2001 4.1r4 October 2000 4.1r3 May 1999 4.1r2 March 1998 4.1r1 February 1997 4.1b1 December 1996 4.0r2 July 1996 4.0r1 February 1996 4.0b2 November 1995 4.0b1 July 1995 4.0.alpha November 1994 ========================================================================== %%%4.2r0%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% HDF 4.2 Release 0 December 2003 INTRODUCTION This document describes the differences between HDF 4.1r5 and HDF 4.2r0. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.2r0 The HDF 4.2r0 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: ftp://hdf.ncsa.uiuc.edu/HDF/Documentation/ First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Support for new platforms - Bugs fixed since HDF4.1r5 - Documentation - Platforms Tested - Known problems New Features and Changes: ======================== o IMPORTANT: HDF4 HAS A NEW CONFIGURATION ZLIB and JPEG libraries were removed from the HDF4 distribution source. ZLIB and JPEG have to be installed on the system before HDF4 library can be built or HDF4 precompiled binaries can be used. Please read INSTALL in the top HDF4 directory for instructions how to build the HDF4 library and applications. o HDF4 has an optional SZIP compression method; in order to use SZIP compression, SZIP library has to be installed on the system. Please refer to the INSTALL file for instructions on how to build with/without the SZIP Library. SZIP in HDF4 is free for non-commercial use; see http://hdf.ncsa.uiuc.edu/doc_resource/SZIP/Commercial_szip.html for information regarding commercial use. For more information about SZIP compression see http://hdf.ncsa.uiuc.edu/doc_resource/SZIP/ and the "HDF4 Reference Manual" entries for the GRsetcompress and SDsetcompress functions. o IMPORTANT note about prebuild binaries: NCSA precompiled binaries has SZIP compression method enabled for all platforms except Crays T3E and SV1 and Linux 2.4 SuSE x86_64. To use the binaries download the SZIP library from http://hdf.ncsa.uiuc.edu/doc_resource/SZIP/ o The following new tools have been added hrepack hdiff hdfimport See "HDF4 Reference Manual" in the HDF 4.2r0 documentation set for more information. o fp2hdf is removed. It is replaced with hdfimport. o Helper scripts to facilitate HDF4 installation and compilation were added: h4cc - to compile C application with the HDF4 Libraries h4fc - to compiler Fortran applications with the HDF4 Libraries h4redeploy - to fix binary installation See "HDF4 Reference Manual" in the HDF 4.2r0 documentation set for more information. Support for new platforms ========================= o HDF4 was ported to the following platforms AIX 5.1 64-bit version MacOSX Linux 2.4 RH8 and RH9 Linux 2.4 RH8 64-bit, SuSE 64-bit Linux 2.4 ia64 Linux 2.4 SGI (Altrix) Bugs fixed since HDF4.1r5 ========================= 1. "hdp dumpgr" and "hdp dumpsds" have two new options: -g to suppress the data of global (or file) attributes -l to suppress the data of local attributes 2. The problem where hdp failed on a very long file name has been fixed. (bug #693) 3. The problem where VSinquire failed when being called on a vdata that had no fields defined, has been fixed. (bug #626) 4. When the values of VGNAMELENMAX and VSNAMELENMAX are changed the hdp output doesn't reflect the change. This problem is now fixed. (bug #606) 5. hdp dumpvg sometimes failed when reading a file that had a vgroup being inserted into another. This has been fixed. (bug #477) 6. hdf2gif failed when a user tried it on JPEG compressed images. (Bug #601). The problem is now fixed. An error message is displayed if the image is not 8-bit. If the image is 24-bit, the message suggests using hdf2jpeg. 7. ncdump failed to read NetCDF files 3.5 when there was more than one variable with unlimited dimensions. Fixed. 8. The NetCDF part of the HDF4 library was not ported to Compaq True64 system. Fixed. 9. The hdp commands dumpsds, dumpgr, dumpvd, and dumpvg now display an informative message when a non-HDF file is given as input. (Bug #817) 10. The compilation warnings and error on the macro HDFclose are fixed. (Bug #818) Documentation ============== Entries for new utilities hdiff, hrepack, hdfimport and helper scripts h4cc, h4fc, and h4redeply were added to the Reference Manual. Platforms Tested ================ HDF 4.2 Release 0 has been tested on the following platforms: AIX 5.1 (32 and 64-bit) xlc 6.0.0.2 xlf 8.1.1 Cray T3E sn6606 2.0.6.08 Cray Standard C Version 6.6.0.2 Cray Fortran Version 3.6.0.2 Cray SV1 sn9617 10.0.1.2 Cray Standard C Version 6.6.0.2 Cray Fortran Version 3.6.0.2 FreeBSD 4.9 gcc 2.95.4 g++ 2.95.4 HP-UX B.11.00 HP C HP92453-01 A.11.01.20 HP F90 v2.4 HP ANSI C++ B3910B A.03.13 IRIX64 6.5 (64 & n32) MIPSpro cc 7.3.1.3m F90 MIPSpro 7.3.1.3m Linux 2.4.20-20.7 (RH8) gcc 3.3.1 Intel(R) C++ Version 7.1 Intel(R) Fortran Compiler Version 7.1 Linux 2.4.20-20.9 (RH9) gcc 3.2.2 Linux 2.4.21-2.9.5ws x86_64 gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-16) g77 based on gcc version 3.2.3 Linux 2.4.19-SMP #1 x86_64 gcc version 3.3.2 (SuSE Linux) g77 version 3.3.2 OSF1 V5.1 Compaq C V6.4-014 Compaq Fortran V5.5-1877 SunOS 5.7(32 and 64 bit) WorkShop Compilers 5.0 98/12/15 C 5.0 (Solaris 2.7) WorkShop Compilers 5.0 98/10/25 FORTRAN 77 5.0 gcc 3.2.2 g77 GNU Fortran (GCC 3.2.2) 3.2.2 SunOS 5.8(32 and 64 bit) Sun WorkShop 6 update 2 C 5.3 (Solaris 2.8) Sun WorkShop 6 update 2 Fortran 77 5.3 IA-32 Linux 2.4.9-31pctr Intel(R) C++ Version 7.0 Intel(R) Fortran Compiler Version 7.0 IA-64 Linux 2.4.16 ia64 Intel(R) C++ Version 7.0 Intel(R) Fortran Compiler Version 7.0 IA-64 Linux 2.4.19-SMP Intel(R) C++ Version 7.1 Intel(R) Fortran Compiler Version 7.1 gcc 3.2 g77 GNU Fortran (GCC 3.2) 3.2 IA-64 Linux 2.4.21-sgi Intel(R) C++ Version 7.1 (Altrix) Intel(R) Fortran Compiler Version 7.1 Windows 2000 (NT5.0) MSVC++ 6.0 DEC Visual Fortran 6.0 Intel C and F90 compilers version 7.1 Windows XP MSVC++.NET MAC OS X Darwin 6.8 gcc Apple Computer, Inc. GCC version 1175, based on gcc version 3.1 Known problems ============== o SZIP Library is not available for Crays SV1 and T3E Fortran APIs do not support SZIP compression. o NetCDF tests nctest (C) and ftest (Fortran) fail to read from NetCDF 3.5 files o On Linux RH8 64-bit SZIP tests fail if library is compiled in production mode. All tests pass in the debug mode. o This release doesn't support VMS system. o On Linux with gcc compilers Fortran NetCDF APIs cannot read attributes, variables, and dimensions when name contain spaces. o HDF4 Library cannot be built with PGI compilers. o N-Bit compression is not supported with Fortran APIs. o Using both fill-value and compression on SD datasets doesn't work. o --prefix defines where the installation path is. This version has the default set as /usr/local which is different from previous versions. o New utilities hdiff and hrepack are not available for Windows 2000. %%%4.2r0-Beta%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF 4.2 Release 0-Beta September 2003 INTRODUCTION This document describes the differences between HDF 4.1r5 and HDF 4.2r0-Beta. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.2r0-Beta The HDF 4.2r0-Beta documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/pub/outgoing/hdf4/4.2-Beta/ First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested New Features and Changes: ======================== o ZLIB and JPEG libraries were removed from the HDF4 distribution source. Please read INSTALL-4.2r0-Beta in the top HDF4 directory for instructions how to build HDF4 Library and applications. o HDF4 has an optional SZIP compression; please refer to the INSTALL-4.2r0-Beta file for instructions how to build with/without the SZIP Library. SZIP in HDF4 is free for non-commercial use; see http://hdf.ncsa.uiuc.edu/doc_resource/SZIP/Commercial_szip.html for information regarding commercial use. For more information about SZIP compression see http://hdf.ncsa.uiuc.edu/HDF5/doc_resource/SZIP/ and the "HDF4 Reference Manual" entries for the GRsetcompress and SDsetcompress functions. o The following new tools have been added hrepack hdiff hdfimport See "HDF4 Reference Manual" in the HDF 4.2r0-Beta documentation set for more information. o HDF4 was ported to the following platforms AIX 5.1 64-bit version MacOSX Linux 2.4 RH8 and RH9 Please refer to the bugs_fixed.txt file for more details on bugs that were fixed. Platforms Tested: ================ HDF 4.2 Release 0-Beta has been tested on the following platforms: FreeBSD 4.9 HP-UX B.11.00 AIX 5.1 (32 and 64-bit) IRIX64 6.5 (-n32, -64) Linux 2.4 Solaris 2.7, 2.8 (32 and 64-bit) MacOSX No precompiled binaries is available for this release. %%%4.1r5%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF 4.1 Release 5 November 2001 INTRODUCTION This document describes the differences between HDF 4.1r4 and HDF 4.1r5. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1r5. The HDF 4.1r5 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/HDF/Documentation/HDF4.1r5/ First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested New Features and Changes: ======================== o The following Vdata routines were added: VSsetblocksize/vsfsetblsz -- sets the block size of the linked-block element. VSsetnumblocks/vsfsetnmbl -- sets the number of blocks for a linked-block element. VSgetblockinfo/vsfgetblinfo -- retrieves the block size and the number of blocks of a linked-block element. o Two routines were added to get compression information for the SD and GR interfaces, including chunked elements: SDgetcompress/sfgcompress and GRgetcompress/mggcompress. Note: - For a JPEG image, GRgetcompress only returns the compression type, not the compression information (i.e, quantity and force_baseline). This information is not currently retrievable. - Getting compression type for JPEG chunked images is not working yet. o "hdp dumpgr" has a new option, -pd, to print palette data only. Also, whenever option -p or -pd is given, only palettes are printed, and no images or file attributes. o A new FORTRAN function, heprntf (HEprint), was added. It takes two arguments: file name and level. If the file name string has 0 length, then error messages will be printed to standard output. o On Windows, the unresolved symbol (error_top) error has been fixed when calling HEclear and linking with the DLL. Users who want to use the HDF DLL should define HDFAPDLL in their applications. Simply go to Project Settings and add HDFAPDLL as the predefined constant. o A memory leak in the netCDF portion of the HDF/mfhdf distribution was fixed. o The "#define NULL" was removed since ANSI C compilers are required to define NULL. o When using "hdp dumpgr", data was being printed in the range of 0-250 when it should have been between 0-168. This problem is now fixed. Please refer to the bugs_fixed.txt file for more details on bugs that were fixed. Platforms Tested: ================ HDF 4.1 Release 5 has been tested on the following platforms: Cray SV1 10.0.0.8 Cray T3E sn6711 2.0.5.55 Compaq Tru64 Unix (OSF1) 5.1 DEC Alpha/OpenVMS AXP 7.2-1 FreeBSD 4.4 HP-UX B.11.00 IBM SP 4.3 IRIX 6.5 IRIX64 6.5 (-n32, -64) Linux 2.2.18smp Solaris 2.7, 2.8 Windows NT/98/2000 For more information on the platforms that were tested and for which we provide pre-compiled binaries, please refer to the following web page (accessible from the HDF home page): http://hdf.ncsa.uiuc.edu/platforms.html Know problmes: Writing n-bit datasets from FORTRAN with the SD interface is not working. SDgetchunkinfo does not return compression coding or modelling type. Using both fill-values and compression on SD datasets is not currently working, don't use one or the other. Dumping compressed Vdatas with vshow or hdp is not working. Reading or writing compressed images with the GR interface is not working. %%%4.1r4%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF 4.1 Release 4 October 2000 INTRODUCTION This document describes the differences between HDF 4.1r3 and HDF 4.1r4. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1r4. The HDF 4.1r4 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/HDF/Documentation/HDF4.1r4/ First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems New Features and Changes: ======================== This release focuses on new features and changes added to the GR interface. o Two new utilities have been added to HDF, gif2hdf and hdf2gif. The gif2hdf utility will convert a GIF image into an HDF file containing a GR image. The hdf2gif utility will convert an HDF GR image into a GIF image. o Chunking and chunking with compression have been added to the GR interface. o JPEG compression with the GR interface was not working properly. This problem has been fixed. Several hdp options have been added: o Added -s option to dumpgr and dumpsds to allow printing data as a stream instead of breaking the lines at 65 characters. o Added option -c to dumpgr and dumpsds to allow printing clean output for attributes with type DFNT_CHAR. With this option, hdp will print space characters, such as horizontal tabs, CRs, and LFs, as they are instead of "\digit" (still the default.) This option also prints "..." for one or more null characters among the data. o Added option -l to dumpgr to allow printing data in different interlace modes. Platforms Tested: ================ HDF 4.1 Release 4 has been tested on the following platforms: Cray J90 (available after initial 4.1r4 release) Cray T3E DEC Alpha/Digital Unix DEC Alpha/OpenVMS Exemplar FreeBSD HP-UX IRIX IRIX64 (-n32, -64) Linux Solaris Solaris x86 SP Windows NT/98/2000 For more information on the platforms that were tested and for which we provide pre-compiled binaries, please refer to the following web page (accessible from the HDF home page): http://hdf.ncsa.uiuc.edu/platforms.html Known Problems: ============== o The ncgen utility fails on the IBM SP. %%%4.1r3%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF 4.1 Release 3 May 7, 1999 INTRODUCTION This document describes the differences between HDF 4.1r2 and HDF 4.1r3. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1r3. The release notes provide more in-depth information concerning the topics discussed here. The HDF 4.1r3 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/HDF/Documentation/HDF4.1r3 First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems - Acknowledgements New Features and Changes: ======================== o HDF 4.1r2 was unable to properly read HDF SDSs created with HDF 3.3x. It did not read the correct SDS names. This problem has been fixed. o Many problems have been fixed with the GR interface, including the following: - The GR interface can now read compressed files created with the DFR8 and DF24 interfaces, except for those which were compressed with IMCOMP compression. - The GR interface can read and write images compressed with RLE, GZIP and Skipping Huffman compression methods. - Palettes can now be written and read properly with the GR interface. - 24-bit raster images can now be read by the GR interface. o You can now create an SDS with a name up to 256 characters in length. The previous limit was 64. o HDF now supports IJP JPEG version 6b and Gzip version 1.1.3. o Numerous hdp problems have been fixed, including the following: - hdp no longer fails on an HDF file which contains a vdata that no records have been written to. - hdp no longer fails on the PC and Mactinosh dumping large SDSs. - GR file attributes can now be displayed. - A palette can now be dumped with the GR command. o SDfileinfo no longer returns the wrong number of datasets for old files created with the DFSD interface. o This will be the last release that SunOS 4.1.4 is supported. Check the ./bugs_fixed.txt for other changes that are not listed here. Platforms Tested: ================ HDF 4.1 Release 3 has been tested on the following platforms: Cray J90 Cray T90 (CFP, IEEE) Cray T3E DEC Alpha/Digital Unix DEC Alpha/OpenVMS DEC Alpha NT VAX OpenVMS Exemplar FreeBSD HP-UX 10.2 IRIX 6.5 IRIX64 6.5 (-n32, -64) Linux Macintosh Solaris Solaris x86 SP SunOS 4.1.4 Windows NT/95 For more information on the platforms that were tested and for which we provide pre-compiled binaries, please refer to the following web page (accessible from the HDF home page): http://hdf.ncsa.uiuc.edu/platforms.html Known Problems: ============== o On Alpha OpenVMS version 6.2, the DF.OLB and MFHDF.OLB Libraries should be created with optimization turned off. Otherwise hdftest fails (the sfgichnk function returns incorrect information). o On VAX Open VMS 6.2, the ncgen utility core dumps and an error occurs when reading GR image data with user-defined fill values. o If you encounter problems building on a platform, please be sure to check the INSTALL file at the top of the HDF source tree, in case these problems are documented in section 2.5, Platform-specific Notes. o On the NT, the hdp utility fails in the debug version when using the list command. Acknowledgements: ================ Fortner Software LLC ("Fortner") created the reference implementations for the Macintosh and Windows NT/95 of the HDF 4.1r3 library. For more information, please refer to the macintosh.txt and windows.txt files in the ./release_notes/ directory. %%%4.1r2%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF 4.1 Release 2 March 16, 1998 INTRODUCTION This document describes the differences between HDF 4.1r1 and HDF 4.1r2. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1r2. The release notes provide more in-depth information concerning the topics discussed here. The HDF 4.1r2 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/Documentation/HDF4.1r2 First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems - Important Fixes - Acknowledgements New Features and Changes: ======================== o Data chunking is now supported with the GR interface. New routines for creating and manipulating chunked GR images have been added. Please refer to the ./release_notes/new_functions.txt file and HDF Reference Manual for information on using chunked GRs. o In previous releases, many C routines existed for which there were no Fortran counterparts. With HDF 4.1r2, we have added a Fortran routine for most C routines. Please check the ./release_notes/new_functions.txt file for a list of the new functions added to HDF. o This is the first release in which the Java Products (the Java-based HDF Viewer (JHV) and the Java HDF interface (JHI)) are incorporated in the HDF release itself. For information on the Java Products, please refer to the HDF home page under Information about HDF (http://hdf.ncsa.uiuc.edu/about.html). o In the SD interface, HDF now defaults to ONLY storing the new version of the dimension representation added in HDF 4.0r1. When the dimension representation was changed in 4.0r1, the HDF library defaulted to include both the new and old dimension representations in an HDF file. Now, this new dimension representation is stored by default. The SDsetdimval_comp function can be used to change the dimension representation stored. Following is a detailed description of the difference between the new and old representations: Prior to HDF 4.0r1, a vgroup was used to represent a dimension. The vgroup had a single field vdata with a class of "DimVal0.0". The vdata had number of records, with each record having a fake value from 0, 1, 2 ... , ( - 1). The fake values were not really required and took up a large amount of space. For applications that created large one dimensional array datasets, the disk space taken by these fake values almost doubled the size of the HDF file. In order to omit the fake values, the new version for a dimension vdata was implemented. The new version uses the same structure as the old version. The only differences are that the vdata has only 1 record with a value of and that the vdata's class is "DimVal0.1", to distinguish it from the old version. o Platforms dropped with this release: Cray Y-MP, T3D, and Linux (a.out) o Extensive changes have been made to the Reference Manual and User's Guide. The updated Reference Manual is available with this release. The updated User's Guide will be available in the near future. Platforms Tested: ================ HDF 4.1 Release 2 has been tested on the following platforms: Cray T90 (CFP, IEEE) IRIX 6.2 Cray T3E IRIX64 6.4 (-n32, -64) DEC Alpha/Digital Unix Linux (elf) Exemplar Solaris FreeBSD Solaris x86 HP-UX 9.03 SP2 HP-UX 10.2 SunOS IRIX 5.3 ** The Windows NT/95, Macintosh, Dec Alpha OpenVMS and VAX OpenVMS releases are not available with this release of HDF4.1r2. Separate releases for these platforms will be available in the near future. For more information on the platforms that were tested and for which we provide pre-compiled binaries, please refer to the following web page (accessible from the HDF home page): http://hdf.ncsa.uiuc.edu/platforms.html Known Problems: ============== o Writing n-bit datasets from FORTRAN with the SD interface is not working. o SDgetchunkinfo does not return compression coding or modeling type. o Using both fill-values and compression on SD datasets is not currently working; don't use one or the other. o Dumping compressed Vdatas with vshow or hdp is not working. o Reading or writing compressed images with the GR interface is not working. o With the GR interface, you cannot create a raster image without writing data to it. Important Fixes: =============== o HDF no longer core dumps when reading a NetCDF file. o HDF now supports little-endian conversion for VAX and Dec Alpha OpenVMS. o The problems that occurred on the Cray with HDF 4.1r1 have been corrected. See the ./release_notes/bugs_fixed.txt file for more information on bugs fixed in this release. Acknowledgements: ================ Fortner Software LLC ("Fortner") created the reference implementations for Macintosh and Windows NT/95 of the HDF 4.1r2 library, which will be available in the near future. For more information, please refer to the macintosh.txt and windows.txt files [in the ./release_notes/ directory]. (see above). ==================new_functions.txt================================== This file contains a list of the new functions added with HDF 4.1r2. The functions in parenthesis were already present in the HDF library, and are included for clarity. C FORTRAN Description -------------------------------------------------------------------------------- (SDsetcompress) sfscompress compresses SDS (SDwritechunk) sfwchnk writes the specified chunk of NUMERIC data to the SDS (SDwritechunk) sfwcchnk writes the specified chunk of CHARACTER data to the SDS (SDreadchunk) sfrchnk reads the specified chunk of NUMERIC data to the SDS (SDreadchunk) sfrcchnk reads the specified chunk of CHARACTER data to the SDS (SDsetchunk) sfschnk makes the SDS a chunked SDS (SDsetchunkcache) sfscchnk sets the maximum number of chunks to cache (SDgetchunkinfo) sfgichnk gets info on SDS (SDsetblocksize) sfsblsz sets block size (SDisrecord) sfisrcrd checks if an SDS is unlimited (GRsetcompress) mgscompress compresses raster image GRsetchunk mgschnk makes a raster image a chunked raster image GRgetchunkinfo mggichnk gets info on a raster image GRsetchunkcache mgscchnk sets the maximum number of chunks to cache (Hgetlibversion) hglibver gets version of the HDF Library (Hgetfileversion) hgfilver gets version of the HDF file Vdeletetagref vfdtr deletes tag/ref pair ( HDF object) from a vgroup (VSfindclass) vsffcls finds class with a specified name in a vdata VSdelete vsfdlte deletes a vdata Vdelete vdelete deletes a vgroup ===================================================================== %%%4.1r1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.1 Release 1 February 21, 1997 INTRODUCTION This document describes the differences between HDF 4.0r2 and HDF 4.1r1. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1r1. The release notes provide more in-depth information concerning the topics discussed here. The HDF 4.1r1 documentation can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory: /HDF/Documentation/HDF4.1r1 First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems - Important Fixes New Features and Changes: ======================== o Attributes are now supported in both the vdata and vgroup APIs. In the vdata API, attributes can be attached to either vdata fields or vdatas; in the vgroup API, attributes can be attached to vgroups. This new functionality can also be used to attach attributes to vdatas and vgroups created by earlier versions of the HDF library. However, the old versions of the HDF library cannot read the new version vdatas and vgroups. A vdata/vgroup having attributes will become a new version vdata/vgroup. For more information, please refer to the file ../release_notes/vattr.txt, the man pages for the new functions, and the HDF 4.1 User's Guide. o Data chunking is now supported in SD scientific data sets. When data chunking is used, an n-dimensional SDS is stored as a series of n-dimensional chunks, improving performance on certain types of partial read operations. New routines for creating and manipulating chunked SD scientific data sets have been provided, and two preexisting SD I/O routines, SDreaddata and SDwritedata, have also been modified to work with chunked SDSs. For more information, please refer to the file ../release_notes/sd_chunk_examples.txt, the man page for sd_chunk, and the HDF 4.1 User's Guide. o Due to certain limitations in the way compressed SDS datasets are stored, data which has been compressed is not completely writable in ways that uncompressed datasets are. The "rules" for writing to a compressed dataset are as follows: (1) Write an entire dataset that is to be compressed. i.e. build the dataset entirely in memory, then write it out with a single call. (2) Append to a compressed dataset. i.e. write to a compressed dataset that has already been written out by adding to the unlimited dimension for that dataset. (3) For users of HDF 4.1, write to any subset of a compressed dataset that is also chunked. Please refer to the HDF 4.1 User's Guide for more information. o HDF now creates free format FORTRAN include files. In order to make FORTRAN 90 programs be able to use HDF include files (*.inc), HDF4.1r1 creates F90 versions of these files during the 'make' process on UNIX platforms, by replacing 'C' or 'c' in column 1 with '!'. Continuation lines in hdf.inc have been eliminated. The F90 version files are named as hdf.f90, dffunc.f90 and netcdf.f90. o Several performance improvements have been added. Test programs on SPARC 20/Solaris 2.5 show that when creating an hdf file with 2500 3D (10x10x10) float32 SDSs, the program execution speed is improved by 2.5 - 4.8 times, and SDend is faster by 4.3 - 20 times. o A new function, SDsetfillmode, has been added. It can be used to prevent SDwritedata from pre-filling the dataset with a user defined or default fill value, so that better performance can be obtained. o SGI has changed some compiler default settings in IRIX 6.2. We decided to explicitly define the settings of various ABI related options. For the 64 bit OS ("uname -s" returns IRIX64), HDF uses "-64 -mips4" code. For the traditional 32 bit OS ("uname -s" returns IRIX), HDF uses "-32 -mips2". To use n32 mode on IRIX64, HDF uses "-n32 -mips3" code. Note that in the previous release (4.0r2), HDF used only "-n32". In IRIX 6.1 and before, "-n32" defaulted to "-mips4" code but in IRIX 6.2, it defaults to mips3 or mips4 code. We decided to explicitly set it to "-n32 -mips3". Therefore, applications linking with the HDF library must be compiled with the same explicit ABI options. o This will be the last release that we support the CM5. HDF 4.1 Beta 1 USERS ONLY ------------------------- o The SD chunking routine names were changed to be more consistent with the SD interface. The names of the routines are now in lower case, after the two initial "SD" characters. For example, SDwriteChunk() has been changed to SDwritechunk(). o The _HDF_ENTIRE_VDATA variable has been changed to _HDF_VDATA. For those users already using it, a macro called _HDF_ENTIRE_VDATA has been added, which is defined as _HDF_VDATA. o You can now create an empty compressed SDS. Please refer to the ../release_notes/bugs_fixed.txt file for changes in this release. Platforms Tested: ================ HDF 4.1 Release 1 has been tested on the following platforms: CM5 Parallel I/O, 4.1.3_U1 DEC Alpha/Digital Unix 3.2 DEC Alpha/OpenVMS AXP 6.2 DEC VAX OpenVMS 6.2 Exemplar 9.03 Free BSD 2.2 HP-UX 9.03 HP-UX 10.10 IRIX 5.3 IRIX 6.2_64 IRIX 6.2_n32 IRIX 6.4_64 IRIX 6.4_n32 Linux A.OUT 1.2.4 Linux ELF 2.0.27 (C only) Macintosh PowerPC (C only) SP2 4.1 Solaris 2.5 Solaris_x86 2.5 (C only) SunOS 4.1.3 Windows NT/95 (C only) Known Problems: ============== o With the SD interface, you are unable to overwrite existing compressed data, that is not stored in "chunked" form. This is due to compression algorithms not being suitable for "local" modifications in a compressed datastream. o There are no plans to add the DF24writeref function to the DF24 interface. This function will be removed from the documentation. Important Fixes: =============== o If you opened a file in Read Only mode with the SD interface (using SDstart), it would create the file if the file did not exist. This no longer occurs. o HDF 4.0r2 did not recognize JPEG images created by HDF 3.3r4. This has been fixed. See the ../release_notes/bugs_fixed.txt file for more information on bugs fixed in this release. %%%4.1b1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.1 Beta 1 December 6, 1996 INTRODUCTION This document describes the differences between HDF 4.0r2 and HDF 4.1b1. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.1b1. The release notes provide more in-depth information concerning the topics discussed here. For documentation, please refer to the HDF 4.0r2 documentation which can be found on the NCSA ftp server (ftp.ncsa.uiuc.edu) in the directory /HDF/Documentation/HDF4.0r2. First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems - Important Fixes New Features and Changes: ------------------------- o Attributes are now supported in both the vdata and vgroup APIs. In the vdata API, attributes can be attached to either vdata fields or vdatas; in the vgroup API, attributes can be attached to vgroups. This new functionality can be used to attach attributes to vdatas and vgroups created by earlier versions of the HDF library. However, the old versions of the HDF library cannot read the new version vdatas and vgroups. A vdata/vgroup having attributes will become a new version vdata/vgroup. For more information, please refer to the file ../release_notes/vattr.txt, as well as the man pages for the new functions. o Data chunking is now supported in SD scientific data sets. When data chunking is used, an n-dimensional SDS is stored as a series of n-dimensional chunks, improving performance on certain types of partial read operations. New routines for creating and manipulating chunked SD scientific data sets have been provided, and two preexisting SD I/O routines, SDreaddata and SDwritedata, have also been modified to work with chunked SDSs. For more information, please refer to the file ../release_notes/sd_chunk_examples.txt, as well as the man page for sd_chunk. More information will be included in the HDF 4.1 documentation, which will be available with the release of HDF 4.1. o Due to certain limitations in the way compressed SDS datasets are stored, data which has been compressed is not completely writable in ways that uncompressed datasets are. The "rules" for writing to a compressed dataset are as follows: (1) Write an entire dataset that is to be compressed. i.e. build the dataset entirely in memory, then write it out with a single call. (2) Append to a compressed dataset. i.e. write to a compressed dataset that has already been written out by adding to the unlimited dimension for that dataset. (3) For users of HDF 4.1, write to any subset of a compressed dataset that is also chunked. Please refer to the ../release_notes/comp_SDS.txt file for more information. o A new file, ../release_notes/compile.txt, contains instructions on compiling applications on the supported platforms. If you encounter problems with it, please let us know at hdfhelp@ncsa.uiuc.edu. o SGI has changed some compiler default settings in IRIX 6.2. We decided to explicitly define the settings of various ABI related options. For the 64 bit OS ("uname -s" returns IRIX64), HDF uses "-64 -mips4" code. For the traditional 32 bit OS ("uname -s" returns IRIX), HDF uses "-32 -mips2". To use n32 mode on IRIX64, HDF uses "-n32 -mips3" code. Note that in the previous release (4.0r2), HDF used only "-n32". In IRIX 6.1 and before, "-n32" defaulted to "-mips4" code but in IRIX 6.2, it defaults to mips3 or mips4 code. We decided to explicitly set it to "-n32 -mips3". Therefore, applications linking with the HDF library must be compiled with the same explicit ABI options. Platforms Tested: ----------------- HDF 4.1b1 has been tested on the following platforms: DEC Alpha/Digital Unix 3.2 DEC Alpha/OpenVMS AXP v6.2 DEC VAX OpenVMS v6.2 Free BSD 2.2 HP-UX 9.03 IRIX 5.3 IRIX 6.2_64 IRIX 6.2_n32 Linux ELF 1.2.13 (C only) Macintosh PowerPC (C only) (not ready yet) SP2 4.1 Solaris 2.5 SunOS 4.1.3 Windows NT/95 (C only) YMP 9.0.2asC Known Problems: --------------- o With the SD interface, you are unable to overwrite existing compressed data, that is not stored in "chunked" form. This is due to compression algorithms not being suitable for "local" modifications in a compressed datastream. For more information, please refer to the ../release_notes/comp_SDS.txt file. o With 4.0r1p1, you could type "hdp list -a to get a list of the file attributes associated with a file. This does not currently work. o There are no plans to add the DF24writeref function to the DF24 interface. This function will be removed from the documentation. o When running "make test" on OpenVMS, Test 3 (float32) of the chunking tests fails, and has therefore been commented out. o When running the tests on Window NT/95, Test 2 (uint16) of the chunking tests fails, and will be commented out. Important Fixes: ---------------- o If you opened a file in Read Only mode with the SD interface (using SDstart), it would create the file if the file did not exist. This no longer occurs. o HDF 4.0r2 did not recognize JPEG images created by HDF 3.3r4. This has been fixed. See the ../release_notes/bug_fixed.txt file for more information on bugs fixed in this release. %%%4.0r2%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.0 Release 2 July 19, 1996 INTRODUCTION This document describes the differences between HDF 4.0r1p1 and HDF 4.0r2. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF 4.0r2. The documentation and release notes provide more in-depth information concerning the topics discussed here. The HDF 4.0 documentation can be found on the NCSA ftp server in the directory /HDF/Documentation/HDF4.0/Users_Guide. First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - New Features and Changes - Platforms Tested - Known Problems New Features and Changes: ------------------------- o HDF now supports unlimited number of access IDs and files IDs. o The vdata field size limit has been increased from 32000 to 65535. o The hdp utility has been updated to: - view a GR object - recognize the new compression methods - view descriptive annotations - display the library version of the HDF file o SDsetattr and GRsetattr now check for both MAX_ORDER and MAX_FIELD_SIZE. o The handling of DFNT_CHAR in all Fortran interfaces has been cleaned up. See release_notes/Fortran_APIs.txt for more information. o When appending compressed data onto the end of an unlimited dimension SDS, the SD interface no longer writes the fill-values in locations where they will immediately be over-written by data. This was done for the compression layer, but has the added enhancement of improving performance. o On the Cray, there were boundary problems when foreign data did not start from the 64-bit boundary. This has been fixed. o There are no longer the following name collisions with the HDF libraries: - AVS (HPread, HPwrite) - Windows SDK (_hread) - ODL library (_HDF_) o The 32-bit mode for IRIX 6.1 previously used the '-32' option which produces mips1 code. It has been changed to use '-n32' which produces mips4 code. This runs faster on the Power Challenges. Users who must use the '-32' option can link their code with the IRIX 5.3 HDF library. o The compression problems have been fixed when using HDF on IRIX 6.1 with the -n32 (see Known Problems below). o The zlib and jpeg libraries have been updated. The versions included with HDF 4.0 Release 2 are: zlib version 1.0.2 jpeg version 6a (7-Feb-96) o The hdfls utility has been updated to: - support the new compression modes - display the library version of the HDF file o Support for the 16-bit architecture has been pulled out of HDF. o The directories separator in the directory variable used by the function HXsetdir (Fortran equivalent: hxsdir) is the verticle bar ('|') now. It used to be the colon (':') symbol, but a colon is a legal symbol for a file pathname in the MacOS system. o The code has been rearranged so that most applications' binaries will be smaller. o A new routine, VSfpack(), has been added. Please see the HDF man page on how to use this routine. o A new routine, GRluttoref(), has been added. Please see the HDF man page on how to use this routine. o Several internal problems have been fixed with the GR interface. Changes in Compiling the Source Code: o A new compile option, '-DHAVE_NETCDF', has been added, to avoid conflicts in linking the HDF/MFHDF library with the original netCDF library. This is only available for the C-interface. However, keep in mind that you cannot read/write HDF files using the netCDF libraries. See section 2.4.3.2 in the INSTALL file for more information. o When compiling and installing HDF, the default location to place the binaries, has been changed from /usr/local/bin to NewHDF in the source directory. For example, assuming the library source is loaded at /usr/local/src/hdf, the following commands will result in the HDF binaries being placed in the directory /usr/local/src/hdf/dev/NewHDF. cd dev ./configure -v make make test make install o The Fortran test output has been cleaned up and shortened, when running "make test". Previously, the Fortran tests on the hdf/ side consisted of multiple Fortran programs invoked by a C frontend. The test programs were changed to subroutines and combined as one Fortran program. The C frontend was also changed to produce a 'directive' file, called fortest.arg, which contains directives to run the Fortran test program. Platforms Tested: ----------------- HDF 4.0r2 has been tested on the following platforms: AIX Linux ELF C90 MAC CM5 SP2 (single node) Digital Unix 3.2 Solaris_2.4 Exemplar 9.03 Solaris 2.5 Free BSD Solaris_x86 2.4 (C only) Fujitsu (C only) SunOS 4.1.4 HP-UX T3D (C only) IRIX 6.1 w/-n32 bit option VMS IRIX 6.1 w/-64 bit option Windows NT/95 IRIX 5.3 YMP Linux A.OUT Known Problems: --------------- o On the SunOS platform, there is a bug when using sfscal()/sfgcal() routines with gcc and f77. o On the VMS platform, there is a bug with float64 data. o For IRIX 6.1, the stdio.h file gives a false warning message if both the '-n32' and '-ansi' options are used for the C compiler. We have temporarily removed the '-ansi' option from our autoconfiguration for the Irix6_32 system, to avoid these messages. We have verified that the culprit in stdio.h has been corrected in IRIX 6.2, and plan to put the '-ansi' option back in our next release. o The compression tests produce errors for FLOAT32 data if the '-O' option is used on IRIX 6.1, for both the '-64' bit and '-n32' bit modes. It did not produce errors when using the '-32' bit option or when not using the '-O' option. We are unsure whether the errors are due to the compression code or the IRIX C optimizer. For now, we have chosen to compile the HDF library without the '-O' option, while we investigate the problem. %%%4.0r1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.0 Release 1 February 7, 1996 INTRODUCTION This document describes the differences between HDF4.0r1 and HDF3.3r4. It is written for people who are familiar with previous releases of HDF and wish to migrate to HDF4.0r1. The documentation and release notes provide more in-depth information concerning the topics discussed here. The HDF 4.0 documentation can be found on the NCSA ftp server in the directory /HDF/Documentation/HDF4.0/Users_Guide. For more history behind the implementation of the items listed here, refer to the ABOUT_4.0.alpha, ABOUT_4.0b1 and ABOUT_4.0b2 files. First-time HDF users are encouraged to read the FAQ in this release for more information about HDF. Users can also look at the home page for HDF at: http://hdf.ncsa.uiuc.edu/ If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu CONTENTS - Important Changes (that will affect you) - New Features and Changes - Changes in Utilities - Known Problems Important Changes: ----------------- 1. Several changes have been made to the libraries in HDF4.0 which affect the way that users compile and link their programs: * The mfhdf library has been renamed to libmfhdf.a from libnetcdf.a in previous releases. * HDF 4.0 libraries now use v5 of the Independent JPEG Group (IJG) JPEG file access library. * Gzip library libz.a is added in HDF4.0r1, in order to support "deflate" style compression of any object in HDF files. Due to these changes, users are required to specify four libraries when compiling and linking a program: libmfhdf.a, libdf.a, libjpeg.a and libz.a, even if your program does not use JPEG or GZIP compression. For example: For C: cc -o myprog myprog.c -I \ \ or cc -o myprog myprog.c -I \ -L -lmfhdf -ldf -ljpeg -lz For FORTRAN: f77 -o myprog myprog.f \ \ or f77 -o myprog myprog.f -L \ -lmfhdf -ldf -ljpeg -lz NOTE: The order of the libraries is important: libmfhdf.a first, then libdf.a, followed by libjpeg.a and libz.a. This is also discussed in Items 1, 2, and 3 of the New Features and Changes section of this document. 2. The HDF 4.0 library will ONLY compile with ANSI C compilers. See Item 4 in the New Features and Changes section of this document for more information. 3. The HDF library and netCDF library on Unix systems can now be automatically configured and built with one command. See Item 5 in the New Features and Changes section of this document for more information. 4. In HDF 4.0, the FORTRAN programs dffunct.i and constant.i have been changed to dffunct.inc and hdf.inc. See Item 16 in the New Features and Changes section of this document for more information. 5. Platforms tested on: IRIX (5.3, 6.1 (32 bit and 64 bit)), SunOS 4.1.4, Solaris (ver 2.4, 2.5), Solaris x86, HP-UX, Digital Unix, AIX, LINUX (A.OUT), CM5, YMP, FreeBSD, C90, Exemplar, Open VMS, and SP2 (single node only). HDF 4.0 is not yet available on the Macintosh for HDF4.0r1. 6. The HDF 4.0 binaries for each tested platform are available. Unix binaries are located in the bin/ directory. Binaries for Windows NT are located in the zip/ directory. New Features and Changes: ------------------------ 1. Changes to the mfhdf library The mfhdf library has been renamed to libmfhdf.a from libnetcdf.a in previous releases. To link a program with HDF4.0r1 libraries, four libraries are required: libmfhdf.a, libdf.a, libjpeg.a and libz.a. See Item 1 of 'Important Changes' for examples of how you would compile and link your programs. 2. JPEG Group v5b library HDF Version 4.0 libraries now use v5 of the Independent JPEG Group (IJG) JPEG file access library. The JPEG library will need to be linked with user's applications whether they are compressed with JPEG or not. See Item 1 of 'Important Changes' for examples of how you would compile and link your programs. 3. Gzip library added New with this release is support for gzip "deflate" style compression of any object in an HDF file. This is supported through the standard compression interface function calls (HCcreate, SDsetcompress, GRsetcompress). The ABOUT_4.0b2 file contains additional information on this. See Item 1 of 'Important Changes' for examples of how you would compile and link your programs. 4. ANSI C only As was previously noted in the HDF newsletters, this release of the HDF library will compile only with ANSI C compilers. This shift to ANSI C compliance has been accompanied by a large clean up in the source code. An attempt has been made to remove all warnings and informational messages that the compilers on supported platforms occasionally emit, but this may not be completely clean for all user sites. 5. Auto configuration Both the HDF library and netCDF library on Unix systems now use the same configure script and can be configured uniformally with one command. See the README and the INSTALL files at the top level of HDF4.0r1 for detailed instructions on configuration and installation. A consequence of the auto configuration is that on UNIX systems without FORTRAN installed, the top level config/mh- will need to have the 'FC' macros defined to "NONE" for correct configuration. 6. New version of dimension record In HDF4.0b1 and previous releases of the SDS interface, a vgroup was used to represent a dimension. The vgroup had a single field vdata with a class of "DimVal0.0". The vdata had number of records, with each record having a fake value from 0, 1, 2 ... , ( - 1). The fake values were not really required and took up a large amount of space. For applications that created large one dimensional array datasets, the disk space taken by these fake values almost doubled the size of the HDF file. In order to omit the fake values, a new version of dimension vdata was implemented. The new version uses the same structure as the old version. The only differences are that the vdata has only 1 record with a value of and that the vdata's class is "DimVal0.1", to distinguish it from the old version. No change was made in unlimited dimensions. Functions added to support this are: - SDsetdimval_comp -- sets backward compatibility mode for a dimension. The default mode is compatible in HDF4.0r1, and will be incompatible in HDF4.1. See the man page of SDsetdimval_comp(3) for detail. - SDisdimval_bwcomp(dimid) -- gets the backward compatibility mode of a dimension. See the man page of SDisdimval_bwcomp(3) for detail. 7. Reading CDF files With HDF 4.0 limited support for reading CDF files was added to the library. This support is still somewhat in the development stage and is therefore limited. To begin with, unlike the netCDF merger, the CDF API is not supported. Rather, the SD and netCDF APIs can be used to access information pulled out of CDF files. The type of files supported are limited to CDF 2.X files. The header information is different between CDF 1.X and 2.X files. In addition, all of the files must be stored as single-file CDFs in network encoding. If there is user demand, and support, the types of CDF files that are readable may be increased in the future. 8. Parallel I/O interface on CM5 An extension using the parallel IO in CM5 has been added to the SDS interface. Initial tests have resulted in about 25 MBytes/second IO throughput using the SDA (Scalable Disk Array) file system. The library provides interfaces for both C* and CMF programming languages. The ABOUT_4.0.alpha file has more information concerning this. Users will find some examples in the directory mfhdf/CM5/Examples. The parallel I/O interface stores scientific datasets in external files. New options have been added to hdfls and hdfpack to handle them. A new utility, hdfunpac, was created for external files handling, too. 9. Support for SGI Power Challenge running IRIX6.1 Power Challenge is now supported, both in the native 64-bit and the 32-bit objects modes. Note that the Power Challenge native 64 bits objects use 64 bits long integers. Users should be careful when using the netcdf interface. They should declare their variables as "nclong", not "long". 10. Multi-file Annotation Interface (ANxxx) The multi-file annotation Interface is for accessing file labels and descriptions, and object labels and descriptions. It allows users to keep open more than one file at a time, and to access more than one annotation at a time. It also allows multiple labels and multiple descriptions to be applied to an HDF object or HDF file. 11. Multi-file Raster Image (GRxxx) interface The new Generic Raster (GR) interface provides a set of functions for manipulating raster images of all kinds. This interface allows users to keep open more than one file at a time, and to "attach" more than one raster image at a time. It supports a general framework for attributes within the RIS data-model, allowing 'name = value' style metadata. It allows access to subsamples and subsets of images. The GRreqlutil and GRreqimageil functions allow for different methods of interlacing images in memory. The images are interlaced in memory only, and are actually written to disk in "pixel" interlacing. 12. Compression for HDF SDS Two new compression functions have been added to the SD interface for HDF 4.0: SDsetcompress and SDsetnbitdataset. SDsetcompress allows users to compress a scientific dataset using any of several compression methods. Initially three schemes, RLE encoding, an adaptive Huffman compression algorithm, and gzip 'deflation' compression are available. SDsetnbitdataset allows for storing a scientific dataset using integers whose size is any number of bits between 1 and 32 (instead of being restricted to 8, 16 or 32-bit sizes). Access to the data stored in an n-bit data item is transparent to the calling program. The ABOUT_4.0.alpha file has an in-depth description concerning this ("n-bit SDS" listed under Item 2). 13. External Path Handling New functions have been added to allow applications to specify directories to create or search for external files. - HXsetcreatedir (hxscdir for FORTRAN) - HXsetdir (hxsdir for FORTRAN) 14. I/O performance improvement HDF 4.0 unofficially supports file page buffering. With HDF 4.1 it will be officially supported. The file page buffering allows the file to be mapped to user memory on a per page basis i.e. a memory pool of the file. With regards to the file system, page sizes can be allocated based on the file system page-size or in a multiple of the file system page-size. This allows for fewer pages to be managed as well as accommodating the user's file usage pattern. See the top level INSTALL file and the release_notes/page_buf.txt file for creating the library with this support and using it. 15. Improvement in memory usage and general optimizations Considerable effort was put into this release (since the b2 release) to reduce the amount of memory used per file and by the library in general. In general terms, we believe that the library should have at least half as large of a memory footprint during the course of its execution and is more frugal about allocating large chunks of memory. Much time was spent optimizing the low-level HDF routines for this release to be faster than they have been in the past also. Applications which make use of files with many (1000+) datasets should notice significant improvements in execution speed. 16. In hdf/ there are two files for FORTRAN programs to include the values and functions defined in HDF. They were originally named as constant.i and dffunct.i. The extension .i caused problems on some machines since *.i is used by cc as an "intermediate" file produced by the cpp preprocessor. In HDF 4.0 dffunct.i has been changed to dffunct.inc, and constant.i has been changed to hdf.inc. Users' existing FORTRAN application programs need to make the corresponding changes, if they include the .i files, in order to compile with HDF4.0. 17. Limits file A new file, limits.txt, has been added to the ftp server. It is aimed at HDF applications programmers and defines the upper bounds of HDF 4.0. This information is also found in the hdf/src/hlimits.h file. Refer to the ABOUT_4.0.alpha for historical information concerning this. 18. Pablo available HDF4.0 supports creating an instrumented version of the HDF library(libdf-inst.a). This library, along with the Pablo performance data capture libraries, can be used to gather data about I/O behavior and procedure execution times. See the top level INSTALL file and the hdf/pablo/README.Pablo file for further information. 19. Support for the IBM SP-2 The HDF library has been ported to run in a single SP2 node. It does not support the parallel or distributed computing for multiple SP-2 nodes yet. 20. Miscellaneous fixes - To avoid conflicts with C++, internal structures' fields which were named 'new' have been renamed. - The maximum number of fields in a vdata now is decided by VSFIELDMAX. - The platform number subclass problem when an external data file was in Little_endian format has been fixed. - Unlimited dimension was not handled correctly by the HDF3.3r4 FORTRAN interface. This problem has been fixed in HDF4.0r1. Changes to utilities: -------------------- o hdf/util/ristosds Ristosds now converts several raster images into a 3D uint8, instead of float32, SDS. o hdf/util/hdfls New options have been added to support the parallel I/O interface on CM5. o hdf/util/hdfpack New options have been added to support the parallel I/O interface on CM5. o hdf/util/hdfunpac This is a new utility for external file handling for the parallel I/O interface on CM5. o mfhdf/dumper/hdp Hdp is a new command line utility designed for quick display of contents and data objects. It can list the contents of hdf files at various levels with different details. It can also dump the data of one or more specific objects in the file. See hdp.txt in the release notes for more information. Known Problems: -------------- o On the IRIX4 platform, fp2hdf creates float32 and float64 values incorrectly. o On the SP2, the hdp command gives a false message of "Failure to allocate space" when the hdf file has no annotations to list. o On the C90, hdfed fails inconsistently when opening hdf files more than once in the same hdfed session. o Currently there is a problem in re-writing data in the middle of compressed objects. o VMS gives an error on the test for Little Endian float64. o If the external element test in hdf/test/testhdf fails and there is no subdirectory "testdir" in hdf/test/, create one via "mkdir" and run the test again. (The "testdir" should have been created by "make". But the "make" in some old systems does not support the creation commands.) %%%4.0b2%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.0 Beta 2 Nov 14, 1995 CONTENTS 1. The mfhdf side library is renamed to libmfhdf.a, versus libnetcdf.a in previous releases 2. New version of dimension record 3. New features GR interface Gzip library is added Unified configuration of library I/O performance improvement 4. New functions added Fortran functions hxscdir and hxsdir SDsetdimval_comp SDisdimval_bwcomp 5. SGI Power Challenge running IRIX6.1 is now supported 6. Pablo available 7. Platforms tested 8. Changes in release notes 9. Bug fixes and Known problems 1. The mfhdf side library is renamed to libmfhdf.a, versus libnetcdf.a in previous releases. To link a program with HDF4.0b2 libraries, one needs four libraries, libmfhdf.a, libdf.a, libjpeg.a and libz.a (see "Gzip library is added" in item 3 below): cc -o myprog myprog.c -I \ libmfhdf.a libdf.a libjpeg.a libz.a Note, the order of the libraries is important. 2. New version of dimension record HDF4.0b1 and previous releases use a vgroup to represent a dimension. The vgroup has a single field vdata with class "DimVal0.0". The vdata has number of records, each record has a fake value from 0, 1, 2 ... , ( - 1 ). The fake values are not really required and take a lot of space. For applications that create large one dimensional array datasets the disk space taken by these fake values almost double the size of the HDF file. In order to omit the fake values, a new version of dimension vdata is proposed. The new version uses the same structure as the old version. The only differences are that the vdata has only 1 record with value and that the vdata's class is "DimVal0.1" to distinguish it from the old version. No change is made in Unlimited dimensions. See file dimval.txt in subdirectoy release_notes/ of HDF4.0b2 release for our policy on the backward compatibility of this dimension version. 3. New features . New with this beta release is the support for different methods of interlacing images in memory. This feature is supported through the GRreqlutil and GRreqimageil functions rescribed in the mf_ris.txt document in this directory. Please note that the images are interlaced in memory only, all images are actually written to disk in "pixel" interlacing. . Gzip library is added New with this release is support for gzip "deflate" style compression of any object in an HDF file. This is supported through the standard compression interface function calls (HCcreate, SDsetcompress, GRsetcompress) by using the COMP_CODE_DEFLATE parameter for the coding type. The comp_info structure has a new member, deflate.level, which specifies how much effort to expend trying to compress data. Values for deflate.level must be between 1-9, with 1 being small amounts of effort (time) and 9 being maximum effort (most time and compression), the default value is 6. Currently, due to our use of the gzip "zlib" library for support of this feature, users must link with the "libz.a" library produced by zlib. (See item1 above). cc -o myprog myprog.c -I \ libmfhdf.a libdf.a libjpeg.a libz.a Note, the order of the libraries is important. Also, this method of compression currently has several known bugs when used on a 64-bit architecture (DEC Alpha processors, Cray machines, and SGI Power Challenge machines in 64-bit "mode"). . Unified configuration of library Both sides of the library now use the same configure script and can be configured uniformly through one makefile fragment. Please see the top-level INSTALL file in the distribution for further details. . I/O performance improvement This version of the distribution also has preliminary support for file page buffering. Note that is a *Beta* release and is not supported officially. As such it is is provided as is. The file page buffering allows the file to be mapped to user memory on a per page basis i.e a memory pool of the file. With regards to the file system, page sizes can be allocated based on the file system page-size or if the user wants in some multiple of the file system page-size. This allows for fewer pages to be managed along with accommodating the users file usage pattern. Please see the documentation in 'release_notes/page_buf.txt'. We have also reduced the memory requirements for several of the internal HDF library data structures, for greater efficiency. 4. Functions added . Fortran interface functions added for the set external path features. They are hxscdir and hxsdir. See the man page of HXsetcreatedir(3) and HXsetdir(3) for detail. . SDsetdimval_comp -- sets backward compatibility mode for a dimension. The default mode is compatible in HDF4.0b2, and will be incompatible in HDF4.1. See the man page of SDsetdimval_comp(3) for detail. . SDisdimval_bwcomp(dimid) -- gets the backward compatibility mode of a dimension. See the man page of SDisdimval_bwcomp(3) for detail. 5. SGI Power Challenge running IRIX6.1 is now supported Power Challenge is now supported, both in the native 64-bit and the 32-bit objects modes. Note that the Power Challenge native 64 bits objects use 64 bits long integers, users should be careful when using the netcdf interface. They should declare their variables as "nclong", not "long". 6. Pablo available This version of the distribution has support to create an instrumented version of the HDF library(libdf-inst.a). This library along the Pablo performance data capture libraries can be used to gather data about I/O behaviour and procedure execution times. Please see the documentation release_notes/Pablo.txt in the distribution for further details. 7. Platforms tested HDF4.0b2 has been tested on the following systems: SunOS 4.1.3, SunOS 5.3 and 5.4(Solaris 2.3 and 2.4), Linux_a.out, Linux_elf, SGI/IRIX5.2, SGI/IRIX5.3, SGI Power Challenge/IRIX6.1 (32- and 64-bit), HP/UX 9.01, IBM RS6000/AIX, Cray C90, Cray YMP, DEC alpha/UNIX (OSF), DecStation/MIPSEL (ncdump doesn't work), Free BSD 2.0, Solaris_x86, Convex Exemplar/HPUX, and CM5 parallel I/O. See the INSTALL file at the top level of HDF4.0b2 for more details. 8. Changes in release notes The directory release_notes/ contains writeups for the alpha and beta releases of HDF4.0. Those files can be used as temporary documents for HDF4.0 before the official documentation is available. Newly added: ABOUT_4.0b2, Pablo.txt, dimval.txt, and page_buf.txt Files changed: bug_fixed.txt and parallel_CM5.txt. AOUBT_4.0.alpha is also included. 9. Fixes and Known problems Problems fixed: . To avoid conflicts with C++, internal structures' fields which were named 'new' have been renamed. . Maximum number of fields in a vdata now is decided by VSFIELDMAX. . Vshow and hdp are fixed. Now they can handle as many fields as defined by VSFIELDMAX. . Fixed platform number subclass problem when external data file was in Little_endian format. . A file hdf/src/hlimits.h has been added to hold definitions for maximum number of open files and other limits. . Miscelianeous fixes Known problems: . Hfidinquire not included in binaries . Gzip doesn't work on 64-bit machines. . Currently there is a problem in appending data to compressed objects. . Hfidinquire is in the source code, but it is not included in the pre-compiled code. If your program uses Hfidinquire, you need to re-compile libdf.a. %%%4.0b1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT HDF4.0 Beta 1 July 25, 1995 CONTENTS 1. New features in the HDF4.0 Beta 1 release 2. Bugs fixed and known problems 3. Platforms tested 4. Installation of HDF4.0 Beta 1 on WindowsNT/95 5. Known problems in compilation, testing and installation of HDF4.0b1 6. Installing without FORTRAN support 1. New features in HDF4.0 Beta 1 release o Auto configuration It is now possible to automatically configure and build both the HDF library and netCDF library with one command. See the README and the INSTALL files at the top level of HDF4.0beta1 for detailed instructions on configuring and installation. o Multi-file Annotation Interface (ANxxx) The multi-file annotation Interface is for accessing file labels and descriptions, and object labels and descriptions. It allows users to keep open more than one file at a time, and to access more than one annotation at a time. It also allows multiple labels and multiple descriptions to be applied to an HDF object or HDF file. A draft of the documentation for this interface is in ./mf_anno.txt. o Multi-file Raster Image (GRxxx) interface The new Generic Raster (GR) interface provides a set of functions for manipulating raster images of all kinds. This interface allows users to keep open more than one file at a time, and to "attach" more than one raster image at a time. It supports a general framework for attributes within the RIS data-model, allowing 'name = value' style metadata. It allows access to subsamples and subsets of images. HDF4.0beta1 includes a C interface only. The Fortran interface will be available in the next release. A draft of the documentation for this interface is in ./mf_ris.txt. o New Compression Algorithms and interface A new low-level compression interface has been added to HDF which allows any data-object to be compressed using a variety of algorithms. Currently only two compression algorithms are supported: Run-Length Encoding (RLE) and adaptive Huffman. A draft of the documentation for this interface is in ./compression.txt o JPEG Group v5b library HDF Version 4.0 libraries now use v5 of the Independent JPEG Group (IJG) JPEG file access library. For more details about JPEG library see ./JPEG_v5b.txt The JPEG library will need to be linked with a user's applications whether they are compressed with JPEG or not. For example on a SUN SPARC, if the .h files are in the directory "incdir", and all libraries are in "libdir," the following command should be used to compile a C program "myprog.c": cc -DSUN -DHDF -Iincdir myprog.c libdir/libnetcdf.a \ libdir/libdf.a /libdir/libjpeg.a -o myprog or cc -DSUN -DHDF -Iincdir myprog.c -L libdir -lnetcdf \ -ldf -ljpeg -o myprog Note that the order is important: libnetcdf.a must occur first, then libdf.a, and then libjpeg.a. For FORTRAN programs, use command line: f77 -o myprogf myprogf.f libdir/libnetcdf.a \ libdir/libdf.a libdir/libjpeg.a or f77 -o myprogf myprogf.f -L libdir -lnetcdf -ldf -ljpeg Note that the order is important: libnetcdf.a, then libdf.a and then libjpeg.a. o Compression for HDF SDS (not completely working) Work is almost complete on the addition of two new compression functions to the SD interface. One function, which still has some known bugs, will allow users to compress a scientific dataset using any of several compression methods. Initially two schemes, RLE encoding and an adaptive Huffman compression algorithms, will be available. A second function is available for storing a scientific dataset using integers whose size is any number of bits between 1 and 32 (instead of being restricted to 8, 16 or 32-bit sizes). A draft of the documentation for these functions is in ./comp_SDS.txt o External Path Handling New functions have been added to allow applications to specify directories to create or search for external files. More explanation can be found in: ./external_path.txt. o Parallel I/O for the CM5 An extension using the parallel I/O facilities on a CM5 has been added to the SDS interface. Initial tests have resulted in about 25 MBytes/second I/O throughput using the SDA (Scalable Disk Array) file system. The library provides interfaces for both C* and CMF programming languages. See: ./parallel_CM5.txt for details. o HDF dumper Hdp is a command line utility designed for quick display of contents and data of HDF3.3 objects, RIS, SDS, Vdata, and Vgroup. It can list the contents of hdf files at various levels with different details. It can also dump the data of one or more specific objects in the file. See: ./hdp.txt for details. Currently hdp works on SunOS and LINUX only. 2. Bugs fixed and known problems Several bugs or problems, such as failure in setting and getting scales for unlimited dimensions, missing Fortran version of VSQxxxx functions, failure in defining more than 36 fields in Vdatas, etc. were fixed in this beta release. For more details about fixed and un-fixed bugs and problems please see: ./bug_fixed.txt. 3. HDF4.0 Beta 1 has been tested on the following systems: SunOS 4.1.3, SunOS 5.3 (Solaris 2.3), Linux, SGI/IRIX5.3, SGI Power Challenge/IRIX6.0 (32-bit mode only), HP/UX 9.01, IBM RS6000/AIX (C only), C3880/ConvexOS,11.0, CM5, Cray C90, DEC alpha/OSF (C only), DecStation/MIPSEL (C only), Windows NT, Free BSD 2.0, and Convex Exemplar/HPUX. See the INSTALL file at the top level of HDF4.0b1 for more details. 4. Installing HDF4.0 Beta 1 on Windows NT and Windows 95 Since Windows NT, Windows '95 (Chicago) and Windows 3.1 (with the Win 32s extensions) all are designed to run the same 32-bit code, we have decided to support only 32-bit libraries and code on the MS-Windows platform. To build the HDF, JPEG and netCDF libraries and utilities, follow the instructions listed in: ./install_winNT.txt. 5. Known problems in compilation,testing and installation of HDF4.0b1: . On SunOS, tsdmmsf.f in hdf/test/fortest fails . On C90, mfhdf/fortran test doesn't configure correctly. The adaptive Huffman algorithm does not work right either. Due to this problem, when running hdf/test/testhdf the test module comp prints out error messages. . On DecStation/MIPSEL, ncdump gives a segmentation fault. . The Fortran interface has not been tested on IBM RS6000, DecStation/MIPSEL, and Dec Alpha/OSF because a Fortran compiler is not available on those machines in our group. . If the external element test in hdf/test/testhdf fails and there is no subdirectory "testdir" in hdf/test/, create one via "mkdir" and run the test again. (The "testdir" should have been created by "make". But the "make" in some old systems does not support the creation commands.) . A bug was found in the "mfhdf.h" file late in the testing stage. The error occured in the CM5 parallel I/O extension only. The fix is not included in the source release, but it is avalable in the binary release for the CM5 version. Please retrieve the fix there. . SDsetcompress does not work correctly. . Hdp now works on SunOS and LINUX only. Commands dumpsds, dumpvd and dumpvg have different problems on other platforms. See mfhdf/dumper/README for more details. 6. Installing without FORTRAN support: . On UNIX systems without a FORTRAN system installed, the config/mh- will need to have the 'FC' macros defined to "NONE" for correct configuration and the target "allnofortran" should be used to build the distribution, instead of the target "all". %%%4.0alpha%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ABOUT_4.0.alpha This file was last updated: November 8, 1994 INTRODUCTION This is a preliminary document describing the differences between HDF4.0 (Alpha) and HDF3.3r3. It is written for people who already use HDF3.3r3 or earlier versions and wish to be HDF4.0 Alpha testers. Special emphasis is given to changes that might be required in existing code. The files ABOUT_3.3r3, ABOUT_3.3r2 and ABOUT_3.3r1 which were released along with previous releases contain detailed descriptions of HDF3.3. Those files can be found in this directory. First-time HDF users are encouraged to read the FAQ file in directory HDF/ for more information about HDF and where to get HDF documentation. If you have any questions or comments, please send them to: hdfhelp@ncsa.uiuc.edu. Contents 1. Changes in include file names for FORTRAN programs 2. New features supported by HDF4.0 ANSI C only n-bit SDS Reading CDF files Parallel I/O interface on CM5 Installing HDF Libraries With CM5 Parallel IO Extension 3. Changes in HDF utilities hdp -- HDF dumper ristosds hdfls hdfpack hdfunpac 4. Platforms tested 5. Limits of the current release 1. Changes in include file names for FORTRAN programs In hdf/ there are two files for FORTRAN programs to include the values and functions defined in HDF. They were originally named as constant.i and dffunct.i. The extension .i causes problems on some machines since *.i is used by cc as an "intermediate" file produced by the cpp preprocessor. In HDF 4.0 dffunct.i is changed to dffunct.inc, and constant.i is changed to hdf.inc. Users' existing FORTRAN application programs need to make the corresponding changes, if they include the .i files, in order to compile with HDF4.0. 2. New Features supported by HDF4.0 ANSI C only As previously noted in the HDF newsletters, the next major release of the HDF library will compile only with ANSI C compilers. Backward compatibility will be provided through an ANSI->K&R filter which will need to be run on each source file in order to convert the ANSI style code into K&R style code. Currently the entire HDF library has been converted to ANSI code, but the filter is not yet in place. Future alpha releases may have the code filter in place, but it will definitely be in place for the first beta release. This shift to ANSI C compliance has been accompanied by a large cleanup in the source code. An attempt has been made to remove all warnings and informational messages that the compilers on supported platforms occasionally emit, but this may not be completely clean for all user sites. n-bit SDS Support for n-bit integer data has been incorporated into this release of the HDF library. The n-bit support is currently incorporated into the call to SDsetnbitdataset, future releases may incorporate high level access through the DFSD interface also. Access to the data stored in an n-bit data item is transparent to the calling program. For example to store an unsigned 12-bit integer (which is represented unpacked in memory as an unsigned 16-bit integer), with no sign extension or bit filling and which starts at bit 14 (counting from the right with bit zero being the lowest) the following setup & call would be appropriate: intn sign_ext = FALSE; intn fill_one = FALSE; intn start_bit= 14; intn bit_len = 12; SDsetnbitdataset(sds_id,start_bit,bit_len,sign_ext,fill_one); Further reads and writes to this dataset would transparently convert the 16-bit unsigned integers from memory into 12-bit unsigned integers stored on disk. The corresponding FORTRAN function name is sfsnbit which takes the same parameters in the same order. A breakdown of the parameters to the SDsetnbitdataset call is as follows: int32 sds_id - The id of a scientific dataset returned from SDcreate or SDselect. intn start_bit - This value determines the bit position of the highest end of the n-bit data to write out. Bits in all number- types are counted from the right starting with 0. For example, in the following bit data, "01111011", bits 2 and 7 are set to 0 and all the other bits are set to one. intn bit_len - The number of bits in the n-bit data to write, including the starting bit, counting towards the right (i.e. lower bit numbers). For example, starting at bit 5 and writing 4 bits from the following bit data, "01111011", would write out the bit data, "1110", to the dataset on disk. intn sign_ext - Whether to use the top bit of the n-bit data to sign-extend to the highest bit in the memory representation of of the data. For example, if 9-bit signed integer data is being extracted from bits 17-25 (nt=DFNT_INT32, start_bit=25, bit_len=9, see below for full information about start_bit & bit_len parameters) and the bit in position 25 is a 1, then when the data is read back in from the disk, bits 26-31 will be set to a 1, otherwise bit 25 will be a zero and bits 26-31 will be set to 0. This bit-filling takes higher precendence (i.e. is performed after) the fill_one (see below) bit-filling. intn fill_one - Whether to fill the "background" bits with 1's or 0's. The "background" bits of a n-bit dataset are those bits in the in-memory representation which fall outside of the actuall n-bit field stored on disk. For example, if 5 bits of an unsigned 16-bit integer (in-memory) dataset located in bits 5-9 are written to disk with the fill_one parameter set to TRUE (or 1), then when the data is read back into memory at a future time, bits 0-4 and 10-15 would be set to 1. If the same 5-bit data was written with a fill_one value of FALSE (or 0), then bits 0-4 and 10-15 would be set to 0. This setting has a lower precedence (i.e. is performed first) than the sign_ext setting. For example, using the sign_ext example above, bits 0-16 and 26-31 will first be set to either 1 or 0 based on the fill_one parameter, and then bits 26-31 will be set to 1 or 0 based on bit-25's value. Reading CDF files With HDF 4.0 limited support for reading CDF files was added to the library. This support is still somewhat in the development stage and is therefore limited. To begin with, unlike the netCDF merger, the CDF API is not supported. Rather, the SD and netCDF APIs can be used to access information pulled out of CDF files. The type of files supported are limited to CDF 2.X files. The header information is different between CDF 1.X and 2.X files. In addition, all of the files must be stored as single-file CDFs in network encoding. If there is user demand, and support, the types of CDF files readable may be increased in the future. Parallel I/O interface on CM5 An extension using the parallel IO in CM5 is added to the SDS interface. Initial tests have resulted in about 25 MBytes/second IO throughput using the SDA (Scalable Disk Array) file system. The library provides interfaces for both C* and CMF programming languages. Read the section "Installing HDF Libraries With CM5 Parallel IO Extension" below for specific installation instructions. Users will find some examples in the directory mfhdf/CM5/Examples. Please send comments, bugs reports, etc. to acheng@ncsa.uiuc.edu. The parallel I/O interface stores scientific datasets in external files. New options have been added to hdfls and hdfpack to handle them. A new utility program, hdfunpac, is created for external files handling too. See the man pages for details. Installing HDF Libraries With CM5 Parallel IO Extension The current alpha version requires two major steps to install the HDF libraries (libdf.a and libnetcdf.a). Works are in progress to make it simpler in the production release. Bear with us for now. 1) Compile and install the ordinary HDF libraries, include files and utilities according to the instructions for a Sun Microsystem machine. 2) To make the HDF library with CM5 parallel IO extension: There are two new libraries, libdfcm5.a and libnetcdfcm5.a that are similar to libdf.a and libnetcdf.a. For libdf.a cd hdf cp MAKE.CM5 Makefile cp src/Makefile.CM5 src/Makefile make libdf # create the parallel IO libdf.a # to install it in /usr/local/lib cp src/libdf.a /usr/local/lib/libdfcm5.a ranlib /usr/local/lib/libdfcm5.a For libnetcdf.a cd mfhdf # edit CUSTOMIZE to use "gcc" as the CC compiler # and add "-DCM5" to the CFLAGS variable. ./configure (cd libsrc; make ) # compile the library # to install it in /usr/local/lib cp libsrc/libnetcdf.a /usr/local/lib/libnetcdfcm5.a ranlib /usr/local/lib/libnetcdfcm5.a 3. Changes in HDF utilities hdp -- HDF dumper A new utility hdp is under development to list contents of HDF files and to dump data of HDF objects. A prototype is included in HDF4.0 Alpha for users to play with and comment on. Development will continue based on users' feedback. More information is contained in HDF/HDF4.0.alpha/mfhdf/dumper/README. ristosds Ristosds now converts several raster images into a 3D uint8, instead of float32, SDS. hdfls New options to recognize external elements. hdfpack New options to pack external elements back into the main file. hdfunpac New utility program to unpack scientific datasets to external elements. Can be used to prepare for CM5 parallel IO access. 4. HDF 4.0 Alpha has been tested on the following machines Platform 'base library' HDF/netCDF --------------------------------------------------------------- Sun4/SunOs X X Sun4/SOLARIS X X IBM/RS6000 X X SGI/IRIX4 X X Convex/ConvexOS * X X Cray Y-MP/UNICOS X X Cray/C90 X X NeXT/NeXTSTEP X X HP/UX 9.01 X X DecStation/MIPSEL X X IBM PC - MSDOS ** *** IBM PC - Windows 3.1 ** *** IBM PC - Windows NT X X DEC Alpha/OSF X X CM5/ X X Fujitsu VP/UXPM X Intel i860 X Mac/MacOS VMS * When compiling the mfhdf section of the library on a Convex3 you will need to set the environment variable 'MACHINE' to 'c3' before running the configure script. ** There is no FORTRAN support for either PC version of HDF4.0 Alpha *** The netCDF half of the HDF/netCDF merger is not working correctly, but the multi-file SD interface is working correctly. 5. Limits of the current release Sometimes it is important for HDF users to be aware of certain limits in using HDF files and HDF libraries. This section is aimed at HDF applications programmers and reflects upperbounds as of HDF 4.0. Limits that are #define'd are fully capitalized and the file where the symbol is defined is given in parentheses at the end of the sentence. If the #define's are changed in order to meet the needs of an application, it is important to make sure that all other users, who would share the HDF library and the hdf files of the application, are aware of the changes. If a limit has no #define, the size of the maximum storage allocated for that item is given; it would, generally, require a large amount of modification of the HDF library to change. If a limit is listed as a number type (e.g. int16) then it refers to the largest number that can be represented using that type. That is: int16 -- 32,767 int32 -- 2,147,483,647. H-Level Limits -------------- MAX_FILE files open at a single time (hfile.h) MAX_ACC access records open at a single time (hfile.h) int16 total tags (fixed) int32 max length and offset of an element in an HDF file (fixed) Vgroup Limits ------------- MAX_VFILE vset files open at a single time (hdf.h) int16 elements in a Vgroup (fixed) VGNAMELENMAX max length of a Vgroup name or class (vg.h) Vdata Limits ------------ MAX_VFILE vset files open at a single time (hdf.h) VSFIELDMAX fields in a Vdata (hdf.h) FIELDNAMELENMAX characters in a single field name (hdf.h) MAX_ORDER max field order in a Vdata (hdf.h) VSNAMELENMAX max length of a Vdata name or class (hdf.h) int16 max width in bytes of a Vdata record. (fixed) Vdatas can have a maximum field width of MAX_FIELD_SIZE bytes. (hdf.h) Raster Images ------------- int32 width or height of a raster image. (fixed) SD Limits --------- MAX_VAR_DIMS dimensions per dataset (defined in netcdf.h included by mfhdf.h) int32 maximum dimension length (fixed) MAX_NC_ATTRS attributes for a given object (defined in netcdf.h included by mfhdf.h) MAX_NC_NAME maximum length of a name of a dataset (defined in netcdf.h included by mfhdf.h) Other Convensions / Issues -------------------------- Some utility programs (e.g. ncgen) expect dataset names to be composed of only alphanumeric, '-' and '_' characters.