2 GB File and Memory Size Limitations

Introduction

The JPL Dell Xeon cluster is built on a 32-bit instruction set architecture, based on the Intel x86. This differs from the other systems at the JPL Supercomputing and Visualization Facility, which are based upon 64-bit instruction set architectures.

This document provides information on some important compiler flags that help developers address the 2 GB file size limit.

Fortran Programs

Pure Fortran programs are not affected by the 2 GB file size limit, however, mixed Fortran and C/C++ programs need to be built with the compiler flags mentioned below.

C/C++ Programs

Programmers writing in C or C++ need to take into account the following limitations imposed by the 32-bit architecture:

• Addressable space in any single malloc on a 32-bit system is limited to 2 GB — this is a limitation of the current OS.
• The 32-bit OS will limit your application to reading/writing files no larger than 2 GB — a workaround is available.

Accessing files larger than 2 GB

Use the following preprocessor flags on the Gnu (gcc) or the Intel (icc) C/C++ compiler command line to enable large file support:

-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE=1 -D_LARGEFILE_SOURCE=1

For mixed Fortran and C/C++ builds, use these flags together with any additional compiler specific flags.

Verifying the build

The following command will help ensure that a C/C++ application has been built correctly.

nm -B <executable> | grep open64

Printer Friendly Version