Environment

This section describes the routines and data structures which help set up Elemental’s programming environment: it discusses initialization of Elemental, call stack manipulation, a custom data structure for complex data, many routines for manipulating real and complex data, a litany of custom enums, and a few useful routines for simplifying index calculations.

Version and configuration information

Every Elemental driver with proper command-line argument processing will run PrintVersion if the --version argument is used. If --build is used, then all of the below information is reported.

Versioning

The following routines print the Git revision, (pre-)release version, and build type. For example:

Elemental version information:
  Git revision: 3c6fbdaad901a554fc27a83378d63dab55af0dd3
  Version:      0.85-dev
  Build type:   PureDebug

C++ API

void PrintVersion(std::ostream &os = std::cout)

C API

ElError ElPrintVersion(FILE* stream)

Python API

PrintVersion(f=ctypes.pythonapi.PyFile_AsFile(sys.stdout))

Configuration

The following routines print several relevant configuration details. For example:

Elemental configuration:
  Math libraries:               /usr/lib/liblapack.so;/usr/lib/libblas.so
  Have FLAME bidiagonal SVD:    YES
  Have OpenMP:                  YES
  Have Qt5:                     YES
  Avoiding complex MPI:         YES
  Have MPI_Reducescatter_block: YES
  Have MPI_IN_PLACE:            YES
  AllReduce ReduceScatterBlock: YES
  Use byte AllGathers:          YES

C++ API

void PrintConfig(std::ostream &os = std::cout)

C API

ElError ElPrintConfig(FILE* stream)

Python API

PrintConfig(f=ctypes.pythonapi.PyFile_AsFile(sys.stdout))

C compiler info

The following routines print various details about the C compilation process. For example:

Elemental's C compiler info:
  EL_CMAKE_C_COMPILER:    /usr/local/bin/gcc
  EL_MPI_C_COMPILER:      /home/poulson/Install/bin/mpicc
  EL_MPI_C_INCLUDE_PATH:  /home/poulson/Install/include
  EL_MPI_C_COMPILE_FLAGS:
  EL_MPI_C_LINK_FLAGS:     -Wl,-rpath  -Wl,/home/poulson/Install/lib
  EL_MPI_C_LIBRARIES:     /home/poulson/Install/lib/libmpich.so;/home/poulson/Install/lib/libopa.so;/home/poulson/Install/lib/libmpl.so;/usr/lib/i386-linux-gnu/librt.so;/usr/lib/i386-linux-gnu/libpthread.so

C++ API

void PrintCCompilerInfo(std::ostream &os = std::cout)

C API

ElError ElPrintCCompilerInfo(FILE* stream)

Python API

PrintCCompilerInfo(f=ctypes.pythonapi.PyFile_AsFile(sys.stdout))

C++ compiler info

The following routines print various details about the C++ compilation process. For example:

Elemental's C++ compiler info:
  EL_CMAKE_CXX_COMPILER:    /usr/local/bin/g++
  EL_CXX_FLAGS:             -Wall
  EL_MPI_CXX_COMPILER:      /home/poulson/Install/bin/mpicxx
  EL_MPI_CXX_INCLUDE_PATH:  /home/poulson/Install/include
  EL_MPI_CXX_COMPILE_FLAGS:
  EL_MPI_CXX_LINK_FLAGS:     -Wl,-rpath  -Wl,/home/poulson/Install/lib
  EL_MPI_CXX_LIBRARIES:     /home/poulson/Install/lib/libmpichcxx.so;/home/poulson/Install/lib/libmpich.so;/home/poulson/Install/lib/libopa.so;/home/poulson/Install/lib/libmpl.so;/usr/lib/i386-linux-gnu/librt.so;/usr/lib/i386-linux-gnu/libpthread.so

C++ API

void PrintCxxCompilerInfo(std::ostream &os = std::cout)

C API

ElError ElPrintCxxCompilerInfo(FILE* stream)

Python API

PrintCxxCompilerInfo(f=ctypes.pythonapi.PyFile_AsFile(sys.stdout))

Set up and clean up

Initialization

Initializes Elemental and (if necessary) MPI. The usage is very similar to MPI_Init, but the argc and argv can be directly passed in.

C++ API

void Initialize(int &argc, char **&argv)
#include "El.hpp"
int main( int argc, char* argv[] )
{
    El::Initialize( argc, argv );
    // ...
    El::Finalize();
    return 0;
}

C API

ElError ElInitialize(int* argc, char*** argv)

Finalization

The following routines free all resources allocated by Elemental and (if necessary) MPI.

C++ API

void Finalize()

C API

ElError ElFinalize()

Testing for initialization

Several routines are provided for querying whether or not Elemental is currently initialized.

C++ API

bool Initialized()

C API

ElError ElInitialized(bool* initialized)

Exception handling

C++ API

void ReportException(std::exception &e)

Used for handling Elemental’s various exceptions, e.g.,

#include "El.hpp"
int main( int argc, char* argv[] )
{
    El::Initialize( argc, argv );
    try {
        // ...
    } catch( std::exception& e ) { ReportException(e); }
    El::Finalize();
    return 0;
}

C API

Please see the :c:type::ElError enum.

Blocksize manipulation

Int Blocksize()

Return the currently chosen algorithmic blocksize. The optimal value depends on the problem size, algorithm, and architecture; the default value is 128.

void SetBlocksize(Int blocksize)

Change the algorithmic blocksize to the specified value.

void PushBlocksizeStack(Int blocksize)

It is frequently useful to temporarily change the algorithmic blocksize, so rather than having to manually store and reset the current state, one can simply push a new value onto a stack (and later pop the stack to reset the value).

void PopBlocksizeStack()

Pops the stack of blocksizes. See above.

Int DefaultBlockHeight()
Int DefaultBlockWidth()

Returns the default block height (width) for BlockDistMatrix<T,U,V>.

void SetDefaultBlockHeight(Int mb)
void SetDefaultBlockWidth(Int nb)

Change the default block height (width) for BlockDistMatrix<T,U,V>.

Default process grid

Grid &DefaultGrid()

Return a process grid built over mpi::COMM_WORLD. This is typically used as a means of allowing instances of the DistMatrix<T,MC,MR> class to be constructed without having to manually specify a process grid, e.g.,

// Build a 10 x 10 distributed matrix over mpi::COMM_WORLD
El::DistMatrix<T,MC,MR> A( 10, 10 );

Call stack manipulation

Note

The following call stack manipulation routines are only available in non-release builds (i.e., PureDebug and HybridDebug) and are meant to allow for the call stack to be printed (via DumpCallStack()) when an exception is caught.

void PushCallStack(std::string s)

Push the given routine name onto the call stack.

void PopCallStack()

Remove the routine name at the top of the call stack.

void DumpCallStack()

Print (and empty) the contents of the call stack.

Error handling

C++ API

type SingularMatrixException

An extension of std::runtime_error which is meant to be thrown when a singular matrix is unexpectedly encountered.

SingularMatrixException(const char *msg = "Matrix was singular")

Builds an instance of the exception which allows one to optionally specify the error message.

throw El::SingularMatrixException();
type NonHPDMatrixException

An extension of std::runtime_error which is meant to be thrown when a non positive-definite Hermitian matrix is unexpectedly encountered (e.g., during Cholesky factorization).

NonHPDMatrixException(const char *msg = "Matrix was not HPD")

Builds an instance of the exception which allows one to optionally specify the error message.

throw El::NonHPDMatrixException();
type NonHPSDMatrixException

An extension of std::runtime_error which is meant to be thrown when a non positive semi-definite Hermitian matrix is unexpectedly encountered (e.g., during computation of the square root of a Hermitian matrix).

NonHPSDMatrixException(const char *msg = "Matrix was not HPSD")

Builds an instance of the exception which allows one to optionally specify the error message.

throw El::NonHPSDMatrixException();

C API

ElError

An enum which can be set to one of the following values:

  • EL_SUCCESS
  • EL_ALLOC_ERROR
  • EL_OUT_OF_BOUNDS_ERROR
  • EL_ARG_ERROR
  • EL_LOGIC_ERROR
  • EL_RUNTIME_ERROR
  • EL_ERROR
const char* ElErrorString(ElError error)

Convert the error code into a (hopefully) descriptive message

Complex data

C++ API

template<>
type Complex<Real>

Currently a typedef of std::complex<Real>

template<>
type Base<F>

The underlying real datatype of the (potentially complex) datatype F. For example, Base<Complex<double>> and Base<double> are both equivalent to double. This is often extremely useful in implementing routines which are templated over real and complex datatypes but still make use of real datatypes.

std::ostream &operator<<(std::ostream &os, Complex<Real> alpha)

Pretty prints alpha in the form a+bi.

type scomplex

typedef Complex<float> scomplex;

type dcomplex

typedef Complex<double> dcomplex;

C API

complex_float

A struct equivalent to struct complex_float { float real, imag; } which is meant to be binary compatible with std::complex<float>.

complex_double

A struct equivalent to struct complex_double { double real, imag; } which is meant to be binary compatible with std::complex<double>.

Other typedefs and enums

C++ API

type byte

typedef unsigned char byte;

type Int

Typically a typedef to int, but if the experimental EL_USE_64BIT_INTS compilation mode is enabled, it becomes a typedef to long long int, which is guaranteed to be at least 64-bit by C++11

type Unsigned

Typically a typedef to unsigned, but if the experimental EL_USE_64BIT_INTS compilation mode is enabled, it becomes a typedef to long long unsigned, which is guaranteed to be at least 64-bit by C++11

type Conjugation

An enum which can be set to either CONJUGATED or UNCONJUGATED.

type Dist

An enum for specifying the distribution of a row or column of a distributed matrix:

  • MC: Column of a standard matrix distribution
  • MD: Diagonal of a standard matrix distribution
  • MR: Row of a standard matrix distribution
  • VC: Column-major vector distribution
  • VR: Row-major vector distribution
  • STAR: Redundantly stored on every process
  • CIRC: Stored on a single process
type ForwardOrBackward

An enum for specifying FORWARD or BACKWARD.

type GridOrder

An enum for specifying either a ROW_MAJOR or COLUMN_MAJOR ordering; it is used to decide how to construct process grids and is also useful for tuning one of the algorithms in HermitianTridiag() which requires building a smaller square process grid from a rectangular process grid, as the ordering of the processes can greatly impact performance. See SetHermitianTridiagGridOrder().

type LeftOrRight

An enum for specifying LEFT or RIGHT.

type SortType

An enum for specifying a sorting strategy:

  • UNSORTED: do not sort
  • DESCENDING: smallest values first
  • ASCENDING: largest values first
type NormType

An enum that can be set to either

  • ONE_NORM:

    \[\|A\|_1 = \max_{\|x\|_1=1} \|Ax\|_1 = \max_j \sum_{i=0}^{m-1} |\alpha_{i,j}|\]
  • INFINITY_NORM:

    \[\|A\|_{\infty} = \max_{\|x\|_{\infty}=1} \|Ax\|_{\infty} = \max_i \sum_{j=0}^{n-1} |\alpha_{i,j}|\]
  • ENTRYWISE_ONE_NORM:

    \[\|\text{vec}(A)\|_1 = \sum_{i,j} |\alpha_{i,j}|\]
  • MAX_NORM:

    \[\|A\|_{\mbox{max}} = \max_{i,j} |\alpha_{i,j}|\]
  • NUCLEAR_NORM:

    \[\|A\|_* = \sum_{i=0}^{\min(m,n)} \sigma_i(A)\]
  • FROBENIUS_NORM:

    \[\|A\|_F = \sqrt{\sum_{i=0}^{m-1} \sum_{j=0}^{n-1} |\alpha_{i,j}|^2} = \sum_{i=0}^{\min(m,n)} \sigma_i(A)^2\]
  • TWO_NORM:

    \[\|A\|_2 = \max_i \sigma_i(A)\]
type Orientation

An enum for specifying whether a matrix, say \(A\), should be implicitly treated as \(A\) (NORMAL), \(A^H\) (ADJOINT), or \(A^T\) (TRANSPOSE).

type UnitOrNonUnit

An enum for specifying either UNIT or NON_UNIT; typically used for stating whether or not a triangular matrix’s diagonal is explicitly stored (NON_UNIT) or is implicitly unit-diagonal (UNIT).

type UpperOrLower

An enum for specifying LOWER or UPPER (triangular).

type VerticalOrHorizontal

An enum for specifying VERTICAL or HORIZONTAL.

C API

The following are analogues to the above C++ definitions.

ElByte

A typedef to unsigned char

ElInt

Typically a typedef to int, but if the experimental EL_USE_64BIT_INTS compilation mode is enabled, it becomes a typedef to long long int

ElUnsigned

Typically a typedef to unsigned, but if the experimental EL_USE_64BIT_INTS compilation mode is enabled, it becomes a typedef to long long unsigned

ElConjugation

An enum equal to either EL_CONJUGATED or EL_UNCONJUGATED

ElDist

An enum equal to one of:

  • EL_MC
  • EL_MD
  • EL_MR
  • EL_STAR
  • EL_VC
  • EL_VR
  • EL_CIRC
ElForwardOrBackward

An enum equal to either EL_FORWARD or EL_BACKWARD

ElGridOrder

An enum equal to either EL_ROW_MAJOR or EL_COLUMN_MAJOR

ElLeftOrRight

An enum equal to either EL_LEFT or EL_RIGHT

ElSortType

An enum equal to EL_UNSORTED, EL_ASCENDING, or EL_DESCENDING

ElNormType

An enum equal to one of:

  • EL_ONE_NORM
  • EL_INFINITY_NORM
  • EL_ENTRYWISE_ONE_NORM
  • EL_MAX_NORM
  • EL_NUCLEAR_NORM
  • EL_FROBENIUS_NORM
  • EL_TWO_NORM
ElOrientation

An enum equal to EL_NORMAL, EL_TRANSPOSE, or EL_ADJOINT

ElUnitOrNonUnit

An enum equal to either EL_UNIT or EL_NON_UNIT

ElUpperOrLower

An enum equal to either EL_UPPER or EL_LOWER

ElVerticalOrHorizontal

An enum equal to either EL_VERTICAL or EL_HORIZONTAL

Indexing utilities

Int Shift(Int rank, Int firstRank, Int numProcs)

Given a element-wise cyclic distribution over numProcs processes, where the first entry is owned by the process with rank firstRank, this routine returns the first entry owned by the process with rank rank.

Int Length(Int n, Int shift, Int numProcs)

Given a vector with \(n\) entries distributed over numProcs processes with shift as defined above, this routine returns the number of entries of the vector which are owned by this process.

Int Length(Int n, Int rank, Int firstRank, Int numProcs)

Given a vector with \(n\) entries distributed over numProcs processes, with the first entry owned by process firstRank, this routine returns the number of entries locally owned by the process with rank rank.

Int MaxLength(Int n, Int numProcs)

The maximum result of Length() with the given parameters. This is useful for padding collective communication routines which are almost regular.

Int Mod(Int a, Int b)

An extension of C++’s % operator which handles cases where a is negative and still returns a result in \([0,b)\).

Int GCD(Int a, Int b)

Return the greatest common denominator of the integers a and b.

Unsigned Log2(Unsigned n)

Return the base-two logarithm of a positive integer.

bool PowerOfTwo(Unsigned n)

Return whether or not a positive integer is an integer power of two.