.. _using-llvmorg-clang-compilers-with-libstdc-and-abi-compatibility:

Using LLVM.org Clang compilers with libstdc++ and ABI compatibility
===================================================================

.. toctree::
   :maxdepth: 1

.. _table-of-contents:

Table of Contents
-----------------

1.  :ref:`About <about>`
2.  :ref:`Clang compilers and libstdc++ <clang-compilers-and-libstdc>`

    -  :ref:`Discoverer LLVM compiler build <discoverer-llvm-compiler-build>`
    -  :ref:`Default behaviour on Linux <default-behaviour-on-linux>`
    -  :ref:`When clang++ can use libstdc++ <when-clang-can-use-libstdc>`
    -  :ref:`How clang++ finds libstdc++ <how-clang-finds-libstdc>`
    -  :ref:`Ensuring libstdc++ is used <ensuring-libstdc-is-used>`
    -  :ref:`Compatibility with GCC-compiled code <compatibility-with-gcc-compiled-code>`
    -  :ref:`When issues may arise <when-issues-may-arise>`
    -  :ref:`Forcing libc++ usage <forcing-libc-usage>`

3.  :ref:`CUDA compilation with clang++ and libstdc++ <cuda-compilation-with-clang-and-libstdc>`

    -  :ref:`Direct CUDA compilation with clang++ <direct-cuda-compilation-with-clang>`
    -  :ref:`Using nvcc with clang++ as host compiler <using-nvcc-with-clang-as-host-compiler>`
    -  :ref:`Pros of using clang++ with CUDA <pros-of-using-clang-with-cuda>`
    -  :ref:`Cons and limitations <cons-and-limitations>`
    -  :ref:`When clang++ with CUDA works <when-clang-with-cuda-works>`
    -  :ref:`When clang++ with CUDA fails <when-clang-with-cuda-fails>`
    -  :ref:`Practical experience: Ginkgo build <practical-experience-ginkgo-build>`
    -  :ref:`Recommendations <recommendations>`

4.  :ref:`Static linking of C++ standard libraries <static-linking-of-c-standard-libraries>`

    -  :ref:`What static linking means <what-static-linking-means>`
    -  :ref:`Example: GoogleTest with statically linked libc++ <example-googletest-with-statically-linked-libc>`

5.  :ref:`ABI compatibility at API boundaries <abi-compatibility-at-api-boundaries>`

    -  :ref:`The critical distinction <the-critical-distinction>`
    -  :ref:`Why API boundaries matter <why-api-boundaries-matter>`
    -  :ref:`Type identity and name mangling <type-identity-and-name-mangling>`
    -  :ref:`Example: GoogleTest’s GetString() function <example-googletests-getstring-function>`
    -  :ref:`Symbol resolution and linker errors <symbol-resolution-and-linker-errors>`

6.  :ref:`When static linking works across different standard libraries <when-static-linking-works-across-different-standard-libraries>`

    -  :ref:`No standard library types in public API <1-no-standard-library-types-in-public-api>`
    -  :ref:`Internal use only <2-internal-use-only>`
    -  :ref:`Template-based APIs with header-only code <3-template-based-apis-with-header-only-code>`
    -  :ref:`Opaque pointer pattern <4-opaque-pointer-pattern>`

7.  :ref:`When static linking does not work across different standard libraries <when-static-linking-does-not-work-across-different-standard-libraries>`

    -  :ref:`Standard library types in function return values <1-standard-library-types-in-function-return-values>`
    -  :ref:`Standard library types in function parameters <2-standard-library-types-in-function-parameters>`
    -  :ref:`Standard library types in public class members <3-standard-library-types-in-public-class-members>`
    -  :ref:`Exception types crossing boundaries <4-exception-types-crossing-boundaries>`
    -  :ref:`Virtual function tables with standard library types <5-virtual-function-tables-with-standard-library-types>`
    -  :ref:`Template specializations with standard library types <6-template-specializations-with-standard-library-types>`
    -  :ref:`Comparison table <comparison-table>`

8.  :ref:`Simple code examples <simple-code-examples>`

    -  :ref:`Example A: Library with C API (works) <example-a-library-with-c-api-works>`
    -  :ref:`Example B: Library with std::string return (does not work) <example-b-library-with-stdstring-return-does-not-work>`
    -  :ref:`Example C: Library with opaque handle (works) <example-c-library-with-opaque-handle-works>`
    -  :ref:`Example D: Library with public std::string member (does not work) <example-d-library-with-public-stdstring-member-does-not-work>`
    -  :ref:`Example E: Header-only template (works) <example-e-header-only-template-works>`

9.  :ref:`Practical examples <practical-examples>`

    -  :ref:`Example 1: GoogleTest - does not work <example-1-googletest---does-not-work>`
    -  :ref:`Example 2: C API wrapper - works <example-2-c-api-wrapper---works>`
    -  :ref:`Example 3: Opaque handle library - works <example-3-opaque-handle-library---works>`
    -  :ref:`Example 4: Container library with public members - does not work <example-4-container-library-with-public-members---does-not-work>`
    -  :ref:`Example 5: Header-only template library - works <example-5-header-only-template-library---works>`

10. :ref:`Quick reference: working vs non-working patterns <quick-reference-working-vs-non-working-patterns>`
11. :ref:`Verification methods <verification-methods>`

    -  :ref:`Check runtime dependencies <check-runtime-dependencies>`
    -  :ref:`Check symbol namespaces <check-symbol-namespaces>`
    -  :ref:`Check API function signatures <check-api-function-signatures>`
    -  :ref:`Check dynamic section <check-dynamic-section>`

12. :ref:`Practical guidelines for clang++ with libstdc++ <practical-guidelines-for-clang-with-libstdc>`

    -  :ref:`Recommended approach <recommended-approach>`
    -  :ref:`Build system configuration <build-system-configuration>`
    -  :ref:`Troubleshooting <troubleshooting>`

13. :ref:`Conclusion <conclusion>`

.. _about:

About
-----

This document explains how to use ``clang``/``clang++`` compilers with libstdc++ (GCC’s C++ standard library), when this is possible, and the relationship between static linking of C++ standard library implementations and ABI (Application Binary Interface) compatibility requirements when libraries expose standard library types in their public APIs.

.. _clang-compilers-and-libstdc:

Clang compilers and libstdc++
-----------------------------

.. _discoverer-llvm-compiler-build:

Discoverer LLVM compiler build
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. important:: At Discoverer, the LLVM compiler collection (including ``clang``/``clang++``) is compiled from source rather than using pre-built binaries. The build configuration differs between the CPU cluster and GPU cluster environments.

Build characteristics:

1. **CUDA-aware compilation (GPU cluster only)** - On Discoverer+ GPU cluster, the LLVM toolchain is compiled with CUDA support enabled, making ``clang++`` aware of CUDA installation paths, headers, and libraries. On the CPU cluster, CUDA awareness is not included in the LLVM build.
2. **Custom configuration** - Optimised for the specific computing environment (CPU or GPU cluster)
3. **Version control** - Specific LLVM version (21.1.0) used in production builds

Cluster-specific configurations:

-  **Discoverer CPU cluster**: LLVM build without CUDA support. ``clang++`` can still compile CUDA code if CUDA toolkit is available, but CUDA paths are not embedded in the compiler.
-  **Discoverer+ GPU cluster**: LLVM build with CUDA support enabled. ``clang++`` has CUDA awareness built-in, enhancing integration with CUDA toolchain.

What CUDA-aware means:

When the LLVM toolchain (both ``clang`` and ``clang++``) is compiled with CUDA support enabled, the compilers have built-in knowledge and capabilities for CUDA compilation.

CUDA awareness in ``clang`` vs ``clang++``:

Both ``clang`` (C compiler) and ``clang++`` (C++ compiler) have CUDA awareness when the LLVM build includes CUDA support. However, there are practical differences:

1. **CUDA as C++ extension** - CUDA is primarily a C++ language extension. Most CUDA code uses C++ features:

   -  Classes, templates, namespaces
   -  C++ standard library in host code
   -  C++-style kernel launches

2. **C support** - CUDA also supports C, but with limitations:

   -  C kernels are supported but less common
   -  C++ features are not available in C mode
   -  Most CUDA libraries and examples use C++

3. **Practical usage** - ``clang++`` is typically used for CUDA compilation:

   .. code:: bash

      clang++ -x cuda source.cu -o program  # Common
      clang -x cuda source.cu -o program     # Less common, C-only features

4. **Feature parity** - Both compilers support the same CUDA compilation flags and options when CUDA-aware, but ``clang++`` is required for C++ CUDA code.

The extent of CUDA awareness includes

Compilation capabilities:

1. **Direct CUDA compilation** - Can compile ``.cu`` files directly without using ``nvcc`` wrapper

   .. code:: bash

      clang++ -x cuda source.cu -o program

2. **CUDA language support** - Understands CUDA-specific syntax:

   -  ``__global__``, ``__device__``, ``__host__`` function qualifiers
   -  ``<<<...>>>`` kernel launch syntax
   -  CUDA built-in variables (``threadIdx``, ``blockIdx``, ``gridDim``, ``blockDim``)
   -  CUDA memory management functions (``cudaMalloc``, ``cudaFree``, etc.)

3. **Dual compilation** - Compiles both host code (CPU) and device code (GPU) in a single pass

   -  Host code compiled for the target CPU architecture
   -  Device code compiled to PTX (Parallel Thread Execution) or device-specific assembly

4. **CUDA header awareness** - Knows where to find CUDA runtime headers:

   -  ``cuda_runtime.h``
   -  ``cuda.h``
   -  Device-specific headers

Path detection and configuration:

1. **Automatic CUDA path detection** - Can find CUDA installations in standard locations

2. **Automatic CUDA path detection** - Can find CUDA installations in standard locations

3. **Embedded search paths** - CUDA installation paths may be embedded during LLVM build

4. **Runtime library linking** - Knows how to link against CUDA runtime libraries (``libcudart.so``)

Limitations of CUDA awareness:

1. **CUDA version support** - May not support the latest CUDA versions immediately

   -  Example: CUDA 12.8 may show warnings about partial support
   -  Newer CUDA features may not be available until ``clang++`` is updated

2. **Not a complete** ``nvcc`` **replacement** - Some ``nvcc``-specific features may not be supported:

   -  Certain compiler-specific pragmas
   -  Some advanced CUDA features
   -  Compatibility modes specific to nvcc

3. **Standard library restrictions** - CUDA device code has limitations:

   -  Cannot use libc++ on x86 systems (CUDA headers reject it)
   -  Device code typically uses a restricted subset of C++ standard library
   -  Host code can use full standard library, but device code cannot

4. **Compiler crashes** - In some cases, ``clang++`` may crash when compiling complex CUDA code:

   -  Very large CUDA files
   -  Complex template instantiations
   -  Certain CUDA language constructs

What CUDA awareness does not provide

CUDA awareness does not affect the following:

1. **Does not change standard library default** - Still uses libstdc++ by default on Linux
2. **Does not guarantee compatibility** - CUDA code compiled with ``clang++`` may not be compatible with ``nvcc``-compiled code in all cases
3. **Does not eliminate need for CUDA toolkit** - Still requires CUDA toolkit installation
4. **Does not provide CUDA runtime** - Runtime libraries must be available separately

*Verification that* ``clang++`` *is CUDA-aware*:

.. code:: bash

   clang++ -x cuda --cuda-path=/usr/local/cuda --help 2>&1 | grep -i cuda

Shows CUDA-specific options:

::

   --cuda-compile-host-device
   --cuda-device-only
   --cuda-host-only
   --cuda-path=<value>
   --cuda-gpu-arch=<value>
   --cuda-feature=<value>
   --cuda-include-ptx=<value>

Testing CUDA awareness:

.. code:: bash

   # Simple CUDA test
   cat > test_cuda.cu << 'EOF'
   __global__ void test() {}
   int main() { test<<<1,1>>>(); return 0; }
   EOF

   # Try to compile with clang++
   clang++ -x cuda --cuda-path=/usr/local/cuda test_cuda.cu -o test_cuda -lcudart

   # If compilation succeeds, clang++ is CUDA-aware
   # If it fails with "unknown argument: -x cuda", clang++ is not CUDA-aware

Impact on standard library usage:

Whether the LLVM toolchain is CUDA-aware or not, it still uses libstdc++ by default on Linux systems. CUDA awareness affects CUDA compilation capabilities (automatic path detection, better integration, direct CUDA compilation) but does not change the default standard library selection. Both CPU and GPU cluster builds use libstdc++ by default.

CUDA awareness vs standard library:

-  **CUDA awareness**: Affects how the compiler handles CUDA language features and finds CUDA headers/libraries
-  **Standard library selection**: Determines which C++ standard library (libc++ or libstdc++) is used for host code
-  These are independent: A CUDA-aware ``clang++`` can use either libc++ or libstdc++ for host code, though libstdc++ is the default and recommended for CUDA compatibility

Module system integration:

The custom LLVM build is provided through the environment module system.

.. code:: bash

   module load llvm/21

This module:

-  Sets up compiler paths (``clang``, ``clang++``, ``flang``)
-  Sets up linker and tool paths (``lld``, ``llvm-ar``, ``llvm-ranlib``, ``llvm-nm``, ``llvm-strip``)
-  Configures library paths for LLVM tools
-  Does not force libc++ usage (libstdc++ remains the default)


.. _default-behaviour-on-linux:

Default behaviour on Linux
~~~~~~~~~~~~~~~~~~~~~~~~~~

On Linux systems, ``clang++`` uses libstdc++ by default, not libc++. This is the standard behaviour and requires no special configuration.

*Verification*:

.. code:: bash

   clang++ -x c++ -E -v /dev/null 2>&1 | grep "include.*c++"

Output shows:

::

   /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11
   /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/x86_64-redhat-linux

These paths point to libstdc++ headers, not libc++ headers.

Preprocessor defines:

.. code:: bash

   clang++ -x c++ -E -dM /dev/null 2>&1 | grep "GLIBCXX\|LIBCPP"

Shows ``_GLIBCXX_USE_CXX11_ABI`` and ``__GLIBCXX__`` defines, confirming libstdc++ usage.

.. _when-clang-can-use-libstdc:

When ``Clang++`` Can Use libstdc++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``Clang++`` can use libstdc++ in all standard scenarios:

1. **Default compilation** - No special flags needed
2. **Standard C++ code** - All standard library features work normally
3. **CUDA compilation** - ``Clang++`` can compile CUDA code using libstdc++
4. **Mixed toolchains** - Can be used alongside GCC-compiled code if ABI is consistent

.. _how-clang-finds-libstdc:

How ``clang++`` finds libstdc++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``Clang++`` automatically discovers libstdc++ through:

1. **GCC installation paths** - Searches for GCC’s include directories
2. **System default paths** - Standard locations like ``/usr/include/c++/``
3. **Environment variables** - Can be influenced by ``C_INCLUDE_PATH``, ``CPLUS_INCLUDE_PATH``

Default search order:

::

   /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11
   /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/x86_64-redhat-linux
   /usr/lib/gcc/x86_64-redhat-linux/11/../../../../include/c++/11/backward

.. _ensuring-libstdc-is-used:

Ensuring libstdc++ is used
~~~~~~~~~~~~~~~~~~~~~~~~~~

To explicitly ensure ``clang++`` uses libstdc++:

1. **Do not specify** ``-stdlib=libc++`` - This would force libc++ usage
2. **Verify with preprocessor** - Check for ``__GLIBCXX__`` defines
3. **Check linker output** - Verify ``-lstdc++`` is used, not ``-lc++``

*Verification command*:

.. code:: bash

   clang++ test.cpp -v 2>&1 | grep "libstdc++\|libc++"

Should show ``-lstdc++`` in the linker command, not ``-lc++``.

.. _compatibility-with-gcc-compiled-code:

Compatibility with GCC-compiled code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When ``clang++`` uses libstdc++, it is compatible with GCC-compiled code if:

1. **Same libstdc++ version** - Both use the same version of libstdc++
2. **Same ABI setting** - Both use the same ``_GLIBCXX_USE_CXX11_ABI`` value
3. **Same C++ standard** - Both compile with compatible C++ standard versions

*Example - compatible compilation*:

.. code:: bash

   # Compile with GCC

   # Compile with GCC
   g++ -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=1 library.cpp -shared -o libexample.so

   # Compile with clang++ (uses libstdc++ by default)
   clang++ -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=1 caller.cpp -L. -lexample -o caller

   # Works: both use libstdc++ with C++11 ABI

.. _when-issues-may-arise:

When issues may arise
~~~~~~~~~~~~~~~~~~~~~

``Clang++`` with libstdc++ may have issues in these scenarios:

1. **Template instantiation differences** - ``Clang++`` and GCC may instantiate templates differently
2. **Compiler-specific extensions** - Code using GCC-specific features may not compile with ``clang++``
3. **Warning/error differences** - ``Clang++`` may catch issues GCC misses, or vice versa
4. **Optimisation differences** - Different optimisation strategies may affect performance

*Example - template instantiation*:

.. code:: cpp

   // May behave differently between GCC and clang++
   template<typename T>
   void process(T value) {
       // Template instantiation may differ
   }

.. _forcing-libc-usage:

Forcing libc++ Usage
~~~~~~~~~~~~~~~~~~~~

If you need to use libc++ instead of libstdc++:

.. code:: bash

   clang++ -stdlib=libc++ -lc++ -lc++abi source.cpp -o program

This forces ``clang++`` to:

-  Use libc++ headers instead of libstdc++ headers
-  Link against libc++ and libc++abi instead of libstdc++
-  Use ``std::__1::`` namespace instead of ``std::__cxx11::``

When to use libc++:

-  When you need libc++ specific features
-  When building a fully LLVM-based toolchain
-  When targeting platforms where libc++ is standard (e.g., macOS, some embedded systems)

When not to use libc++:

-  When compatibility with GCC-compiled code is required
-  When using CUDA (CUDA toolchain typically expects libstdc++)
-  When system libraries are compiled with libstdc++

.. _cuda-compilation-with-clang-and-libstdc:

CUDA compilation with ``clang++`` and libstdc++
-----------------------------------------------

.. _direct-cuda-compilation-with-clang:

Direct CUDA compilation with ``clang++``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``Clang++`` can compile ``.cu`` files directly:

.. code:: bash

   # Compile CUDA code directly with clang++
   clang++ -x cuda --cuda-path=/usr/local/cuda source.cu -o program

   # With architecture specification
   clang++ -x cuda --cuda-path=/usr/local/cuda \
       --cuda-gpu-arch=sm_75 source.cu -o program

*Verification that* ``clang++`` *is CUDA-aware*:

.. code:: bash

   clang++ -x cuda --cuda-path=/usr/local/cuda --help 2>&1 | grep -i cuda

Shows CUDA-specific options like:

::

   --cuda-compile-host-device
   --cuda-device-only
   --cuda-host-only
   --cuda-path=<value>

.. _using-nvcc-with-clang-as-host-compiler:

Using ``nvcc`` with ``clang++`` as host compiler
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Alternatively, use ``nvcc`` with ``clang++`` as the host compiler:

.. code:: bash

   export CUDAHOSTCXX=clang++
   nvcc --allow-unsupported-compiler --compiler-options=-std=c++17 source.cu -o program

Or in CMake:

.. code:: cmake

   set(CMAKE_CUDA_HOST_COMPILER clang++)
   set(CMAKE_CUDA_FLAGS "--allow-unsupported-compiler")

When ``--allow-unsupported-compiler`` is needed:

``nvcc`` maintains a list of officially supported host compilers (typically specific GCC versions). When you use ``clang++`` as the host compiler, ``nvcc`` may not recognise it as a supported compiler and will refuse to compile, showing an error like:

::

   nvcc fatal   : The version ('clang++') of the host compiler ('clang++') is not supported

The ``--allow-unsupported-compiler`` flag tells ``nvcc`` to proceed anyway, even though ``clang++`` is not officially supported. This is necessary when:

1. **Using** ``clang++`` **with** ``nvcc`` - ``clang++`` is not in ``nvcc``\ ’s official supported compiler list
2. **Using newer compiler versions** - Even if a compiler is supported, newer versions may not be recognised
3. **Custom compiler builds** - Custom-built compilers (like Discoverer’s LLVM build) may not be recognised

.. note:: This flag is specific to ``nvcc``, not ``clang++``. It’s an ``nvcc`` option that allows ``nvcc`` to use an unsupported host compiler. When using ``clang++`` directly for CUDA compilation (without ``nvcc``), this flag is not needed or applicable.

.. _pros-of-using-clang-with-cuda:

Pros of using ``clang++`` with CUDA
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. **Unified compiler** - Same compiler for host and device code
2. **Better error messages** - ``clang++`` often provides clearer diagnostics
3. **Faster compilation** - ``clang++`` may compile faster than ``nvcc`` in some cases
4. **CUDA-aware** - Native CUDA support without wrapper
5. **libstdc++ compatibility** - Works with libstdc++ by default, ensuring ABI consistency

.. _cons-and-limitations:

Cons and limitations
~~~~~~~~~~~~~~~~~~~~

1. **CUDA version support** - ``clang++`` may not support the latest CUDA versions immediately

   -  Example: CUDA 12.8 may show warnings: ``warning: CUDA version 12.8 is only partially supported``
   -  Can be suppressed with: ``-Wno-unknown-cuda-version``

2. **Compiler crashes** - In some cases, ``clang++`` may crash when compiling CUDA code

   -  Example: ``clang++: error: unable to execute command: Segmentation fault (core dumped)``
   -  This can occur with complex template code or certain CUDA features
   -  Workaround: Use ``nvcc`` instead, or simplify the code

3. **libc++ restrictions** - CUDA headers on x86 systems do not support libc++

   -  Error: ``libc++ is not supported on x86 system``
   -  Solution: Use libstdc++ (default) or avoid ``-stdlib=libc++`` for CUDA compilation

4. **ABI consistency** - Must ensure host and device code use the same standard library

   -  If host code uses libc++ and CUDA code uses libstdc++, ABI mismatches occur
   -  Solution: Use libstdc++ for both (``clang++`` default on Linux)

.. _when-clang-with-cuda-works:

When ``clang++`` with CUDA works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``clang++`` with CUDA works well when:

1. **Using libstdc++** - Default behaviour on Linux, ensures ABI consistency
2. **Supported CUDA versions** - Using CUDA versions that ``clang++`` fully supports
3. **Standard CUDA code** - Not using cutting-edge CUDA features
4. **Simple to moderate complexity** - Avoiding extremely complex template code

*Example - working configuration*:

.. code:: bash

   # Host code with libstdc++ (default)

   # Host code with libstdc++ (default)
   clang++ -std=c++17 host.cpp -c -o host.o

   # CUDA code with libstdc++ (default)
   clang++ -x cuda --cuda-path=/usr/local/cuda \
       -std=c++17 device.cu -c -o device.o

   # Link together
   clang++ host.o device.o -lcudart -o program

.. _when-clang-with-cuda-fails:

When ``clang++`` with CUDA fails
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``clang++`` with CUDA may fail when:

1. **Using libc++** - CUDA headers reject libc++ on x86

   .. code:: bash

      clang++ -x cuda -stdlib=libc++ source.cu
      # Error: libc++ is not supported on x86 system

2. **Compiler crashes** - Segmentation faults during compilation

   .. code:: bash

      clang++ -x cuda source.cu
      # Error: Segmentation fault (core dumped)

   This may occur with:

   -  Complex template instantiations
   -  Large CUDA files
   -  Certain CUDA language features

3. **Unsupported CUDA features** - Using features not yet supported by ``clang++``

   -  May require using ``nvcc`` instead
   -  Check ``clang++`` CUDA documentation for supported features

4. **ABI mismatches** - Mixing libc++ (host) with libstdc++ (CUDA)

   .. code:: bash

      # Host code with libc++
      clang++ -stdlib=libc++ host.cpp -c

      # CUDA code with libstdc++ (default)
      clang++ -x cuda device.cu -c

      # Linking fails due to ABI mismatch

.. _practical-experience-ginkgo-build:

Practical experience: Ginkgo build
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

During the Ginkgo 1.11.0 build with CUDA support, several approaches were attempted.

Initial attempt using ``clang++`` as CUDA compiler:

The first approach attempted to use ``clang++`` directly as the CUDA compiler:

.. code:: bash

   export CUDAHOSTCXX=clang++
   cmake -DCMAKE_CUDA_COMPILER=clang++ \
         -DCMAKE_CUDA_FLAGS="-Xcompiler=-stdlib=libc++" \
         -DGINKGO_BUILD_CUDA=ON

This approach encountered several issues: 1. CUDA headers rejected libc++: ``libc++ is not supported on x86 system`` 2. After removing libc++ flag, compiler crash: ``Segmentation fault (core dumped)`` 3. ABI mismatch when mixing libc++ (host) with libstdc++ (CUDA device code)

Second attempt using ``nvcc`` with ``clang++`` as host compiler:

The next approach used ``nvcc`` with ``clang++`` as the host compiler:

.. code:: bash

   export CUDAHOSTCXX=clang++
   # Use nvcc (not clang++ directly) as CUDA compiler
   # --allow-unsupported-compiler is required because nvcc doesn't officially support clang++
   cmake -DCMAKE_CUDA_HOST_COMPILER=clang++ \
         -DCMAKE_CUDA_FLAGS="--allow-unsupported-compiler" \
         -DGINKGO_BUILD_CUDA=ON

The ``--allow-unsupported-compiler`` flag is needed because ``nvcc`` checks the host compiler version against its list of supported compilers. Since ``clang++`` is not officially supported by ``nvcc``, this flag is required to bypass ``nvcc``\ ’s compiler version check.

Final solution using NVIDIA HPC SDK (``nvc++``):

The final successful approach used NVIDIA HPC SDK’s ``nvc++`` compiler:

.. code:: bash

   # Use nvc++ which natively supports CUDA and uses libstdc++
   export CC=nvc
   export CXX=nvc++
   cmake -DCMAKE_C_COMPILER=mpicc \
         -DCMAKE_CXX_COMPILER=mpic++ \
         -DGINKGO_BUILD_CUDA=ON

This approach provides several advantages:

-  Uses libstdc++ throughout (host and device)
-  Avoids ABI mismatches
-  Provides native CUDA support without compiler crashes
-  Ensures consistent toolchain


.. _recommendations:

Recommendations
~~~~~~~~~~~~~~~

For CUDA development with ``clang++``:

1. **Use libstdc++** - Default on Linux, compatible with CUDA
2. **Test compilation early** - Verify ``clang++`` can compile your CUDA code
3. **Have** ``nvcc`` **as fallback** - If ``clang++`` crashes, use ``nvcc`` with ``clang++`` as host compiler
4. **Consider** ``nvc++`` - For complex projects, NVIDIA HPC SDK’s ``nvc++`` provides robust CUDA support
5. **Monitor CUDA version support** - Check ``clang++`` release notes for CUDA version compatibility

*Example CMake configuration*:

.. code:: cmake

   # Try clang++ as CUDA compiler first
   find_program(CLANG_CUDA_COMPILER
       NAMES clang++
       PATHS /usr/bin /usr/local/bin
   )

   if(CLANG_CUDA_COMPILER)
       # Test if it can compile CUDA
       try_compile(CAN_USE_CLANG_CUDA
           ${CMAKE_BINARY_DIR}/test_cuda
           ${CMAKE_SOURCE_DIR}/test_cuda_simple.cu
           CMAKE_FLAGS "-DCMAKE_CUDA_COMPILER=${CLANG_CUDA_COMPILER}"
       )
       
       if(CAN_USE_CLANG_CUDA)
           set(CMAKE_CUDA_COMPILER ${CLANG_CUDA_COMPILER})
           set(CMAKE_CUDA_FLAGS "--cuda-path=${CUDA_PATH}")
       else()
           # Fall back to nvcc
           set(CMAKE_CUDA_COMPILER nvcc)
           set(CMAKE_CUDA_HOST_COMPILER clang++)
       endif()
   else()
       # Use nvcc
       set(CMAKE_CUDA_COMPILER nvcc)
       set(CMAKE_CUDA_HOST_COMPILER clang++)
   endif()

.. _static-linking-of-c-standard-libraries:

Static linking of C++ standard libraries
----------------------------------------

.. _what-static-linking-means:

What static linking means
~~~~~~~~~~~~~~~~~~~~~~~~~

When a library is compiled with static linking of the C++ standard library:

1. The standard library code (libc++ or libstdc++) is embedded directly into the library binary
2. No runtime dependency on external standard library shared objects is required
3. The library becomes self-contained for standard library functionality

.. _example-googletest-with-statically-linked-libc:

Example: GoogleTest with statically linked libc++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

GoogleTest can be compiled with static linking of libc++ using the following flags:

.. code:: bash

   -stdlib=libc++
   -nostdlib++
   ${LLVM_LIB_PATH}/libc++.a ${LLVM_LIB_PATH}/libc++abi.a

The resulting shared library (``libgtest.so``) will:

-  Contain all libc++ code embedded within it
-  Have no ``NEEDED`` entries for ``libc++.so`` or ``libstdc++.so`` in its dynamic section
-  Only depend on system libraries: ``libm.so.6``, ``libc.so.6``, ``ld-linux-x86-64.so.2``


This can be verified using ``ldd``:

.. code:: bash

   ldd /opt/software/googletest/1/1.17.0-llvm/lib64/libgtest.so
   # Shows only: libm.so.6, libc.so.6, ld-linux-x86-64.so.2
   # No libc++ or libstdc++ dependencies

.. _abi-compatibility-at-api-boundaries:

ABI compatibility at API boundaries
-----------------------------------

.. _the-critical-distinction:

The critical distinction
~~~~~~~~~~~~~~~~~~~~~~~~

Static linking eliminates runtime library dependencies, but it does not eliminate ABI compatibility requirements at API boundaries where standard library types are exposed.

.. _why-api-boundaries-matter:

Why API boundaries matter
~~~~~~~~~~~~~~~~~~~~~~~~~

When a library’s public API exposes standard library types (such as ``std::string``, ``std::vector``, ``std::unique_ptr``), the following occurs:

1. The library’s internal implementation uses its own standard library (e.g., libc++ with ``std::__1::`` namespace)
2. The public API functions are compiled with that standard library’s type definitions
3. Callers must use compatible type definitions to interact with those functions

.. _type-identity-and-name-mangling:

Type identity and name mangling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Different C++ standard library implementations use different internal namespaces:

-  libc++: Uses inline namespace ``std::__1::``, so ``std::string`` becomes ``std::__1::basic_string<...>``
-  libstdc++ (C++11 ABI): Uses ``std::__cxx11::``, so ``std::string`` becomes ``std::__cxx11::basic_string<...>``
-  libstdc++ (old ABI): Uses ``std::``, so ``std::string`` becomes ``std::basic_string<...>``

These are different types at the binary level, even though they represent the same logical type (``std::string``) in source code.

.. _example-googletests-getstring-function:

Example: GoogleTest’s GetString() function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

GoogleTest’s ``testing::Message::GetString()`` function signature:

.. code:: cpp

   std::string GetString() const;

When compiled with libc++ (statically linked), the mangled symbol is:

::

   _ZNK7testing7Message9GetStringEv

This function returns ``std::__1::string`` (libc++’s string type).

When code compiled with libstdc++ calls this function:

-  The caller expects to receive ``std::__cxx11::string`` (libstdc++’s string type)
-  The function actually returns ``std::__1::string`` (libc++’s string type)
-  These are incompatible types at the binary level


*Conclusion*: Even though GoogleTest has libc++ statically linked (no runtime dependency), the ABI mismatch occurs because the public API exposes ``std::string`` as a return type. To avoid this issue, the caller must use the same standard library ABI as the library. This means either:

-  Compile the caller with libc++ to match the library’s ABI, or
-  Use a GoogleTest build compiled with libstdc++ to match the caller’s ABI


Static linking eliminates runtime dependencies but does not eliminate the need for ABI compatibility at API boundaries when standard library types are involved.

.. _symbol-resolution-and-linker-errors:

Symbol resolution and linker errors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When linking code compiled with libstdc++ against a library compiled with libc++:

1. The linker looks for symbols matching the caller’s expectations
2. For ``std::string`` return types, it may look for ``[abi:cxx11]`` decorated symbols (libstdc++ C++11 ABI)
3. The library provides symbols with ``std::__1::`` namespace (libc++)
4. Mismatch results in undefined reference errors

Example error:

::

   undefined reference to `testing::Message::GetString[abi:cxx11]() const'

The ``[abi:cxx11]`` decoration indicates the caller expects libstdc++ C++11 ABI, but the library provides libc++ symbols.

.. _linkers-role-in-abi-compatibility:

Linker’s role in ABI compatibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The linker (whether ``ld`` or ``lld``) does not create ABI compatibility issues, but it detects and reports them during the linking phase.

What the linker does:

1. **Symbol resolution** - Matches function names and their mangled signatures from object files and libraries

2. **Symbol resolution** - Matches function names and their mangled signatures from object files and libraries

3. **Type checking** - Verifies that symbol signatures match between caller and callee

4. **Error reporting** - Reports undefined references when symbols cannot be found or signatures do not match

What the linker does not do:

1. **Does not create ABI mismatches** - The mismatch is created at compile time by the compiler’s name mangling
2. **Does not choose the standard library** - The compiler selects which standard library to use during compilation
3. **Does not affect type mangling** - Name mangling is determined by the compiler based on which standard library headers are used

How ABI mismatch is created:

1. **Compile time** - Compiler uses standard library headers (libc++ or libstdc++)
2. **Name mangling** - Compiler mangles types based on the standard library’s namespace:

   -  libc++: ``std::string`` → ``std::__1::basic_string<...>``
   -  libstdc++: ``std::string`` → ``std::__cxx11::basic_string<...>``

3. **Object file creation** - Mangled names are embedded in object files
4. **Link time** - Linker tries to match mangled names and fails if they don’t match

*Example - linker’s role*:

.. code:: bash

   # Compile library with libc++
   clang++ -stdlib=libc++ library.cpp -c -o library.o
   # Object file contains: _ZN7testing7Message9GetStringEv (returns std::__1::string)

   # Compile caller with libstdc++
   clang++ caller.cpp -c -o caller.o
   # Object file expects: _ZN7testing7Message9GetStringEv[abi:cxx11] (returns std::__cxx11::string)

   # Linker tries to match symbols
   ld caller.o library.o
   # Error: undefined reference - symbols don't match due to different name mangling

Linker choice (ld vs lld):

The choice between GNU linker (``ld``) and LLVM linker (``lld``) does not affect ABI compatibility:

-  Both linkers perform the same symbol resolution
-  Both report the same undefined reference errors for ABI mismatches
-  The ABI mismatch is in the symbol names themselves, not in how the linker processes them

When linker choice matters:

-  **Performance** - ``lld`` is generally faster than ``ld``
-  **Error messages** - ``lld`` may provide slightly different error formatting
-  **Feature support** - Some advanced linker features may differ
-  **libc++ integration** - ``lld`` may have better integration with libc++ in some edge cases, but this does not affect ABI compatibility

*Conclusion*: The linker detects ABI mismatches but does not cause them. ABI compatibility is determined at compile time by the compiler’s choice of standard library and resulting name mangling. Using ``lld`` instead of ``ld`` will not resolve ABI mismatches - the solution is to ensure consistent standard library usage at compile time.

.. _when-static-linking-works-across-different-standard-libraries:

When Static Linking Works Across Different Standard Libraries
-------------------------------------------------------------

.. important:: Static linking allows a library compiled with one standard library to be used by code compiled with a different standard library in the following scenarios:

.. _1-no-standard-library-types-in-public-api:

1. No standard library types in public API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Library uses only C types or POD structures in its public API.

*Example - C API wrapper*:

A library that uses only C types in its public API can be compiled with libc++ while being used by code compiled with libstdc++:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   extern "C" {
       void* create_object(const char* name, int value);
       void destroy_object(void* obj);
       int get_value(void* obj);
   }

*Why it works*:

-  No standard library types cross the API boundary
-  C types (``char*``, ``int``, ``void*``) have consistent representation
-  Caller can use any standard library


*Verification*:

The symbols can be verified to use C linkage without namespace decorations:

.. code:: bash

   nm library.so | grep "create_object\|destroy_object"
   # Shows C linkage symbols without namespace decorations

.. _2-internal-use-only:

2. Internal use only
~~~~~~~~~~~~~~~~~~~~

*Scenario*: Standard library types used only internally, never in public API.

*Example - Internal implementation*:

A library can use standard library types internally while exposing only C types in its public API:

.. code:: cpp

   // Library header (public API)
   class DataProcessor {
   public:
       void process(const char* input, char* output, size_t size);
       int get_result() const;
   private:
       // Implementation details hidden
   };

   // Library implementation (compiled with libc++, statically linked)
   #include <vector>
   #include <string>

   void DataProcessor::process(const char* input, char* output, size_t size) {
       std::vector<char> buffer(size);  // Internal use of libc++ vector
       std::string temp(input);         // Internal use of libc++ string
       // ... processing using libc++ types internally ...
   }

*Why it works*:

-  Public API uses only C types (``char*``, ``int``, ``size_t``)
-  Standard library types (``std::vector``, ``std::string``) never cross the boundary
-  Caller never sees or interacts with libc++ types


*Verification*:

The exported symbols can be checked to confirm they don’t include standard library types:

.. code:: bash

   nm -D library.so | grep "process"
   # Shows symbol without std:: types in signature

.. _3-template-based-apis-with-header-only-code:

3. Template-based APIs with header-only code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: API provided as header-only templates.

*Example - Header-only template library*:

Header-only template libraries work because templates are instantiated by the caller’s compiler:

.. code:: cpp

   // header_only_lib.hpp (compiled by caller, not pre-compiled)
   template<typename T>
   class Container {
   public:
       void add(const T& item) {
           data_.push_back(item);  // Uses caller's std::vector
       }
       T get(size_t index) const {
           return data_[index];
       }
   private:
       std::vector<T> data_;  // Instantiated with caller's std::vector
   };

*Why it works*:

-  Templates are instantiated by the caller’s compiler
-  Caller’s standard library is used for template instantiations
-  No pre-compiled binary with embedded standard library types


*Verification*:

-  Library has no ``.so`` or ``.a`` file, only header files
-  All code is in headers, compiled by the user


.. _4-opaque-pointer-pattern:

4. Opaque pointer pattern
~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Library uses opaque pointers to hide implementation details.

*Example - Opaque handle API*:

.. code:: cpp

   // Public API (C-compatible)
   typedef void* Handle;

   Handle create_handle();
   void set_data(Handle h, const char* data);
   const char* get_data(Handle h);
   void destroy_handle(Handle h);

   // Implementation (compiled with libc++, statically linked)
   #include <string>
   #include <unordered_map>

   static std::unordered_map<Handle, std::string> storage;  // Internal use

   Handle create_handle() {
       Handle h = new int(0);
       storage[h] = std::string();
       return h;
   }

*Why it works*:

-  Public API uses only C types (``void*``, ``char*``)
-  Standard library types (``std::string``, ``std::unordered_map``) are hidden
-  Caller never directly interacts with standard library types


*Verification*:

.. code:: bash

   nm -D library.so | c++filt | grep "create_handle\|set_data"
   # Shows functions with C types only, no std:: types visible

.. _when-static-linking-does-not-work-across-different-standard-libraries:

When static linking does not work across different standard libraries
---------------------------------------------------------------------

.. important:: Static linking does not eliminate ABI compatibility requirements when standard library types cross the API boundary.

The following scenarios will fail:

.. _1-standard-library-types-in-function-return-values:

1. Standard library types in function return values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Function returns a standard library type.

*Example - GoogleTest’s GetString()*:

When a function returns a standard library type, the return type must match between library and caller:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   namespace testing {
       class Message {
       public:
           std::string GetString() const;  // Returns std::__1::string
       };
   }

*Why it fails*:

-  Library returns ``std::__1::string`` (libc++ type)
-  Caller expects ``std::__cxx11::string`` (libstdc++ type)
-  These are different types at binary level
-  Linker error: ``undefined reference to testing::Message::GetString[abi:cxx11]() const``


*Error message*:

The linker will report an undefined reference error:

::

   /usr/bin/ld: undefined reference to `testing::Message::GetString[abi:cxx11]() const'

*Verification*:

The symbol mismatch can be verified by examining the library symbols:

.. code:: bash

   nm library.so | grep "GetString" | c++filt
   # Shows: testing::Message::GetString() const
   # But caller looks for: testing::Message::GetString[abi:cxx11]() const

.. _2-standard-library-types-in-function-parameters:

2. Standard Library Types in Function Parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Function takes standard library type by reference or value.

*Example*:

When function parameters include standard library types, the parameter types must match:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   class Config {
   public:
       void set_name(const std::string& name);  // Takes std::__1::string&
       void add_value(int value, const std::vector<int>& vec);  // Takes std::__1::vector&
   };

*Why it fails*:

-  Caller passes ``std::__cxx11::string&`` (libstdc++ type)
-  Function expects ``std::__1::string&`` (libc++ type)
-  Type mismatch at call site
-  May cause linker errors or runtime crashes


*Error example*:

The compiler will report a type conversion error:

::

   error: cannot convert 'std::__cxx11::string' to 'const std::__1::string&'

*Verification*:

The parameter types can be verified by examining the mangled symbols:

.. code:: bash

   nm library.so | grep "set_name" | c++filt
   # Shows parameter type with std::__1:: namespace

.. _3-standard-library-types-in-public-class-members:

3. Standard library types in public class members
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Public class exposes standard library types as members.

*Example*:

When public class members include standard library types, the memory layouts must match:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   class PublicData {
   public:
       std::string name;           // std::__1::string
       std::vector<int> values;    // std::__1::vector
       std::unique_ptr<Data> ptr;  // std::__1::unique_ptr
   };

*Why it fails*:

-  Caller code expects ``std::__cxx11::string`` layout
-  Library provides ``std::__1::string`` layout
-  Different memory layouts cause access violations
-  Virtual function tables may be incompatible


*Runtime error example*:

Accessing members with mismatched layouts can cause runtime errors:

::

   Segmentation fault (core dumped)
   # Or: std::bad_alloc, corrupted double-linked list, etc.

*Verification*:

The class layout can be examined through symbol inspection:

.. code:: bash

   nm library.so | grep "PublicData" | c++filt
   # Shows members with std::__1:: namespace

.. _4-exception-types-crossing-boundaries:

4. Exception Types Crossing Boundaries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Library throws exceptions derived from standard library types.

*Example*:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   void process_data() {
       if (error_condition) {
           throw std::runtime_error("error");  // std::__1::runtime_error
       }
   }

   // Caller code compiled with libstdc++
   try {
       process_data();
   } catch (const std::runtime_error& e) {  // Expects std::__cxx11::runtime_error
       // ...
   }

*Why it fails*:

-  Library throws ``std::__1::runtime_error`` (libc++ type)
-  Caller catches ``std::__cxx11::runtime_error`` (libstdc++ type)
-  Exception type mismatch prevents proper catch
-  Exception may propagate uncaught or cause undefined behaviour


*Runtime behaviour*:

-  Exception may not be caught by the expected handler
-  Program may terminate unexpectedly
-  Stack unwinding may fail


*Verification*:

.. code:: bash

   nm library.so | grep "runtime_error" | c++filt
   # Shows exception types with std::__1:: namespace

.. _5-virtual-function-tables-with-standard-library-types:

5. Virtual function tables with standard library types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Classes with virtual functions use standard library types in signatures.

*Example*:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   class Base {
   public:
       virtual std::string get_name() const = 0;  // Returns std::__1::string
       virtual void process(const std::vector<int>& data) = 0;  // Takes std::__1::vector&
   };

   class Derived : public Base {
   public:
       std::string get_name() const override;  // Returns std::__1::string
       void process(const std::vector<int>& data) override;  // Takes std::__1::vector&
   };

*Why it fails*:

-  Virtual function table contains function pointers
-  Function signatures include ``std::__1::`` types
-  Caller’s vtable expects ``std::__cxx11::`` types
-  Virtual function calls use wrong function signatures
-  Results in crashes or undefined behaviour


*Runtime error example*:

::

   pure virtual method called
   terminate called without an active exception

*Verification*:

.. code:: bash

   nm library.so | grep "vtable" | c++filt
   # Shows virtual function signatures with std::__1:: types

.. _6-template-specializations-with-standard-library-types:

6. Template specializations with standard library types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*Scenario*: Library provides template specializations for standard library types.

*Example*:

.. code:: cpp

   // Library compiled with libc++ (statically linked)
   template<typename T>
   void process(const T& value);

   template<>
   void process(const std::string& value);  // Specialization for std::__1::string

   // Caller code compiled with libstdc++
   std::string s = "test";  // std::__cxx11::string
   process(s);  // May not match the specialization

*Why it fails*:

-  Specialization is for ``std::__1::string``
-  Caller passes ``std::__cxx11::string``
-  Template matching fails or uses wrong specialization
-  May cause linker errors or wrong function to be called


*Error example*:

::

   undefined reference to `void process<std::__cxx11::basic_string<...>>(const std::__cxx11::basic_string<...>&)'

.. _comparison-table:

Comparison table
----------------

+-----------------------------------------+--------------------------------------+------------------------------------------+
| Scenario                                | Works Across Libraries               | Reason                                   |
+=========================================+======================================+==========================================+
| C API (char\ *, int, void*)             | Yes                                  | No standard library types in API         |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Opaque pointers                         | Yes                                  | Standard library types hidden            |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Internal use only                       | Yes                                  | Types don’t cross API boundary           |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Header-only templates                   | Yes                                  | Instantiated by caller’s compiler        |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Return std::string                      | No                                   | Return type must match caller’s ABI      |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Parameter std::string&                  | No                                   | Parameter type must match caller’s ABI   |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Public member std::string               | No                                   | Memory layout must match                 |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Exception std::runtime_error            | No                                   | Exception type must match                |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Virtual functions with std:: types      | No                                   | Vtable signatures must match             |
+-----------------------------------------+--------------------------------------+------------------------------------------+
| Template specialization for std:: types | No                                   | Specialization must match caller’s types |
+-----------------------------------------+--------------------------------------+------------------------------------------+

.. _simple-code-examples:

Simple code examples
--------------------

.. _example-a-library-with-c-api-works:

Example A: Library with C API (works)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates a library that uses standard library types internally but exposes only C types in its API.

*Library code* (compiled with libc++, statically linked):

.. code:: cpp

   // library.cpp
   #include <string>
   #include <vector>

   static std::vector<std::string> storage;  // Internal use of libc++

   extern "C" {
       void* create_item(const char* name) {
           storage.push_back(std::string(name));  // Uses libc++ internally
           return reinterpret_cast<void*>(storage.size() - 1);
       }
       
       const char* get_item_name(void* item) {
           size_t index = reinterpret_cast<size_t>(item);
           return storage[index].c_str();
       }
       
       void destroy_item(void* item) {
           // Cleanup if needed
       }
   }

*Caller code* (compiled with libstdc++):

.. code:: cpp

   // caller.cpp
   #include <iostream>

   extern "C" {
       void* create_item(const char* name);
       const char* get_item_name(void* item);
       void destroy_item(void* item);
   }

   int main() {
       void* item = create_item("test");
       std::cout << get_item_name(item) << std::endl;  // Uses libstdc++
       destroy_item(item);
       return 0;
   }

*Why it works*: API uses only C types (``void*``, ``char*``). Standard library types never cross the boundary.

*Compilation and linking*:

The library and caller can be compiled with different standard libraries:

.. code:: bash

   # Compile library with libc++ (statically linked)
   clang++ -stdlib=libc++ -nostdlib++ libc++.a libc++abi.a \
       -shared -fPIC library.cpp -o libexample.so

   # Compile caller with libstdc++
   g++ caller.cpp -L. -lexample -o caller

   # Works: no ABI mismatch

.. _example-b-library-with-stdstring-return-does-not-work:

Example B: Library with std::string return (does not work)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates why returning standard library types causes ABI mismatches.

*Library code* (compiled with libc++, statically linked):

.. code:: cpp

   // library.cpp
   #include <string>

   std::string get_message() {  // Returns std::__1::string
       return std::string("Hello from library");
   }

*Caller code* (compiled with libstdc++):

.. code:: cpp

   // caller.cpp
   #include <iostream>
   #include <string>

   std::string get_message();  // Expects std::__cxx11::string

   int main() {
       std::string msg = get_message();  // Type mismatch!
       std::cout << msg << std::endl;
       return 0;
   }

*Why it fails*: Function returns ``std::__1::string`` but caller expects ``std::__cxx11::string``.

*Error message*:

The linker will report an undefined reference:

::

   undefined reference to `get_message[abi:cxx11]()'

*Compilation attempt*:

.. code:: bash

   # Compile library with libc++ (statically linked)
   clang++ -stdlib=libc++ -nostdlib++ libc++.a libc++abi.a \
       -shared -fPIC library.cpp -o libexample.so

   # Compile caller with libstdc++
   g++ caller.cpp -L. -lexample -o caller
   # Error: undefined reference

.. _example-c-library-with-opaque-handle-works:

Example C: Library with opaque handle (works)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates how opaque handles can hide standard library types from the API.

*Library header* (public API):

.. code:: cpp

   // library.h
   #ifndef LIBRARY_H
   #define LIBRARY_H

   typedef void* StringHandle;

   #ifdef __cplusplus
   extern "C" {
   #endif

   StringHandle create_string(const char* value);
   const char* get_string_value(StringHandle h);
   void destroy_string(StringHandle h);

   #ifdef __cplusplus
   }
   #endif

   #endif

*Library implementation* (compiled with libc++, statically linked):

.. code:: cpp

   // library.cpp
   #include "library.h"
   #include <string>
   #include <unordered_map>

   static std::unordered_map<StringHandle, std::string> storage;  // Internal

   StringHandle create_string(const char* value) {
       StringHandle h = new int(0);
       storage[h] = std::string(value);  // Uses libc++ internally
       return h;
   }

   const char* get_string_value(StringHandle h) {
       return storage[h].c_str();
   }

   void destroy_string(StringHandle h) {
       storage.erase(h);
       delete static_cast<int*>(h);
   }

*Caller code* (compiled with libstdc++):

.. code:: cpp

   // caller.cpp
   #include "library.h"
   #include <iostream>
   #include <string>  // Uses libstdc++

   int main() {
       StringHandle h = create_string("Hello");
       std::cout << get_string_value(h) << std::endl;  // Uses libstdc++
       destroy_string(h);
       return 0;
   }

*Why it works*: Public API uses only C types. Standard library types are hidden internally.

*Compilation and linking*:

.. code:: bash

   # Compile library with libc++ (statically linked)
   clang++ -stdlib=libc++ -nostdlib++ libc++.a libc++abi.a \
       -shared -fPIC library.cpp -o libexample.so

   # Compile caller with libstdc++
   g++ caller.cpp -L. -lexample -o caller

   # Works: no std:: types in API

.. _example-d-library-with-public-stdstring-member-does-not-work:

Example D: Library with public std::string member (does not work)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates why public members with standard library types cause ABI mismatches.

*Library header* (public API):

.. code:: cpp

   // library.h
   #include <string>

   class Data {
   public:
       std::string name;  // Public member: std::__1::string when compiled with libc++
       int value;
   };

*Library implementation* (compiled with libc++, statically linked):

.. code:: cpp

   // library.cpp
   #include "library.h"

   // Implementation uses libc++ types internally

*Caller code* (compiled with libstdc++):

.. code:: cpp

   // caller.cpp
   #include "library.h"
   #include <iostream>

   int main() {
       Data d;
       d.name = "test";  // Assigns std::__cxx11::string to std::__1::string member
       d.value = 42;
       std::cout << d.name << std::endl;  // Accesses wrong memory layout
       return 0;
   }

*Why it fails*: Public member ``name`` has different memory layout (``std::__1::string`` vs ``std::__cxx11::string``).

*Runtime error*:

::

   Segmentation fault (core dumped)
   # Or: std::bad_alloc, corrupted memory

*Compilation and linking*:

.. code:: bash

   # Compile library with libc++ (statically linked)
   clang++ -stdlib=libc++ -nostdlib++ libc++.a libc++abi.a \
       -shared -fPIC library.cpp -o libexample.so

   # Compile caller with libstdc++
   g++ caller.cpp -L. -lexample -o caller

   # Compiles but crashes at runtime due to memory layout mismatch

.. _example-e-header-only-template-works:

Example E: Header-only template (works)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates how header-only template libraries avoid ABI issues.

*Library header* (no compiled binary):

.. code:: cpp

   // library.hpp
   #ifndef LIBRARY_HPP
   #define LIBRARY_HPP

   #include <vector>

   template<typename T>
   class Stack {
   public:
       void push(const T& item) {
           data_.push_back(item);  // Uses caller's std::vector
       }
       
       T pop() {
           T item = data_.back();
           data_.pop_back();
           return item;
       }
       
       bool empty() const {
           return data_.empty();
       }

   private:
       std::vector<T> data_;  // Instantiated with caller's std::vector
   };

   #endif

*Caller code* (compiled with libstdc++):

.. code:: cpp

   // caller.cpp
   #include "library.hpp"
   #include <iostream>
   #include <string>  // Uses libstdc++

   int main() {
       Stack<std::string> stack;  // Uses libstdc++ std::string and std::vector
       stack.push("hello");
       stack.push("world");
       
       while (!stack.empty()) {
           std::cout << stack.pop() << std::endl;
       }
       return 0;
   }

*Why it works*: Templates are instantiated by caller’s compiler using caller’s standard library.

*Compilation*:

.. code:: bash

   # No library compilation needed - header only
   # Compile caller with libstdc++
   g++ caller.cpp -o caller

   # Works: all code compiled together with same standard library

.. _practical-examples:

Practical examples
------------------

.. _example-1-googletest---does-not-work:

Example 1: GoogleTest - does not work
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates a real-world ABI mismatch scenario with GoogleTest.

*Library*: GoogleTest compiled with libc++ (statically linked)

*Public API*:

.. code:: cpp

   namespace testing {
       class Message {
       public:
           std::string GetString() const;  // Returns std::__1::string
       };
   }

*Caller code*: Compiled with libstdc++

*Result*: Fails

*Why*:

-  ``GetString()`` returns ``std::__1::string`` (libc++ type)
-  Caller expects ``std::__cxx11::string`` (libstdc++ type)
-  Linker error: ``undefined reference to testing::Message::GetString[abi:cxx11]() const``


*Solution*: Match ABI requirements

To resolve this issue, ensure ABI compatibility by using matching standard library implementations:

1. **Option 1**: Compile caller with libc++ to match the library

   .. code:: bash

      clang++ -stdlib=libc++ caller.cpp -lgtest -o caller

2. **Option 2**: Use GoogleTest compiled with libstdc++ to match the caller

   .. code:: bash

      # Use GoogleTest built with libstdc++ (same as caller)
      g++ caller.cpp -lgtest -o caller

3. **Option 3**: Use bundled GoogleTest compiled with the same toolchain as the main project

   -  This ensures automatic ABI matching
   -  No external dependency conflicts
   -  Example: Ginkgo bundles GoogleTest via FetchContent, compiling it with the same compiler and standard library

*Key principle*: Static linking of libc++ in GoogleTest eliminates runtime dependencies but does not eliminate the need for ABI compatibility. When the API exposes standard library types, both the library and the caller must use compatible ABIs, regardless of whether the library statically links its standard library.

.. _example-2-c-api-wrapper---works:

Example 2: C API wrapper - works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates how a C API wrapper successfully avoids ABI issues.

*Library*: Compiled with libc++ (statically linked), exposes C API

*Public API*:

.. code:: cpp

   extern "C" {
       void* create_processor(const char* config);
       void process_data(void* proc, const void* input, size_t size, void* output);
       void destroy_processor(void* proc);
   }

*Caller code*: Compiled with libstdc++

*Result*: Works

*Why*:

-  API uses only C types (``void*``, ``char*``, ``size_t``)
-  No standard library types cross the boundary
-  Internal use of ``std::vector``, ``std::string`` is hidden


*Verification*:

.. code:: bash

   nm -D library.so | grep "create_processor\|process_data"
   # Shows C linkage, no namespace decorations

.. _example-3-opaque-handle-library---works:

Example 3: Opaque handle library - works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates how opaque handles successfully hide standard library types.

*Library*: Compiled with libc++ (statically linked)

*Public API*:

.. code:: cpp

   typedef void* DataHandle;

   DataHandle create_data();
   void set_string(DataHandle h, const char* str);
   const char* get_string(DataHandle h);
   void destroy_data(DataHandle h);

*Implementation* (hidden):

.. code:: cpp

   #include <string>
   #include <unordered_map>

   static std::unordered_map<DataHandle, std::string> storage;

   DataHandle create_data() {
       DataHandle h = new int(0);
       storage[h] = std::string();
       return h;
   }

*Caller code*: Compiled with libstdc++

*Result*: Works

*Why*:

-  Public API uses only C types (``void*``, ``char*``)
-  Standard library types (``std::string``, ``std::unordered_map``) are internal
-  Caller never directly interacts with standard library types


.. _example-4-container-library-with-public-members---does-not-work:

Example 4: Container library with public members - does not work
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates why public members with standard library types cause failures.

*Library*: Compiled with libc++ (statically linked)

*Public API*:

.. code:: cpp

   class Container {
   public:
       std::string name;           // Public member: std::__1::string
       std::vector<int> values;    // Public member: std::__1::vector
       
       void add(int value);
       int get(size_t index) const;
   };

*Caller code*: Compiled with libstdc++

*Result*: Fails

*Why*:

-  Public members expose ``std::__1::string`` and ``std::__1::vector``
-  Caller code expects ``std::__cxx11::string`` and ``std::__cxx11::vector``
-  Different memory layouts cause access violations
-  Direct member access uses wrong offsets


*Runtime error*:

::

   Segmentation fault (core dumped)
   # Or: std::bad_alloc when accessing members

.. _example-5-header-only-template-library---works:

Example 5: Header-only template library - works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: This example demonstrates how header-only template libraries avoid ABI issues.

*Library*: Header-only, no pre-compiled binary

*Public API* (in header file):

.. code:: cpp

   template<typename T>
   class Stack {
   public:
       void push(const T& item) {
           data_.push_back(item);  // Uses caller's std::vector
       }
       T pop() {
           T item = data_.back();
           data_.pop_back();
           return item;
       }
   private:
       std::vector<T> data_;  // Instantiated with caller's std::vector
   };

*Caller code*: Compiled with libstdc++

*Result*: Works

*Why*:

-  Templates are instantiated by caller’s compiler
-  Caller’s ``std::vector`` is used for instantiation
-  No pre-compiled binary with embedded standard library types
-  All code compiled together with same standard library


*Verification*:

-  Library has no ``.so`` or ``.a`` file
-  Only header files provided
-  User compiles the code themselves


.. _bundled-dependencies:

Bundled dependencies
~~~~~~~~~~~~~~~~~~~~

When a project bundles dependencies (like Ginkgo bundles GoogleTest via FetchContent):

-  The bundled dependency is compiled with the same compiler and standard library as the main project
-  ABI compatibility is automatically ensured
-  No external dependency conflicts occur


.. _mixed-toolchain-scenarios:

Mixed toolchain scenarios
~~~~~~~~~~~~~~~~~~~~~~~~~

When using mixed toolchains (e.g., ``clang++`` with libstdc++ for host code, ``nvc++`` with libstdc++ for CUDA code):

-  All components must use the same standard library (libstdc++)
-  Static linking of libc++ in one component does not help if its API exposes std:: types
-  Consistent ABI across all components is required


.. _quick-reference-working-vs-non-working-patterns:

Quick reference: working vs non-working patterns
------------------------------------------------

.. _pattern-1-c-api-works:

Pattern 1: C API (works)
~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library
   extern "C" {
       void* create(const char* name);
       const char* get_name(void* obj);
   }
   // Internal use of std::string is hidden

.. _pattern-2-opaque-handle-works:

Pattern 2: Opaque handle (works)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library
   typedef void* Handle;
   Handle create();
   void set_data(Handle h, const char* data);
   // Internal use of std::string is hidden

.. _pattern-3-return-stdstring-does-not-work:

Pattern 3: Return std::string (does not work)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library
   std::string get_name();  // Returns std::__1::string
   // Caller expects std::__cxx11::string

.. _pattern-4-parameter-stdstring-does-not-work:

Pattern 4: Parameter std::string& (does not work)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library
   void set_name(const std::string& name);  // Takes std::__1::string&
   // Caller passes std::__cxx11::string&

.. _pattern-5-public-member-does-not-work:

Pattern 5: Public member (does not work)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library
   class Data {
   public:
       std::string name;  // std::__1::string layout
   };
   // Caller expects std::__cxx11::string layout

.. _pattern-6-header-only-template-works:

Pattern 6: Header-only template (works)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: cpp

   // Library (header only)
   template<typename T>
   class Container {
       std::vector<T> data_;  // Instantiated by caller
   };
   // Caller's std::vector is used

.. _verification-methods:

Verification methods
--------------------

.. _check-runtime-dependencies:

Check runtime dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   ldd library.so | grep -i "c++"

If no C++ standard library appears, the library has statically linked its standard library.

.. _check-symbol-namespaces:

Check symbol namespaces
~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   nm -D library.so | grep "basic_string"

Look for namespace indicators:

-  ``NSt3__1`` indicates libc++ (``std::__1::``)
-  ``NSt7__cxx11`` indicates libstdc++ C++11 ABI (``std::__cxx11::``)
-  ``NSt`` without these indicates libstdc++ old ABI


.. _check-api-function-signatures:

Check API function signatures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   nm library.so | grep "function_name" | c++filt

This shows the actual function signature with namespace information.

.. _check-dynamic-section:

Check dynamic section
~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

   readelf -d library.so | grep NEEDED

Shows all runtime library dependencies. Absence of libc++ or libstdc++ indicates static linking.

.. _practical-guidelines-for-clang-with-libstdc:

Practical guidelines for ``clang++`` with libstdc++
---------------------------------------------------

.. _recommended-approach:

Recommended approach
~~~~~~~~~~~~~~~~~~~~

For maximum compatibility and to avoid ABI issues:

1. **Use** ``clang++`` **with libstdc++ by default** - No special flags needed on Linux
2. **Ensure consistent ABI** - Use the same ``_GLIBCXX_USE_CXX11_ABI`` setting across all components
3. **Match GCC version** - Use ``clang++`` with the same libstdc++ version that GCC uses
4. **Verify with tools** - Use ``nm``, ``ldd``, and ``readelf`` to verify ABI consistency

.. _build-system-configuration:

Build system configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~

*CMake example*:

.. code:: cmake

   # Use clang++ with libstdc++ (default on Linux)
   set(CMAKE_CXX_COMPILER clang++)

   # Ensure C++11 ABI is used consistently
   add_compile_definitions(_GLIBCXX_USE_CXX11_ABI=1)

   # Verify standard library
   message(STATUS "C++ standard library: ${CMAKE_CXX_STANDARD_LIBRARIES}")

*Makefile example*:

.. code:: makefile

   CXX = clang++
   CXXFLAGS = -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=1
   LDFLAGS = -lstdc++

   # Verify libstdc++ is used
   verify:
       @clang++ -x c++ -E -dM /dev/null 2>&1 | grep -q "__GLIBCXX__" && \
           echo "Using libstdc++" || echo "Not using libstdc++"

.. _troubleshooting:

Troubleshooting
~~~~~~~~~~~~~~~

*Issue*: ``Clang++`` not finding libstdc++ headers

*Solution*:

.. code:: bash

   # Check GCC installation
   g++ -print-search-dirs

   # Manually specify include path if needed
   clang++ -I/usr/include/c++/11 -I/usr/include/c++/11/x86_64-redhat-linux source.cpp

*Issue*: ABI mismatch errors

*Solution*:

.. code:: bash

   # Verify ABI setting is consistent
   clang++ -E -dM source.cpp | grep "_GLIBCXX_USE_CXX11_ABI"

   # Ensure all components use the same setting
   # Recompile dependencies if necessary

*Issue*: Template instantiation differences

*Solution*:

-  Use explicit template instantiation
-  Avoid relying on compiler-specific template behaviour
-  Test with both GCC and ``clang++`` if compatibility is required


.. _code-patching-and-abi-problems:

Code patching and ABI problems
------------------------------

Code patching, in various forms, cannot reliably resolve ABI compatibility problems when standard library types cross API boundaries. Here’s why and what alternatives exist:

.. _why-patching-cannot-solve-abi-mismatches:

Why patching cannot solve ABI mismatches
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**1. Binary-level type incompatibility**:

ABI mismatches occur at the binary level - the actual memory layout and symbol names are different:

-  ``std::__1::string`` (libc++) has different memory layout than ``std::__cxx11::string`` (libstdc++)
-  Function signatures are mangled differently
-  Virtual function tables have different structures

*Example - memory layout difference*:

.. code:: cpp

   // libc++ std::string layout (simplified)
   struct string {
       union {
           char __s[23];           // Small string optimisation
           struct {                // Long string
               char* __data;
               size_t __size;
               size_t __cap;
           };
       };
   };

   // libstdc++ std::string layout (simplified)
   struct string {
       char* _M_dataplus;         // Pointer to data + allocator
       size_t _M_string_length;   // Size
       union {
           char _M_local_buf[15]; // Small string optimisation
           size_t _M_allocated_capacity;
       };
   };

These are fundamentally different structures. Patching cannot convert between them because:

-  Field offsets are different
-  Field names are different
-  Memory allocation strategies may differ


**2. Symbol name mangling**:

Function names are mangled at compile time based on the standard library used:

.. code:: cpp

   // Source code
   std::string get_name();

   // Compiled with libc++
   _Z9get_namev  // Returns std::__1::string

   // Compiled with libstdc++
   _Z9get_namev[abi:cxx11]  // Returns std::__cxx11::string

The linker looks for exact symbol matches. Patching symbol names would require:

-  Modifying all object files and libraries
-  Maintaining a mapping of all affected symbols
-  Ensuring all call sites are updated
-  This is impractical for real-world applications


**3. Virtual function tables**:

Virtual function tables contain function pointers with specific signatures. If the signatures include standard library types, the vtable entries are incompatible:

.. code:: cpp

   class Base {
   public:
       virtual std::string get_name() const = 0;  // Vtable entry depends on std::string type
   };

Patching vtables would require:

-  Understanding the vtable layout
-  Modifying function pointers
-  Ensuring all derived classes match
-  This is extremely fragile and error-prone


.. _when-patching-might-be-possible-but-not-recommended:

When patching might be possible (but not recommended)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**1. Wrapper functions**:

Creating wrapper code that bridges ABI differences:

.. code:: cpp

   // Library provides libc++ interface
   std::__1::string library_get_name();

   // Wrapper to convert to libstdc++
   std::string get_name_wrapper() {
       std::__1::string libc_str = library_get_name();
       // Convert libc++ string to libstdc++ string
       return std::string(libc_str.data(), libc_str.size());
   }

*Limitations*:

-  Requires source code access
-  Must create wrappers for every function
-  Performance overhead from conversions
-  Memory copies for string/vector conversions
-  Complex for nested types (vector, etc.)


**2. Binary patching (theoretical)**:

Modifying compiled binaries to change symbol names or function signatures.

*Why it doesn’t work in practice*:

-  Extremely fragile and error-prone
-  Requires deep understanding of binary formats
-  May break with compiler updates
-  Legal and security concerns
-  Not maintainable


**3. Runtime type conversion**:

Converting types at runtime when crossing boundaries.

*Example*:

.. code:: cpp

   // Library function (libc++)
   std::__1::string lib_func();

   // Caller wrapper (libstdc++)
   std::string caller_func() {
       auto libc_result = lib_func();
       // Convert at runtime
       return std::string(libc_result.c_str());
   }

*Limitations*:

-  Only works if you have source code
-  Performance overhead
-  Memory allocations for conversions
-  Complex for nested containers
-  Exception safety issues


.. _why-matching-abis-is-the-correct-solution:

Why matching ABIs is the correct solution
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**1. Compile-time resolution**:

Matching ABIs ensures compatibility at compile time:

-  No runtime overhead
-  No conversion code needed
-  Type safety maintained
-  Compiler can optimise


**2. Maintainability**:

Using consistent ABIs:

-  Simplifies build system
-  Reduces complexity
-  Easier to debug
-  Standard practice


**3. Performance**:

No conversion overhead:

-  Direct type usage
-  No memory copies
-  No wrapper function calls
-  Optimal performance


**4. Reliability**:

Matching ABIs:

-  No fragile workarounds
-  No edge cases
-  Works with all language features
-  Future-proof


.. _practical-solutions-instead-of-patching:

Practical solutions instead of patching
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**1. Recompile with matching ABI**:

The correct solution is to ensure all components use the same ABI:

.. code:: bash

   # Recompile library to match caller's ABI
   clang++ -stdlib=libstdc++ library.cpp -shared -o libexample.so

   # Or recompile caller to match library's ABI
   clang++ -stdlib=libc++ caller.cpp -lexample -o caller

**2. Use bundled dependencies**:

Build dependencies with the same toolchain as the main project:

.. code:: cmake

   # Ginkgo bundles GoogleTest via FetchContent
   # GoogleTest is compiled with the same compiler/ABI as Ginkgo
   # Automatic ABI matching

**3. C API wrappers**:

If you control the library, provide a C API that avoids standard library types:

.. code:: cpp

   // C API (no std:: types)
   extern "C" {
       void* create_object(const char* name);
       const char* get_name(void* obj);
   }

**4. Opaque handles**:

Hide standard library types behind opaque pointers:

.. code:: cpp

   typedef void* StringHandle;
   StringHandle create_string(const char* value);
   const char* get_string_value(StringHandle h);

.. _conclusion-on-code-patching:

Conclusion on code patching
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Code patching cannot reliably resolve ABI compatibility problems when standard library types are involved because:

1. **Binary incompatibility** - Different memory layouts cannot be patched
2. **Symbol mangling** - Mangled names are embedded at compile time
3. **Type system** - C++ type system prevents safe conversions between incompatible types
4. **Complexity** - Patching would require modifying all affected code

The correct solution is to ensure ABI compatibility at compile time by using consistent standard library implementations across all components. This is simpler, more reliable, and more performant than any patching approach.

.. _conclusion:

Conclusion
----------

``Clang++`` compilers can use libstdc++ by default on Linux systems, providing compatibility with GCC-compiled code when the same ABI settings are used. Static linking of C++ standard libraries eliminates runtime dependencies but does not eliminate ABI compatibility requirements at API boundaries. When a library’s public API exposes standard library types, callers must use a compatible standard library implementation, regardless of whether the library statically links its standard library.

Key principles:

-  ``Clang++`` uses libstdc++ by default on Linux
-  no special configuration needed
-  Static linking affects runtime dependencies, but API boundaries require compile-time ABI compatibility when standard library types are involved
-  Consistent ABI settings (``_GLIBCXX_USE_CXX11_ABI``) across all components are essential for compatibility
-  Code patching cannot resolve ABI mismatches
-  the solution is to match ABIs at compile time