summaryrefslogtreecommitdiffstats
path: root/gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst
diff options
context:
space:
mode:
authorpatrick <patrick@openbsd.org>2020-08-03 15:06:44 +0000
committerpatrick <patrick@openbsd.org>2020-08-03 15:06:44 +0000
commitb64793999546ed8adebaeebd9d8345d18db8927d (patch)
tree4357c27b561d73b0e089727c6ed659f2ceff5f47 /gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst
parentAdd support for UTF-8 DISPLAY-HINTs with octet length. For now only (diff)
downloadwireguard-openbsd-b64793999546ed8adebaeebd9d8345d18db8927d.tar.xz
wireguard-openbsd-b64793999546ed8adebaeebd9d8345d18db8927d.zip
Remove LLVM 8.0.1 files.
Diffstat (limited to 'gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst')
-rw-r--r--gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst295
1 files changed, 0 insertions, 295 deletions
diff --git a/gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst b/gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst
deleted file mode 100644
index 27a1950a160..00000000000
--- a/gnu/llvm/tools/clang/docs/SourceBasedCodeCoverage.rst
+++ /dev/null
@@ -1,295 +0,0 @@
-==========================
-Source-based Code Coverage
-==========================
-
-.. contents::
- :local:
-
-Introduction
-============
-
-This document explains how to use clang's source-based code coverage feature.
-It's called "source-based" because it operates on AST and preprocessor
-information directly. This allows it to generate very precise coverage data.
-
-Clang ships two other code coverage implementations:
-
-* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
- various sanitizers. It can provide up to edge-level coverage.
-
-* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
- This is enabled by ``-ftest-coverage`` or ``--coverage``.
-
-From this point onwards "code coverage" will refer to the source-based kind.
-
-The code coverage workflow
-==========================
-
-The code coverage workflow consists of three main steps:
-
-* Compiling with coverage enabled.
-
-* Running the instrumented program.
-
-* Creating coverage reports.
-
-The next few sections work through a complete, copy-'n-paste friendly example
-based on this program:
-
-.. code-block:: cpp
-
- % cat <<EOF > foo.cc
- #define BAR(x) ((x) || (x))
- template <typename T> void foo(T x) {
- for (unsigned I = 0; I < 10; ++I) { BAR(I); }
- }
- int main() {
- foo<int>(0);
- foo<float>(0);
- return 0;
- }
- EOF
-
-Compiling with coverage enabled
-===============================
-
-To compile code with coverage enabled, pass ``-fprofile-instr-generate
--fcoverage-mapping`` to the compiler:
-
-.. code-block:: console
-
- # Step 1: Compile with coverage enabled.
- % clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
-
-Note that linking together code with and without coverage instrumentation is
-supported. Uninstrumented code simply won't be accounted for in reports.
-
-Running the instrumented program
-================================
-
-The next step is to run the instrumented program. When the program exits it
-will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
-environment variable. If that variable does not exist, the profile is written
-to ``default.profraw`` in the current directory of the program. If
-``LLVM_PROFILE_FILE`` contains a path to a non-existent directory, the missing
-directory structure will be created. Additionally, the following special
-**pattern strings** are rewritten:
-
-* "%p" expands out to the process ID.
-
-* "%h" expands out to the hostname of the machine running the program.
-
-* "%Nm" expands out to the instrumented binary's signature. When this pattern
- is specified, the runtime creates a pool of N raw profiles which are used for
- on-line profile merging. The runtime takes care of selecting a raw profile
- from the pool, locking it, and updating it before the program exits. If N is
- not specified (i.e the pattern is "%m"), it's assumed that ``N = 1``. N must
- be between 1 and 9. The merge pool specifier can only occur once per filename
- pattern.
-
-.. code-block:: console
-
- # Step 2: Run the program.
- % LLVM_PROFILE_FILE="foo.profraw" ./foo
-
-Creating coverage reports
-=========================
-
-Raw profiles have to be **indexed** before they can be used to generate
-coverage reports. This is done using the "merge" tool in ``llvm-profdata``
-(which can combine multiple raw profiles and index them at the same time):
-
-.. code-block:: console
-
- # Step 3(a): Index the raw profile.
- % llvm-profdata merge -sparse foo.profraw -o foo.profdata
-
-There are multiple different ways to render coverage reports. The simplest
-option is to generate a line-oriented report:
-
-.. code-block:: console
-
- # Step 3(b): Create a line-oriented coverage report.
- % llvm-cov show ./foo -instr-profile=foo.profdata
-
-This report includes a summary view as well as dedicated sub-views for
-templated functions and their instantiations. For our example program, we get
-distinct views for ``foo<int>(...)`` and ``foo<float>(...)``. If
-``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays sub-line
-region counts (even in macro expansions):
-
-.. code-block:: none
-
- 1| 20|#define BAR(x) ((x) || (x))
- ^20 ^2
- 2| 2|template <typename T> void foo(T x) {
- 3| 22| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
- ^22 ^20 ^20^20
- 4| 2|}
- ------------------
- | void foo<int>(int):
- | 2| 1|template <typename T> void foo(T x) {
- | 3| 11| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
- | ^11 ^10 ^10^10
- | 4| 1|}
- ------------------
- | void foo<float>(int):
- | 2| 1|template <typename T> void foo(T x) {
- | 3| 11| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
- | ^11 ^10 ^10^10
- | 4| 1|}
- ------------------
-
-To generate a file-level summary of coverage statistics instead of a
-line-oriented report, try:
-
-.. code-block:: console
-
- # Step 3(c): Create a coverage summary.
- % llvm-cov report ./foo -instr-profile=foo.profdata
- Filename Regions Missed Regions Cover Functions Missed Functions Executed Lines Missed Lines Cover
- --------------------------------------------------------------------------------------------------------------------------------------
- /tmp/foo.cc 13 0 100.00% 3 0 100.00% 13 0 100.00%
- --------------------------------------------------------------------------------------------------------------------------------------
- TOTAL 13 0 100.00% 3 0 100.00% 13 0 100.00%
-
-The ``llvm-cov`` tool supports specifying a custom demangler, writing out
-reports in a directory structure, and generating html reports. For the full
-list of options, please refer to the `command guide
-<https://llvm.org/docs/CommandGuide/llvm-cov.html>`_.
-
-A few final notes:
-
-* The ``-sparse`` flag is optional but can result in dramatically smaller
- indexed profiles. This option should not be used if the indexed profile will
- be reused for PGO.
-
-* Raw profiles can be discarded after they are indexed. Advanced use of the
- profile runtime library allows an instrumented program to merge profiling
- information directly into an existing raw profile on disk. The details are
- out of scope.
-
-* The ``llvm-profdata`` tool can be used to merge together multiple raw or
- indexed profiles. To combine profiling data from multiple runs of a program,
- try e.g:
-
- .. code-block:: console
-
- % llvm-profdata merge -sparse foo1.profraw foo2.profdata -o foo3.profdata
-
-Exporting coverage data
-=======================
-
-Coverage data can be exported into JSON using the ``llvm-cov export``
-sub-command. There is a comprehensive reference which defines the structure of
-the exported data at a high level in the llvm-cov source code.
-
-Interpreting reports
-====================
-
-There are four statistics tracked in a coverage summary:
-
-* Function coverage is the percentage of functions which have been executed at
- least once. A function is considered to be executed if any of its
- instantiations are executed.
-
-* Instantiation coverage is the percentage of function instantiations which
- have been executed at least once. Template functions and static inline
- functions from headers are two kinds of functions which may have multiple
- instantiations.
-
-* Line coverage is the percentage of code lines which have been executed at
- least once. Only executable lines within function bodies are considered to be
- code lines.
-
-* Region coverage is the percentage of code regions which have been executed at
- least once. A code region may span multiple lines (e.g in a large function
- body with no control flow). However, it's also possible for a single line to
- contain multiple code regions (e.g in "return x || y && z").
-
-Of these four statistics, function coverage is usually the least granular while
-region coverage is the most granular. The project-wide totals for each
-statistic are listed in the summary.
-
-Format compatibility guarantees
-===============================
-
-* There are no backwards or forwards compatibility guarantees for the raw
- profile format. Raw profiles may be dependent on the specific compiler
- revision used to generate them. It's inadvisable to store raw profiles for
- long periods of time.
-
-* Tools must retain **backwards** compatibility with indexed profile formats.
- These formats are not forwards-compatible: i.e, a tool which uses format
- version X will not be able to understand format version (X+k).
-
-* Tools must also retain **backwards** compatibility with the format of the
- coverage mappings emitted into instrumented binaries. These formats are not
- forwards-compatible.
-
-* The JSON coverage export format has a (major, minor, patch) version triple.
- Only a major version increment indicates a backwards-incompatible change. A
- minor version increment is for added functionality, and patch version
- increments are for bugfixes.
-
-Using the profiling runtime without static initializers
-=======================================================
-
-By default the compiler runtime uses a static initializer to determine the
-profile output path and to register a writer function. To collect profiles
-without using static initializers, do this manually:
-
-* Export a ``int __llvm_profile_runtime`` symbol from each instrumented shared
- library and executable. When the linker finds a definition of this symbol, it
- knows to skip loading the object which contains the profiling runtime's
- static initializer.
-
-* Forward-declare ``void __llvm_profile_initialize_file(void)`` and call it
- once from each instrumented executable. This function parses
- ``LLVM_PROFILE_FILE``, sets the output path, and truncates any existing files
- at that path. To get the same behavior without truncating existing files,
- pass a filename pattern string to ``void __llvm_profile_set_filename(char
- *)``. These calls can be placed anywhere so long as they precede all calls
- to ``__llvm_profile_write_file``.
-
-* Forward-declare ``int __llvm_profile_write_file(void)`` and call it to write
- out a profile. This function returns 0 when it succeeds, and a non-zero value
- otherwise. Calling this function multiple times appends profile data to an
- existing on-disk raw profile.
-
-In C++ files, declare these as ``extern "C"``.
-
-Collecting coverage reports for the llvm project
-================================================
-
-To prepare a coverage report for llvm (and any of its sub-projects), add
-``-DLLVM_BUILD_INSTRUMENTED_COVERAGE=On`` to the cmake configuration. Raw
-profiles will be written to ``$BUILD_DIR/profiles/``. To prepare an html
-report, run ``llvm/utils/prepare-code-coverage-artifact.py``.
-
-To specify an alternate directory for raw profiles, use
-``-DLLVM_PROFILE_DATA_DIR``. To change the size of the profile merge pool, use
-``-DLLVM_PROFILE_MERGE_POOL_SIZE``.
-
-Drawbacks and limitations
-=========================
-
-* Prior to version 2.26, the GNU binutils BFD linker is not able link programs
- compiled with ``-fcoverage-mapping`` in its ``--gc-sections`` mode. Possible
- workarounds include disabling ``--gc-sections``, upgrading to a newer version
- of BFD, or using the Gold linker.
-
-* Code coverage does not handle unpredictable changes in control flow or stack
- unwinding in the presence of exceptions precisely. Consider the following
- function:
-
- .. code-block:: cpp
-
- int f() {
- may_throw();
- return 0;
- }
-
- If the call to ``may_throw()`` propagates an exception into ``f``, the code
- coverage tool may mark the ``return`` statement as executed even though it is
- not. A call to ``longjmp()`` can have similar effects.