Summary:
Currently the FDR log writer, upon flushing, dumps a sequence of buffers from
its freelist to disk. A reader can read the first buffer up to an EOB record,
but then it is unclear how far ahead to scan to find the next threads traces.
There are a few ways to handle this problem.
1. The reader has externalized knowledge of the buffer size.
2. The size of buffers is in the file header or otherwise encoded in the log.
3. Only write out the portion of the buffer with records. When released, the
buffers are marked with a size.
4. The reader looks for memory that matches a pattern and synchronizes on it.
2 and 3 seem the most flexible and 2 does not rule 3 out.
This is an implementation of 2.
In addition, the function handler for fdr more aggressively checks for
finalization and makes an attempt to release its buffer.
Reviewers: pelikan, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31384
llvm-svn: 298982
Summary:
This change exercises the end-to-end functionality defined in the FDR
logging implementation. We also prepare for being able to run traces
generated by the FDR logging implementation from being analysed with the
llvm-xray command that comes with the LLVM distribution.
This also unblocks D31385, D31384, and D31345.
Reviewers: kpw, pelikan
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D31452
llvm-svn: 298977
There are several problems with the current annotations (AnnotateRWLockCreate and friends):
- they don't fully support deadlock detection (we need a hook _before_ mutex lock)
- they don't support insertion of random artificial delays to perturb execution (again we need a hook _before_ mutex lock)
- they don't support setting extended mutex attributes like read/write reentrancy (only "linker init" was bolted on)
- they don't support setting mutex attributes if a mutex don't have a "constructor" (e.g. static, Java, Go mutexes)
- they don't ignore synchronization inside of lock/unlock operations which leads to slowdown and false negatives
The new annotations solve of the above problems. See tsan_interface.h for the interface specification and comments.
Reviewed in https://reviews.llvm.org/D31093
llvm-svn: 298809
Summary:
Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled
in a way to (optionally) pass their first call argument to your logging handler.
For practical and performance reasons, only the first argument is supported, and
only up to 64 bits.
Reviewers: javed.absar, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29703
llvm-svn: 297000
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 293015
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 290852
Summary: The function computes full module name and coverts pc into offset.
Reviewers: kcc
Subscribers: kubabrecka
Differential Revision: https://reviews.llvm.org/D26820
llvm-svn: 288711
Summary:
This change depends on D23986 which adds tail call-specific sleds. For
now we treat them first as normal exits, and in the future leave room
for implementing this as a different kind of log entry.
The reason for deferring the change is so that we can keep the naive
logging implementation more accurate without additional complexity for
reading the log. The accuracy is gained in effectively interpreting call
stacks like:
A()
B()
C()
Which when tail-call merged will end up not having any exit entries for
A() nor B(), but effectively in turn can be reasoned about as:
A()
B()
C()
Although we lose the fact that A() had called B() then had called C()
with the naive approach, a later iteration that adds the explicit tail
call entries would be a change in the log format and thus necessitate a
version change for the header. We can do this later to have a chance at
releasing some tools (in D21987) that are able to handle the naive log
format, then support higher version numbers of the log format too.
Reviewers: echristo, kcc, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, dberris
Differential Revision: https://reviews.llvm.org/D23988
llvm-svn: 284178
This patch extends __sanitizer_finish_switch_fiber method to optionally return previous stack base and size.
This solves the problem of coroutines/fibers library not knowing the original stack context from which the library is used. It's incorrect to assume that such context is always the default stack of current thread (e.g. one such library may be used from a fiber/coroutine created by another library). Bulding a separate stack tracking mechanism would not only duplicate AsanThread, but also require each coroutines/fibers library to integrate with it.
Author: Andrii Grynenko (andriigrynenko)
Reviewed in: https://reviews.llvm.org/D24628
llvm-svn: 282582
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
This addresses some comments from D21612, which contains the following changes:
- Update __xray_patch() and __xray_unpatch() API documentation to not imply asynchrony.
- Introduce a scope cleanup mechanism to make sure we can roll-back changes to the XRayPatching global atomic.
- Introduce a few more comments for potential extension points for other platforms (for the implementation details of patching and un-patching).
Reviewers: eugenis, rnk, kcc, echristo, majnemer
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D22911
llvm-svn: 277124
Summary:
This is a fixed-up version of D21612, to address failure identified post-commit.
Original commit description:
This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting).
Fixes include:
- Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built
- Including <cstddef> to fix std::size_t issue
Reviewers: kcc, rnk, echristo
Subscribers: mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D22611
llvm-svn: 276251
and also the follow-up "[xray] Only build xray on Linux for now"
Two build errors were reported on the llvm-commits list:
[ 88%] Building CXX object lib/xray/CMakeFiles/clang_rt.xray-x86_64.dir/xray_flags.cc.o
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/lib/xray/xray_init.cc:23:10: fatal error: 'llvm/Support/ELF.h' file not found
#include "llvm/Support/ELF.h"
^
and
In file included from /w/src/llvm.org/projects/compiler-rt/lib/xray/xray_interface.cc:16:
/w/src/llvm.org/projects/compiler-rt/lib/xray/xray_interface_internal.h:36:8: error:
no type named 'size_t' in namespace 'std'
std::size_t Entries;
~~~~~^
llvm-svn: 276186
Summary:
This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting).
Depends on D19904
Reviewers: echristo, kcc, rnk
Subscribers: rnk, mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D21612
llvm-svn: 276117
Adds a new esan public interface routine __esan_get_sample_count() and uses
it to ensure that tests of sampling receive the minimum number of samples.
llvm-svn: 275948
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
Summary:
Adds a new public interface routine __esan_report() which can be used to
request profiling results prior to abnormal termination (e.g., for a server
process killed by its parent where the normal exit does not allow for
normal result reporting).
Implements this for the working-set tool. The cache frag tool is left
unimplemented as it requires missing iteration capabilities.
Adds a new test.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D22098
llvm-svn: 274964
This patch adds the __sanitizer_start_switch_fiber and
__sanitizer_finish_switch_fiber methods inspired from what can be found here
2ea64dd249 .
These methods are needed when the compiled software needs to implement
coroutines, fibers or the like. Without a way to annotate them, when the program
jumps to a stack that is not the thread stack, __asan_handle_no_return shows a
warning about that, and the fake stack mechanism may free fake frames that are
still in use.
Author: blastrock (Philippe Daouadi)
Reviewed in http://reviews.llvm.org/D20913
llvm-svn: 273260
Summary:
Merge "exitcode" flag from ASan, LSan, TSan and "exit_code" from MSan
into one entity. Additionally, make sure sanitizer_common now uses the
value of common_flags()->exitcode when dying on error, so that this
flag will automatically work for other sanitizers (UBSan and DFSan) as
well.
User-visible changes:
* "exit_code" MSan runtime flag is now deprecated. If explicitly
specified, this flag will take precedence over "exitcode".
The users are encouraged to migrate to the new version.
* __asan_set_error_exit_code() and __msan_set_exit_code() functions
are removed. With few exceptions, we don't support changing runtime
flags during program execution - we can't make them thread-safe.
The users should use __sanitizer_set_death_callback()
that would call _exit() with proper exit code instead.
* Plugin tools (LSan and UBSan) now inherit the exit code of the parent
tool. In particular, this means that ASan would now crash the program
with exit code "1" instead of "23" if it detects leaks.
Reviewers: kcc, eugenis
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D12120
llvm-svn: 245734
Summary:
Add a weak hook to be called from dfsan's custom memcmp.
The primary user will be lib/Fuzzer.
If this works well we'll add more hooks (strcmp, etc).
Test Plan: Will be covered by lib/Fuzzer tests.
Reviewers: pcc
Reviewed By: pcc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9541
llvm-svn: 236679
Summary:
Add an interface function which can be used to periodically trigger
leak detection in a long-running process.
NB: The meaning of the kIgnored tag has been changed to allow easy clean-up
between subsequent leak checks. Previously, this tag was applied to explicitly
ignored (i.e. with __lsan_disable() or __lsan_ignore_object()) chunks *and* any
chunks only reachable from those. With this change, it's only applied to
explicitly ignored chunks.
Reviewers: samsonov
Reviewed By: samsonov
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9159
llvm-svn: 235728
Introduce -mllvm -sanitizer-coverage-8bit-counters=1
which adds imprecise thread-unfriendly 8-bit coverage counters.
The run-time library maps these 8-bit counters to 8-bit bitsets in the same way
AFL (http://lcamtuf.coredump.cx/afl/technical_details.txt) does:
counter values are divided into 8 ranges and based on the counter
value one of the bits in the bitset is set.
The AFL ranges are used here: 1, 2, 3, 4-7, 8-15, 16-31, 32-127, 128+.
These counters provide a search heuristic for single-threaded
coverage-guided fuzzers, we do not expect them to be useful for other purposes.
Depending on the value of -fsanitize-coverage=[123] flag,
these counters will be added to the function entry blocks (=1),
every basic block (=2), or every edge (=3).
Use these counters as an optional search heuristic in the Fuzzer library.
Add a test where this heuristic is critical.
llvm-svn: 231166
Catherine Moore