Summary:
This patch allows the use of LIT's ShTest format in the libc++ test suite. ShTests have the suffix '.sh.cpp'. It also introduces a series of other changes. These changes are:
- More functionality including parsing test metadata has been moved into LIT.
- LibcxxTestFormat now supports multi-part suffixes.
- the `CXXCompiler` functionality has been used to shrink the size of LibcxxTestFormat.
- The recursive loading of the site config has been turned into `libcxx.test.config.loadSiteConfig` so it can be used with libc++abi.
- Temporary files are now created in the build directory of libc++. This follows how it is down in ShTest.
- `not.py` was added as a utility executable that mirrors the functionality of LLVM's `not` executable.
- The first ShTest test was added under test/libcxx/double_include.sh.cpp
Reviewers: jroelofs, danalbert
Reviewed By: danalbert
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D7073
llvm-svn: 226844
Summary:
I've moved the bulk of `lit.cfg` into `test/libcxx/testconfig.py` and
`test/libcxx/testformat.py`. All that remains in `lit.cfg` is the
logic to discover lit.site.cfg if lit.cfg was run directly, and the
logic for loading configuration variants.
The configuration variant flow has changed with this patch. Rather
than instantiating an object of type `<VARIANT>Configuration`, we now
instatiate an object of type `Configuration` that was loaded from the
module `<VARIANT>.testconfig.py`.
This has to be done on a per-project basis rather than in LIT itself
because LIT doesn't actually know where the real test directory is,
only where the site configuration is (which is usually in the output
directory). It's simple enough to do though, so it's fine to require
each project to do it themselves.
I also cleaned up all the pylint issues while I was here, which was
mostly just a matter of fixing long lines.
Reviewers: mclow.lists, jroelofs, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D6881
llvm-svn: 225532
Summary:
In order to fully replace the testit script we need to update LIT so it provides the same functionality.
This patch adds a number of different configuration options to LIT to do that. It also adds documentation for all of the command line parameters that LIT supports.
Generic options added:
- `libcxx_headers`
- `libcxx_library`
- `compile_flags`
Generic options modified:
- `link_flags`: Changed from overriding the default args to adding extra args instead (to match compile flags)
- `use_sanitizer`: Renamed from `llvm_use_sanitizer`
Please see the added documentation for more information about the switches. As for the actual documentation I'm not sure if it should be kept in libc++ forever since it adds an undue maintenance burden, but I think it should be added for the time being while the changes are new. I'm verify unskilled with HTML so if the documentation needs any changes please let me know.
Hopefully this will kill testit.
Reviewers: jroelofs, mclow.lists, danalbert
Reviewed By: danalbert
Subscribers: alexfh, cfe-commits
Differential Revision: http://reviews.llvm.org/D5877
llvm-svn: 224728
Summary:
Currently to run tests in tree you need to symlink the lit.site.cfg file generated by the cmake build into the source tree, and teach your VCS to ignore it.
This allows the user to specify where to find the lit.site.cfg file two different ways:
* lit_site_config lit parameter
* LIT_SITE_CONFIG enviroment variable.
example usage:
```
lit -sv --param=libcxx_site_config=path/to/libcxx-build/test/lit.site.cfg path/to/tests
```
Or
```
export LIBCXX_SITE_CONFIG=path/to/libcxx-build/test/lit.site.cfg
lit -sv path/to/tests
```
The command line parameter will override the environment variable.
If neither options are present a warning is issued and the `lit.cfg` file is loaded directly.
Reviewers: mclow.lists, jroelofs, danalbert
Reviewed By: danalbert
Subscribers: ddunbar, cfe-commits
Differential Revision: http://reviews.llvm.org/D6255
llvm-svn: 224671
Summary:
In order to get the bots running quicker I would like to be able to use ccache
with the test suite. This patch adds support for running the test suite using
ccache. To use ccache pass `--param=use_ccache=true` when running the test suite.
ccache will not cache any command that invokes ld, so the build step needs to be
split into two separate compile commands. The cost of splitting the build step
into two parts when not using ccache seems to be minimal. On my machine I saw a
difference of ~5 seconds on a 5 minute test suite run.
A full test suite run with ccache generates about 250MB of cached data.
I recorded the following times for running the test suite in the following configurations:
- no ccache: 340s
- initial ccache run: 380s
- rerun with ccache (no changes): 53s.
- rerun with ccache (<string> changed): 80s
- rerun with ccache (<cmath> changed): 169s
- rerun with ccache (<valarray> changed): 69s
Reviewers: mclow.lists, jroelofs, danalbert
Reviewed By: jroelofs
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D6647
llvm-svn: 224603
Summary:
This patch probes the cxx compiler used during testing by getting it to dump its predefined macros. Based on the value of these macros the compiler name and compiler name + version are added to the available features.
There are three compiler names:
- `clang`
- `apple-clang`
- `gcc`.
The available features added are equivalent to:
- `'%s' % compiler_name`
- `'%s-%s.%s' % compiler_name, major_version, minor_version`
This information can be used to XFAIL tests on different compilers / versions.
Reviewers: mclow.lists, danalbert, jroelofs
Reviewed By: jroelofs
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D6399
llvm-svn: 223593
Currently hacks must be used in to configure and build libc++ without threads
when using CMake. This patch adds CMake options to enable/disable building with
threads and a monotonic clock.
This patch also propagates the configuration information to lit so the tests
are properly configured as well.
llvm-svn: 223591
Summary:
I finally got around to merging the many, many changes to lit.cfg into
Android's libc++. This patch makes it simpler to actually use a custom
configuration and test format.
First, I've factored out _build, _run, and _clean methods from
_execute_test, since these are the likely parts that will need to be
overridden. This is likely a first step in the work jroelofs has been
doing with improving cross-compiling test execution.
Second, I've added a `configuration_variant` to the config. This
entry, if present, is a string that forms the prefix of the class that
is to be used to configure the test runner. For example, Android sets
`config.configuration_variant = 'Android'`, and this causes an object
of type `AndroidConfiguration` to be constructed.
As an example of how this will be used, see:
https://android-review.googlesource.com/#/c/116022/
Reviewers: jroelofs, mclow.lists, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D6373
llvm-svn: 222698
For targets that end it `redhat-linux` and `suse-linux` manually add the `-gnu`
section of the target since `linux-gnu` is needed in the testsuite.
This patch also moves the removal of minor and patchlevel numbers from OSX
triples to be handled when deducing the triple instead of when adding available
features.
llvm-svn: 220724
Although the current method is valid up till python 3.3 (which is not supported)
this seems to be a clearer way of checking for linux and moves the tests towards
python 3 compatibility.
llvm-svn: 220534
Clang 3.6 no longer links the sanitizer runtime library dependancies when
-nodefaultlibs is used. This patch manually links in a missing dependancy.
llvm-svn: 220463
Summary:
This is the second attempt at allowing for the use of libraries that the linker cannot find. The first attempt used `CMAKE_LIBRARY_PATH` and `find_library` to select which ABI library should be used. There were a number of problems with this approach:
- `find_library` didn't work with cmake targets (ie in-tree libcxxabi build)
- It wasn't always possible to determine where `find_library` actually found your library.
- `target_link_libraries` inserted the path of the ABI library into libc++'s RPATH when `find_library` was used.
- Linking libc++ and it's ABI library is a special case. It's a lot easier to keep it simple.
After discussion with @cbergstrum a new approach was decided upon.
This patch achieve the same ends by simply using `LIBCXX_CXX_ABI_LIBRARY_PATH` to specify where to find the library (if the linker won't find it). When this variable is defined it is simply added as a library search path when linking libc++. It is a lot easier to duplicate this behavior in LIT. It also prevents libc++ from being linked with an RPATH.
Reviewers: mclow.lists, cbergstrom, chandlerc, danalbert
Reviewed By: chandlerc, danalbert
Subscribers: chandlerc, cfe-commits
Differential Revision: http://reviews.llvm.org/D5860
llvm-svn: 220157
Summary:
This patch adds support for building/testing libc++ with an ABI library that the linker would not normally find.
- `CMAKE_LIBRARY_PATH` is used to specify the list of search directories.
- The ABI library is now found using `find_library` instead of assuming its along the linker's search path.
- `CMAKE_LIBRARY_PATH` is passed to our LIT config as `library_paths`.
- For each path in `library_paths` the following flags are added `-L<path> -Wl,-rpath -Wl,<path>`
Some changes in existing behavior were also added:
- `target_link_libraries` is now passed the ABI library file instead of the library name. Ex `target_link_libraries(cxx "/usr/lib/libc++abi.so")` vs `target_link_libraries(cxx "c++abi")`.
- `-Wl,-rpath -Wl,<path>` is now used on OSX to link to libc++ instead of env['DYLD_LIBRARY_PATH'] if `use_system_lib=False`.
Reviewers: mclow.lists, danalbert, EricWF
Reviewed By: EricWF
Subscribers: emaste, cfe-commits
Differential Revision: http://reviews.llvm.org/D5038
llvm-svn: 220118
LLVM_USE_SANITIZER=Undefined support was added to the LLVM CMake configuration.
Update libc++'s handling of LLVM_USE_SANITIZER to support this as well.
llvm-svn: 219987
This modifies the use_clang_verify parameter I added in r217009 to
only apply to tests that specifically ask for it via // USE_VERIFY.
This allows us to incrementally convert tests, but start enjoying the
benefits right away.
Suggested by Eric Fiselier in code review.
llvm-svn: 217017
Currently, failure tests work by checking that compilation exits 1.
This can lead to tests that fail for the wrong reason, so it'd be
preferable to convert them to check for specific errors.
This adds use_clang_verify parameter that runs failure tests using
clang's -verify flag. I'll convert some tests in subsequent commits,
and once all of the tests are converted we should key this on whether
cxx_under_test is clang.
I've also converted one of the unique.ptr tests, since it's the one
that motivated the idea of using clang -verify when possible in the
review of r216317.
llvm-svn: 217009
The purely imperative format of the configuration performed in lit.cfg
was making merge conflicts with changes I have for Android an
unbelievable pain in the ass. I've moved all of the configuration into a
Configuration class, with each piece of configuration happening in a
different method. This way I can avoid merge conflicts, and any new
features that get added (as have been done with the sanitizers, the -std
flag, etc.) can be easily applied to Android as well.
Reviewers: mclow.lists, EricWF
Reviewed By: EricWF
Differential Revision: http://reviews.llvm.org/D4952
llvm-svn: 216196
Summary:
This patch adds support for // UNSUPPORTED: feature. If an excluded feature is found in the list of available features then the test is marked unsupported.
I hope to use this to mark test unsupported if the fail with msan on asan. As well as tests that fail in particular standard modes.
Reviewers: mclow.lists, danalbert
Reviewed By: danalbert
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D4950
llvm-svn: 215883
Summary:
This patch adds support for LLVM_USE_SANITIZER when being built in-tree and standalone.
This patch does the following things:
1. define the LLVM_USE_SANITIZER option to "" when being built standalone. This also helps show we support it.
2. Translate LLVM_USE_SANITIZER when standalone in a very similar way done in llvm/cmake/HandleLLVMOptions.cmake.
3. Add config.llvm_use_sanitizer to lit.site.cfg.in
4. Add code to translate config.llvm_use_sanitizer's value into the needed compile flags in lit.cfg.
Currently lit.cfg assumes that that the compiler supports '-fno-omit-frame-pointer' while CMakeLists.txt actually checks to see if its supported. We could pass this information to lit but I'm not sure its needed.
Reviewers: mclow.lists, danalbert
Reviewed By: danalbert
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D4949
llvm-svn: 215872
Summary:
This patch does two things:
CMake Update:
- Add compiler flag checks for -std=c++11 and -std=c++1y and remove check for -std=c++0x.
- Add configuration option LIBCXX_ENABLE_CXX1Y to prevent/allow -std=c++1y from being chosen as the std version. LIBCXX_ENABLE_CXX1Y is set to OFF by default.
- if LIBCXX_ENABLE_CXX1Y is enabled then set LIBCXX_STD_VERSION to c++1y and fail if the compiler does not support -std=c++1y
- If c++1y is not enabled then use c++11 and fail if the compiler does not support c++11.
Lit Update:
- Update lit.site.cfg.in to capture LIBCXX_STD_VERSION information as config.std.
- Remove mentions of has_cxx0X configuration option.
- Check for `--param std=X' passed to lit on the command line.
- Choose the std for the tests either from command line parameter or (if it doesn't exist) the lit.site.cfg.
Reviewers: mclow.lists, danalbert
Reviewed By: danalbert
Subscribers: emaste, rnk, ajwong, danalbert, cfe-commits
Differential Revision: http://reviews.llvm.org/D4329
llvm-svn: 215802
Add initial support for using LIT to run the tests on FreeBSD.
More work may need to be done to add support for FreeBSD but this is a
good initial step.
llvm-svn: 215742
Eric Fiselier