Summary:
If test is decorated with expectedFlakey*, run teardown and setup before retry
Don't run retry if the test is already decorated with xfail or skip
Test Plan:
Mark TestMultithreaded.test_sb_api_listener_event_process_state as expectedFlakey
Run ./dotest.py -p TestMultithreaded.py -A x86_64 -C gcc4.9.2
Reviewers: vharron, tberghammer
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D10830
llvm-svn: 241202
SUMMARY
Flakey tests get two chances to pass
Also, switched a bunch of tests to use new decorator.
TEST PLAN
Add one of these decorators to a test
Edit a test to pass on the first invocation, confirm test appears as pass
Edit a test to pass on the first invocation, pass on the second, confirm test appears as xfail
Edit a test to fail on two consecutive runs, confirm test appears in results as fail/error
Differential Revision: http://reviews.llvm.org/D10721
llvm-svn: 240789
Summary:
Currently, the local-only path fails about 50% of the tests, which means that: a) nobody is using
it; and b) the remote debugging path is much more stable. This commit removes the local-only
linux debugging code (ProcessLinux) and makes remote-loopback the only way to debug local
applications (the same architecture as OSX). The ProcessPOSIX code is moved to the FreeBSD
directory, which is now the only user of this class. Hopefully, FreeBSD will soon move to the new
architecture as well and then this code can be removed completely.
Test Plan: Test suite passes via remote stub.
Reviewers: emaste, vharron, ovyalov, clayborg
Subscribers: tberghammer, emaste, lldb-commits
Differential Revision: http://reviews.llvm.org/D10661
llvm-svn: 240543
Summary:
This change adds all the necessary infrastructure required to
selectively enable and make TestLoadUnload work for Android. One test,
which tests the module search paths, is still kept disabled for remote
as search paths (because of module caching) are local anyway.
Reviewers: tberghammer
Reviewed By: tberghammer
Subscribers: emaste, lldb-commits, tberghammer
Differential Revision: http://reviews.llvm.org/D10469
llvm-svn: 239965
Summary: Opened files on Windows cannot be modified, so this test doesn't make sense.
Reviewers: ovyalov, zturner, flackr, clayborg
Reviewed By: clayborg
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D10295
llvm-svn: 239359
Summary: Updated `append_to_remote_wd` to work for both remote and local.
Reviewers: clayborg, ovyalov
Reviewed By: ovyalov
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10288
llvm-svn: 239203
Summary:
This change also adds the infrastructure required to specify the API
levels for which tests should be skipped.
Reviewers: chying, labath
Reviewed By: labath
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10282
llvm-svn: 239183
Summary:
urlparse.ParseResult.hostname has only lowercase characters even if the
input URL had uppercase characters. Since Android device IDs can have
uppercase characters as well, use urlparse.ParseResult.netloc instead
and extract the device ID from it.
This change also improves the error message when lookup of the Android
device's API fails.
Reviewers: chaoren
Reviewed By: chaoren
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10278
llvm-svn: 239173
Summary:
This change adds the infrastructure to mark tests as xfail for specific
Android API levels.
Test Plan: dotest.py TestChangeProcessGroup on an Android API 16 device.
Reviewers: chying, labath, chaoren
Reviewed By: labath, chaoren
Subscribers: tberghammer, lldb-commits
Differential Revision: http://reviews.llvm.org/D10261
llvm-svn: 239126
Summary:
Before:
AssertionError: False is not True : Process is launched successfully
After:
AssertionError: False is not True : Command 'run a.out' failed.
>>> error: invalid target, create a target using the 'target create' command
>>> Process could not be launched successfully
Reviewers: clayborg
Reviewed By: clayborg
Subscribers: lldb-commits, vharron
Differential Revision: http://reviews.llvm.org/D9948
llvm-svn: 238363
On non-Windows platforms, os.rename() will silently replace the
destination file if it already exists. On Windows, it doesn't do
this, and the filesystem has no mechanism to simulate the same type
of atomic rename operation. So on Windows, delete the file first
before calling os.rename().
llvm-svn: 238239
`lock` depends on `fcntl`, which doesn't exist on Windows. Until
someone implements an equivalent locking mechanism on Windows, we
can't have lock imported globally.
llvm-svn: 237946
SUMMARY
dosep.py starts lots and lots of dotest instances.
This option helps you find if two (or more) dotest instances are using
the same directory at the same time.
Enable it to cause test failures and stderr messages if dotest
instances try to run in the same directory simultaneously.
It is disabled by default because it litters the test directories with
".dirlock" files
TEST PLAN
Set lldbtest.debug_confirm_directory_exclusivity = True
run ./dosep.py
Differential Revision: http://reviews.llvm.org/D9868
llvm-svn: 237935
That way, if the test gets killed (by a dosep timeout) we get to see the session
trace
Test Plan:
Run dotest.py
Kill it while it's running, check the session dir for Test* files
Differential Revision: http://reviews.llvm.org/D9845
llvm-svn: 237926
The lldb executable was referenced through the code by 7 different
(effectively) global variables.
global lldbExecutablePath
global lldbExecutable
os.environ['LLDB_EXEC']
os.environ['LLDB_TEST']
dotest.lldbExec
dotest.lldbHere
lldbtest.lldbExec
This change uses one global variable lldbtest_config.lldbExec to
replace them all.
Differential Revision: http://reviews.llvm.org/D9817
llvm-svn: 237600
When compiling programs for the test suite we currently choose which stdlib to
use based on the host platform, but should be basing this on the target
platform.
Test Plan: ./dotest.py $DOTEST_OPTS -t -p TestThreadExit.py
This test previously failed mac->linux most of the time due to using the mac
host's atomic declaration.
Differential Revision: http://reviews.llvm.org/D9797
llvm-svn: 237466
TestPluginCommands.py attempts to build a library which links against the host
built lldb library. This will only work if the remote is compatible with
binaries produced by the host.
Test Plan:
./dotest.py $DOTEST_OPTS -v -t -p TestPluginCommands.py
Test is skipped if remote platform is incompatible with host platform (i.e. mac
-> linux).
Differential Revision: http://reviews.llvm.org/D9770
llvm-svn: 237444
Select expected remote libc++ library name based on remote platform. This is
used to verify libc++ is in the loaded image list by various tests which use it.
Test Plan:
Passes the following tests mac -> linux with this patch:
TestDataFormatterLibccIterator.py
TestDataFormatterLibccMap.py
TestDataFormatterLibccMultiMap.py
TestDataFormatterLibcxxMultiSet.py
TestDataFormatterLibcxxSet.py
TestDataFormatterUnordered.py
Differential Revision: http://reviews.llvm.org/D9759
llvm-svn: 237288
This is a very minor change.
Just reshuffle the elements of the log filename so that the test name
comes before the arch/compiler
Before:
Failure-x86_64-clang-TestCModules.CModulesTestCase.test_expr_with_dwarf.log
After:
Failure-TestCModules.CModulesTestCase.test_expr_with_dwarf-x86_64-clang.log
This way, results are grouped by test name, not by arch. I think this
is much more useful.
Differential Revision: http://reviews.llvm.org/D9648
llvm-svn: 237076
Summary:
Since we don't yet have remote windows debugging, it should be safe to assume
that the remote target uses unix path separators.
Reviewers: ovyalov, zturner, clayborg, vharron
Reviewed By: vharron
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D9633
llvm-svn: 237006
Summary:
New dotest options that allow arbitrary log channels and
categories to be enabled. Also enables logging for locally run
debug servers.
Log messages are separated into separate files per test case.
(this makes it possible to log in dosep runs)
These new log files are stored side-by-side with trace files in the
session directory.
These files are deleted by default if the test run is successful.
If --log-success is specified, even successful logs are retained.
--log-success is useful for creating reference log files.
Test Plan:
add '--channel "lldb all" --channel "gdb-remote packets" --log-success'
to your dotest options
Tested on OSX and Linux
Differential Revision: http://reviews.llvm.org/D9594
llvm-svn: 236956
Refactored TestInitializerList to not be an inline test.
Refactored Makefiles to use USE_LIBCPP instead of adding FLAGS directly
Fixed copy/paste error in TestDataFormatterUnordered class name
Differenttial Revision: http://reviews.llvm.org/D9426
llvm-svn: 236401
These tests link against host lldb API. Compiler's target triple
must match liblldb triple. Instead of naively skipping i386, I added
a check of the liblldb arch against the compiler target arch.
This is useful for 32 bit API builds (planned for Windows)
Since remote is disabled, we can assume the os is the same.
Also, removed skipIfLinuxClang because it's passing
llvm-svn: 236396
Summary:
- add decorator functions to xfail and skip test on specific os, architecture and version of comipler
- xfail failing test with gcc-4.9.2 on linux
- add one usage of skipIf function
Test Plan:
Run tests with different archs, and version of compilers to verify decorator function working as expected
Run tests with gcc-4.9.2 and no failure reported
Reviewers: sivachandra, ovyalov, vharron, chaoren
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D8851
llvm-svn: 235368
Uses the target platform to determine the platform context to use. This fixes an
early failure in some tests when running on a different remote target.
Test Plan:
./dotest.py $DOTEST_OPTS -t -p TestGlobalVariables.py|TestSharedLib.py
Running on macosx -> linux, this fails at a later stage in the test - rather
than in registerSharedLibrariesWithTarget looking for a library without the
correct "lib" prefix and ".so" suffix.
Differential Revision: http://reviews.llvm.org/D9047
llvm-svn: 235164
Summary:
This patch fixes the following bug: https://llvm.org/bugs/show_bug.cgi?id=23181
For some reason some lldb-server tests should be kicked using SIGHUP and SIGINT before termination, otherwise it will leave a zombie process.
I think the reason is that the lldb-server will terminate a slave process if it gets the SIGHUP/SIGINT and if so it should be fixed in lldb-server.
The solution is to terminate process like it does the pexpect (including the delayafterterminate interval).
Also this patch reverts the following commits:
* r234549 - Skip lldb-server tests according to bug 23181
* r234765 - Skip a few tests on OS X according to the bug 23181
* r234803 - Skip the TestGdbRemoteRegisterState.test_grp_register_save_restore_works_no_suffix_debugserver_dsym test on OS X according to the bug 23181
Test Plan:
```
$ ps | grep Z | grep a.out | wc -l
447
$ ./dosep.py --options "-v --executable $INSTALLDIR/bin/lldb"
[...]
$ ps | grep Z | grep a.out | wc -l
447
```
Reviewers: jasonmolenda, clayborg, tberghammer
Reviewed By: clayborg
Subscribers: lldb-commits, dawn, tberghammer, clayborg, jasonmolenda
Differential Revision: http://reviews.llvm.org/D9019
llvm-svn: 235008
The decorators to skip if or unless certain platforms relied on calling
skipTest, which only worked if they were decorating individual test cases.
However for decorating entire classes it needs different behavior. This behavior
is already encapsulated in unittest2.skipIf and unittest2.skipUnless so this
modifies the platform decorators to use these unittest skip decorators.
Test Plan:
./dotest.py -t -p TestObjCMethods2.py
See skipped tests on linux, and see tests run on macosx.
Differential Revision: http://reviews.llvm.org/D8903
llvm-svn: 234514
expectedFailureLLGS has an early return false if the platform is not linux
except it should be checking the target platform on which the server is
running for remote tests.
Test Plan:
Verify expected failure when running from mac to remote linux llgs:
TestBreakAfterJoin.py, TestCreateDuringStep.py, TestExitDuringBreak.py,
TestProcessLaunch.py, TestThreadStates.py
Differential Revision: http://reviews.llvm.org/D8869
llvm-svn: 234492
Summary:
Updated test to reflect that Linux and Darwin behave the same now.
Removed @expectedFailureLinux for passing tests.
Test Plan: run tests
Reviewers: clayborg, sivachandra
Subscribers: lldb-commits
Differential Revision: http://reviews.llvm.org/D8678
llvm-svn: 233989
There was a lot of code that was checking "if self.getPlatform() == 'darwin'" which is not correct. I fixed this by adding a:
lldbtest.platformIsDarwin()
which returns true if the current platform's OS is "macosx", "ios" or "darwin". These three valid darwin are now returned by a static function:
lldbtest.getDarwinOSTriples()
Fixed up all places that has 'if self.getPlatform() == "darwin":' with "if self.platformIsDarwin()" and all instances of 'if self.getPlatform() != "darwin":' with "if not self.platformIsDarwin()". I also fixed some darwin decorator functions to do the right thing as well.
llvm-svn: 233933
On FreeBSD LLDB's triple ends up as e.g. "x86_64-unknown-freebsd10.1"
but getPlatform() consumers expect just the name with no version
number.
llvm-svn: 233705
Adds @skipIfPlatform and @skipUnlessPlatform decorators which will skip if /
unless the target platform is in the provided platform list.
Test Plan:
ninja check-lldb shows no regressions.
When running cross platform, tests which cannot run on the target platform are
skipped.
Differential Revision: http://reviews.llvm.org/D8665
llvm-svn: 233547
Most expected OS failures or skipped tests are about the target platform on
which the test binary is being run, not the host platform launching the tests.
This changes expectedFailureOS and skipIfPlatform to check against the remote
platform when running remote tests.
Test Plan:
Run ./do_sep.py test suite against a remote target on a different platform and
verify that tests which should be excluded on the remote platform are excluded.
Differential Revision: http://reviews.llvm.org/D8611
llvm-svn: 233311
tear down hooks run as part of Base.tearDown(). Some of these
hooks rely on accessing the debugger instance. So although it
looks awkward, we need to call "del self.dbg" after calling
Base.tearDown().
llvm-svn: 233306
Tests derive from TestBase, which derives from Base. In the
test setUp() methods, we always call TestBase.setUp() first and
then call implementation-specific setup. Tear down needs to do
the reverse.
This was causing over 20 failures on Windows, and was the culprit
behind about 80% of the files not being cleaned up after test run.
TestBase.tearDown() is responsible for deleting all targets created
during the test run and without this step, on Windows files will
be locked and cannot be deleted. But TestBase.tearDown() was
calling Base.tearDown() before its own cleanup (i.e. deleting the
targets) and in some cases one of the teardown hooks would be to
call make clean. So make clean would be run before the targets
had been deleted, and fail to remove the files, and subsequently
result in a failed test as well.
llvm-svn: 233284
Abstracted away some POSIX-isms that caused MAKE to issue invalid
commands on Windows. Added a new force-include for the test
programs so that we can use platform-specific macros.
TestSharedLib now builds and cleans up on Windows, though the test
still fails some of the expectations.
Differential Revision: http://reviews.llvm.org/D8277
Patch by: Adrian McCarthy
Issue Tracker: http://llvm.org/pr21727
llvm-svn: 232220
Android have more file descriptor opened by the shell what are inherited
to the process (different ones on device and on emulator).
Differential revision: http://reviews.llvm.org/D8299
llvm-svn: 232150
Sean Callanan