Assign the test method name to self.testMethodName. This can be useful for the
test directory (see test/types for a good example) which houses a bunch of executables
compiled from different source files. The default build action is to create a.out as
the binary executable, which can confuse the module cacheing mechanism and result in
the debugger getting a stale image as the target to be debugged, and chaos ensues.
o AbstractBase.py, TestThreadAPI.py:
Use self.testMethodName to our advantage.
o TestLoadUnload.py:
Add expected failure marker to test case test_modules_search_paths().
llvm-svn: 133768
if not already specified by the test driver (via ./dotest -w). Remove the AbstractBase.setUp()
method definition when/if we find out the cause of the failures if no delays are inserted
between these test cases.
llvm-svn: 133495
remove the self.runStarted attribute since the automatic shutdown of processes associated
with the targets are now performed automatically.
llvm-svn: 133092
bool SBDebugger::DeleteTarget(lldb::SBTarget &target);
which is used in the test tearDown() phase to cleanup the debugger's target list
so that it won't grow larger and larger as test cases are executed. This is also
a good opportunity to get rid of the arcane requirement that test cases exercising
the Python API must assign the process object to self.process so that it gets
shutdown gracefully. Instead, the shutdown of the process associated with each
target is now being now automatically.
Also get rid of an API from SBTarget class:
SBTarget::DeleteTargetFromList(lldb_private::TargetList *list);
llvm-svn: 133091
Change one test sequence to detect the '** End Stop Hooks **' marker emitted by the
stop hooks mechanism and check for whether the 'expr ptr' stop-hook has been run.
Also, change the TestBase.tearDown() to wait for 2 seocnds before forcefully kill
the pexpect-spawned child lldb process.
llvm-svn: 130767
and TestAliases.py. Pass the keyword argument 'check=False' to:
self.runCmd("script my.date()", check=False)
since we want to restore sys.stdout no matter what the outcome of the runCmd is.
llvm-svn: 129949
currently in trace mode (-t to dotest.py), i.e., tracing the lldb command execution.
Change TestInferiorCrashing.inferior_crashing_python(self) to check this flag in
order to print the stack trace of the inferior thread.
llvm-svn: 129785
rdar://problem/9173060 lldb hangs while running unique-types
disappears if running with clang version >= 3. Modify the TestUniqueTypes.py
to detect if we are running with clang version < 3 and, if true, skip the test.
Update the lldbtest.system() function to return a tuple of (stdoutdata, stderrdata)
since we need the stderr data from "clang -v" command. Modify existing clients of
lldbtest.system() to now use, for example:
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
- go = system("./a.out", sender=self)
+ go = system("./a.out", sender=self)[0]
# This golden list contains a list of (variable, value) pairs extracted
# from the golden output.
gl = []
And add two utility functions to lldbutil.py.
llvm-svn: 128162
This uses pexpect module to spawn a 'lldb' program and uses pseudo-TTY to talk to
the child application.
The test cases test setting breakpoints, adding a stop-hook with line range, and
verifies that when the inferior stops, the stop-hook will fire off when it is
within range and will not fire off when it is out of range.
llvm-svn: 127519
"process launch" or "run" interpreter command. Let's do the sleep only if
the process launch failed. This saves about 135 seconds from the whole test
suite run time.
llvm-svn: 123806
get the argument values of the call stacks when stopped on the breakpoint.
Radar has been filed for the expected failures:
test failure: ./dotest.py -v -w -t -p TestFrames (argument values are wrong)
llvm-svn: 122460
ALso add a teardown hook to the LoadUnloadTestCase.test_dyld_library_path()
test case to have it restore the DYLD_LIBRARY_PATH environment variable.
llvm-svn: 121802
Add an attribute __python_api_test__ (set to True) to the @python_api_test decorated
test method to distinguish them from the lldb command line tests.
llvm-svn: 121500
Example:
@python_api_test
def test_evaluate_expression_python(self):
"""Test SBFrame.EvaluateExpression() API for evaluating an expression."""
...
The opposite of Python APIs only test is an lldb command line test, which sends
commands to the lldb command interpreter. Add a '-a' option to the test driver
to skip Python API only tests.
Modify TestExprs.py to mark a test as @python_api_test and remove an @expectedFailure
decorator as the bug has been fixed.
llvm-svn: 121442
Initial test case test_modify_source_file_while_debugging() in TestSourceManager.py
tests the caching mechanism of the source manager.
llvm-svn: 121389
as the args and the envs to the launched process.
o lldbtest.py:
Forgot to check in some assertion messages changes for lldbtest.py.
o dotest.py:
Also add "api" category to the default lldb log option list.
llvm-svn: 121220
test classes or test cases to be excludued from the test suite.
Check in an example blacklist file: blacklist.py:
"""
'blacklist' is a Python dictionary, it stores the mapping of a string describing
either a testclass or a testcase, i.e, testclass.testmethod, to the reason (a
string) it is blacklisted.
Following is an example which states that test class IntegerTypesExprTestCase
should be skipped because 'This test class crashed' and the test case
FoundationTestCase.test_data_type_and_expr_with_dsym should be skipped because
it is 'Temporarily disabled'.
blacklist = {'IntegerTypesExprTestCase': 'This test class crashed',
'FoundationTestCase.test_data_type_and_expr_with_dsym': 'Temporarily disabled'
}
"""
blacklist = {}
An example of invoking the test driver and specifying a blacklist file:
./dotest.py -b blacklist.py -v types
This runs the tests under 'types' directory but excludes the tests specified in
balcklist.py.
llvm-svn: 120620
the process status is indeed 'stopped' before invoking 'frame variable -t val'
command to inspect the frame variable. When I run the test suite, it sometimes
emits an output like:
runCmd: frame variable -t val
runCmd failed!
error: you must be stopped in a valid stack frame to view frame variables.
The expect() statement inserted is to make sure that the process is stopped and
we have a valid frame.
llvm-svn: 120318
Increase the delta between successive test cases, enabled with the '-w' option to
the test driver, from 0.5 to 1.0 second, to arrive at a cleaner state before the
next test case.
llvm-svn: 120315
output from clang and llvm-gcc compiled program; both generate the correct debug
info with respect to the typedef scoped inside a namespace.
Add a TestBase.getCompiler(self) method which returns the compiler in effect the
test suite is now running with. Subclasses (like TestNamespace) can use it to
distinguish among different compilers.
llvm-svn: 119445
invoke the test driver to rerun the very same test. Example output:
/Volumes/data/lldb/svn/trunk/test $ tail 2010-11-09-14_51_34/ExpectedFailure-TestSettings.SettingsCommandTestCase.test_set_output_path.log
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 136, in test_set_output_path
"'stdout.txt' exists due to target.process.output-path.")
AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.
To rerun this test, issue the following command from the 'test' directory:
./dotest.py -v -t -f SettingsCommandTestCase.test_set_output_path
llvm-svn: 118646
identification of the test failures/errors by human beings as well as automatic
processings.
The prefix which identifies the type can be: Error, Failure, or ExpectedFailure.
llvm-svn: 118315
local variable and an argument variable.
Add ValueTypeString() utility function into lldbutil.py which converts the enum
into a human readable string.
Modify TestBase.DebugSBValue() to also dump the value type of an SBValue object.
llvm-svn: 118197
idea to also dump the session information for expected failures in addition to
failures and errors (tests with unexpected exceptions).
Modify the LLDBTestResult class which derives from unittest2.TextTestResult to
intercept the addExpectedFailure() method in order to mark the test instance as
needing its session information dumped.
llvm-svn: 118185
to execute the debugger command. This will result in a more semantic assert message than just the
command failed message like before:
AssertionError: False is not True : Command 'thread list' returns successfully
And now, we will have:
AssertionError: False is not True : Process state is stopped due to breakpoint
llvm-svn: 117569
directory used to dump the session info for test failures/errors.
Example:
/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -s jason -v array_types
Session info for test errors or failures will go into directory jason
----------------------------------------------------------------------
Collected 4 tests
test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... FAIL
======================================================================
FAIL: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 27, in test_with_dwarf_and_run_command
self.array_types()
File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 62, in array_types
'stop reason = breakpoint'])
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 594, in expect
self.runCmd(str, trace = (True if trace else False), check = not error)
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 564, in runCmd
msg if msg else CMD_MSG(cmd, True))
AssertionError: False is not True : Command 'thread list' returns successfully
----------------------------------------------------------------------
Ran 4 tests in 3.086s
FAILED (failures=1)
/Volumes/data/lldb/svn/trunk/test $ ls jason
TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
/Volumes/data/lldb/svn/trunk/test $ head -10 jason/TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
Session info generated @ Thu Oct 21 09:54:15 2010
os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=NO']]
stdout: rm -rf "a.out" "a.out.dSYM" main.o main.d
cc -arch x86_64 -gdwarf-2 -O0 -c -o main.o main.c
cc -arch x86_64 -gdwarf-2 -O0 main.o -o "a.out"
stderr: None
retcode: 0
/Volumes/data/lldb/svn/trunk/test $
llvm-svn: 117028
parent call frame information. And comment out the check for stop reason for the
time being. The stop reason disappeared from the "thread backtrace" output for
breakpoint stop with condition.
llvm-svn: 116939
to be run during tearDown() to effect the restore action instead of executing it inline
during the test method, because the test may already fail and bailout before the inline
restore action.
Fix test_set_output_path() and pass_run_args_and_env_vars() to use this mechanism.
llvm-svn: 116881
now goes into a timestamp-specific directory instead of the previous .session-*
files.
[17:24:34] johnny:/Volumes/data/lldb/svn/trunk $ ls -l test/2010-10-18-16:56:12.935342
total 48
-rw-r--r-- 1 johnny admin 1695 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dsym_and_run_command.log
-rw-r--r-- 1 johnny admin 1652 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r-- 1 johnny admin 2967 Oct 18 16:56 TestBreakpointCommand.BreakpointCommandTestCase.test_with_dsym.log
-rw-r--r-- 1 johnny admin 1648 Oct 18 16:56 TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r-- 1 johnny admin 1665 Oct 18 16:56 TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase.test_with_dsym_and_python_api.log
-rw-r--r-- 1 johnny admin 3873 Oct 18 16:58 TestFloatTypesExpr.FloatTypesTestCase.test_float_types_with_dsym.log
[17:24:37] johnny:/Volumes/data/lldb/svn/trunk $
Also, the dumping happens when a test errored in additioned to when it failed.
llvm-svn: 116778
pass in a 'sender' arg to the buildDefault(), buildDsym(), buildDwarf(), and
cleanup() functions. The sender arg will be the test instance itself (i.e.,
an instance of TestBase). This is so that the relevant command execution can be
recorded in the TestBase.session object if sender is available.
The lldbtest.system() command has been modified to pop the 'sender' arg out of
the keyword arguments dictionary and use it as the test instance to facilitate
seesion recordings. An example is in test/types/AbstractBase.py:
def generic_type_tester(self, atoms, quotedDisplay=False):
"""Test that variables with basic types are displayed correctly."""
# First, capture the golden output emitted by the oracle, i.e., the
# series of printf statements.
go = system("./a.out", sender=self)
There are cases when sender is None. This is the case when the @classmethod is
involved in the use of these APIs. When this happens, there is no recording
into a session object, but printing on the sys.stderr is still honored if the
trace flag is ON.
An example is in test/settings/TestSettings.py:
@classmethod
def classCleanup(cls):
system(["/bin/sh", "-c", "rm -f output.txt"])
system(["/bin/sh", "-c", "rm -f stdout.txt"])
llvm-svn: 116648
breakpoint modify -c 'val == 3' 1
after:
breakpoint set -n c
which sets a breakpoint on function 'c'. The breakpoint should only stop if
expression 'val == 3' evaluates to true.
llvm-svn: 116607
session info after a test case failure, allowing more direct inspection of
debugger session which leads to the test failure.
For a simple usage scenario:
[18:06:26] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log
...
[18:14:43] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l .session-*
-rw-r--r-- 1 johnny admin 1359 Oct 14 18:06 .session-TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command
-rw-r--r-- 1 johnny admin 2054 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser
-rw-r--r-- 1 johnny admin 2055 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser
-rw-r--r-- 1 johnny admin 1351 Oct 14 17:57 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command
[18:14:51] johnny:/Volumes/data/lldb/svn/trunk/test $
The test case which failed will have its recorded session info dumped to a
.session-* file in the current working directory. For test suite using
relocated directory, expect to find the .session-* files there.
In this checkin, I also add @skip decorator to the two test methods in
test/foundation/TestObjCMethods.py as it looks like the test suite is
deadlocking when running the tests. More investigations are needed.
llvm-svn: 116552
and 'process handle'. The test suite would like to control the asynch/sync
execution of the interpreter during the middle of the test method, so the
CommandInterpreter::SetSynchronous(bool value) is modified to allow the mode to
be changed more than once.
In practice, it would be advisable to control the process and to set the
async/sync mode from a single thread, too.
llvm-svn: 116467
within a file. This is to be used within the test case to avoid hardcoded line number.
array_types/TestArrayTypes.py is modified first to use this pattern. Other test modules
to follow.
rdar://problem/8537816
Testsuite: don't set breakpoints by exact file & line number
llvm-svn: 116270
files to a different top level directory than those specified on the command line.
When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all. This allows for an easier
postmortem analysis of test failures.
Example:
./dotest.py -v -t -r /tmp/lldbtest types
will create a /tmp/lldbtest directory which houses the types directory and its supported
files.
Files modified:
o dotest.py, lldbtest.py:
Add logic to process '-r dir' option to support relocating the tests to a different
top level directory instead of exected in place.
o darwin.py, test/types/Makefile:
The 'make clean' should only clean the minimum .o and .d files.
llvm-svn: 116255
Update the expected match string.
o lldbtest.py:
Indicate when a command fails, even if there is nothing in the error stream.
o TestHelp.py:
Add a regression test case for 'help image dump symtab'.
o CommandObjectHelp.cpp:
Some of the logic branches with successful help command results were not tagged
with a Success Status. They are fixed now. This is important for Python
interaction.
llvm-svn: 116062
Temporarily commenting out the deprecated LaunchProcess() method.
SWIG is not able to handle the overloaded functions.
o dotest.py/lldbtest.py:
Add an '-w' option to insert some wait time between consecutive test cases.
o TestClassTypes.py:
Make the breakpoint_creation_by_filespec_python() test method more robust and
more descriptive by printing out a more insightful assert message.
o lldb.swig: Coaches swig to treat StateType as an int type, instead of a C++ class.
llvm-svn: 115899
To not skip long running tests, pass '-l' to the test driver (dotest.py).
An example:
@unittest2.skipIf(TestBase.skipLongRunningTest(), "Skip this long running test")
def test_foundation_disasm(self):
...
o Added a long running disassemble test to the foundation directory, which iterates
the code symbols from Foundation.framework and kicks off a disassemble command for
for the named function symbol. Found a crasher: rdar://problem/8504895.
o Plus added/updated some comments for the TestBase class.
llvm-svn: 115368
testing various combinations of displaying variales of basic types.
The generic_type_tester() works by first capturing the golden output produced
by the printf stmts of ./a.out, creating a list of (var, value) pairs, and then
running the a.out to a stop point, and comparing the 'frame variable var' output
against the list of (var, value) pairs.
Modified int_type() and added long_type() to use generic_type_tester().
Also modified TestBase.expect() such that substring matching also return ok if
the substring starts at the 0-th position.
llvm-svn: 114708
actually test-and-compare anything yet. The lldbtest.TestBase has an added
method setTearDownCleanup(dictionary=None) to facilitate running the cleanup
right after each data type test is run. The test case can pass a dictionary
object when registering the test case cleanup.
There is currently only int_type test in the repository.
llvm-svn: 114600
themselves. Right now, it tests a breakpoint both before and after it has been
resolved.
Updated lldbtest.TestBase.expect() with an additional keyword argument 'exe' (
default to True), which if set to False, will treat the mandatory first argument
as just the string to be matched/or not-matched against the golden input.
llvm-svn: 114501
of 'breakpoint command add/list/remove' commands to set breakpoint callbacks,
list them, and then remove one.
Modified the lldbtest.TestBase.expect() method to add two additional keyword
arguments:
o matching (default to True), which, if set to False, reverses the semantics of
'expect' to 'expect not'
o patterns (default to None), which specifies a list of regexp patterns to match
against the output from running the command
TestBreakpointCommand.py uses the matching=False and the patterns=[...] expect()
API.
llvm-svn: 114480
to evaluate expressions. Marked with @expectedFailure decorators for the time
being.
Enhanced the lldbtest.TestBase.expect() API to allow an additional keyword arg
named "error". If the client passes "error=True", it signifies that an error
while running the command is expected. The golden input is then compared
against the return object's error output.
llvm-svn: 114228
Johnny Chen