Commit Graph

286 Commits

Author SHA1 Message Date
Johnny Chen 7209d84f1c Add an API test script file for SBCommandInterpreter.
llvm-svn: 131035
2011-05-06 23:26:12 +00:00
Johnny Chen 44932b6805 For a test with unexpected success status, we also dump its session info into a unique file.
llvm-svn: 131011
2011-05-06 20:30:22 +00:00
Johnny Chen 3f620ed8ab Comment.
llvm-svn: 130965
2011-05-05 23:28:17 +00:00
Johnny Chen 87e8dd1077 Add comment for self.child_in_script_interpreter.
llvm-svn: 130963
2011-05-05 23:18:53 +00:00
Johnny Chen b79646cb2e Use standard lldb enum -- lldb.eStopReasonBreakpoint.
llvm-svn: 130960
2011-05-05 22:17:31 +00:00
Johnny Chen 0bade020a0 Add a test script for verifying that the convenience variables:
o lldb.debugger
o lldb.target
o lldb.process
o lldb.thread
o lldb.frame

"just works" when we stop at a breakpoint.

llvm-svn: 130904
2011-05-05 00:36:31 +00:00
Johnny Chen 0934268ee1 Use a more gentle way of shutting down the child process spawned during the test execution using pexpect.
Change some child.expect() to child.expect_exact() as they try to match the string, not a regular expression.

llvm-svn: 130797
2011-05-03 22:14:19 +00:00
Johnny Chen b2a27481c1 A more robust pexpect-based test case for testing against the firing of stop hooks.
Change one test sequence to detect the '** End Stop Hooks **' marker emitted by the
stop hooks mechanism and check for whether the 'expr ptr' stop-hook has been run.

Also, change the TestBase.tearDown() to wait for 2 seocnds before forcefully kill
the pexpect-spawned child lldb process.

llvm-svn: 130767
2011-05-03 17:38:06 +00:00
Johnny Chen de90f1dd93 Change the rest of lldbutil.py's function names to all lower case formats to be consistent.
And modify the test cases accordingly.

llvm-svn: 130314
2011-04-27 17:43:07 +00:00
Johnny Chen 9ee96e7b40 Add a HideStdout() method to our TestBase class and call it from TestAbbreviations.py
and TestAliases.py.  Pass the keyword argument 'check=False' to:

    self.runCmd("script my.date()", check=False)

since we want to restore sys.stdout no matter what the outcome of the runCmd is.

llvm-svn: 129949
2011-04-21 22:50:23 +00:00
Johnny Chen ff4a08372e Modify docstring.
llvm-svn: 129786
2011-04-19 19:12:24 +00:00
Johnny Chen f1251cc820 Add a TraceOn(self) method to the base test class, which returns True if we are
currently in trace mode (-t to dotest.py), i.e., tracing the lldb command execution.
Change TestInferiorCrashing.inferior_crashing_python(self) to check this flag in
order to print the stack trace of the inferior thread.

llvm-svn: 129785
2011-04-19 19:04:38 +00:00
Johnny Chen 1ee61a7f3b Add a test script for exercising the "taregt create", "target list", and "target select" commands.
llvm-svn: 129717
2011-04-18 21:08:05 +00:00
Johnny Chen 1691a1676c Forgot to check in this change with http://llvm.org/viewvc/llvm-project?view=rev&revision=129542.
llvm-svn: 129574
2011-04-15 16:44:48 +00:00
Johnny Chen ac77f3b2ac Turns out that the test failure wrt:
rdar://problem/9173060 lldb hangs while running unique-types

disappears if running with clang version >= 3.  Modify the TestUniqueTypes.py
to detect if we are running with clang version < 3 and, if true, skip the test.

Update the lldbtest.system() function to return a tuple of (stdoutdata, stderrdata)
since we need the stderr data from "clang -v" command.  Modify existing clients of
lldbtest.system() to now use, for example:

         # First, capture the golden output emitted by the oracle, i.e., the
         # series of printf statements.
-        go = system("./a.out", sender=self)
+        go = system("./a.out", sender=self)[0]
         # This golden list contains a list of (variable, value) pairs extracted
         # from the golden output.
         gl = []

And add two utility functions to lldbutil.py.

llvm-svn: 128162
2011-03-23 20:28:59 +00:00
Johnny Chen b877f1efe9 Add a test directory stop-hook to test the newly added "target stop-hook" command.
This uses pexpect module to spawn a 'lldb' program and uses pseudo-TTY to talk to
the child application.

The test cases test setting breakpoints, adding a stop-hook with line range, and
verifies that when the inferior stops, the stop-hook will fire off when it is
within range and will not fire off when it is out of range.

llvm-svn: 127519
2011-03-12 01:18:19 +00:00
Johnny Chen 7fdf1d13f0 Discover the 'lldb' executable in the setUp() hook of the base test class.
llvm-svn: 127481
2011-03-11 19:55:31 +00:00
Johnny Chen 3ff8e96cf7 Some cleanup to plugins/darwin.py after the recent additions of '-A arch' and '-C compiler'
command line options to the test driver.  Replace TestBase.getRunSpec() with TestBase.getRunOptions().

llvm-svn: 124353
2011-01-27 02:58:54 +00:00
Greg Clayton 6f907e69e9 Deprecated old forms of SBTarget::Launch. There is not just one and no
SWIG renaming done to work around deprecated APIs.

llvm-svn: 124075
2011-01-23 17:46:22 +00:00
Johnny Chen d2047fa6af Fix a typo in the comment.
llvm-svn: 123840
2011-01-19 18:18:47 +00:00
Johnny Chen cf7f74e31c The test suite was unnecessarily doing a time.sleep() after performing the
"process launch" or "run" interpreter command.  Let's do the sleep only if
the process launch failed.  This saves about 135 seconds from the whole test
suite run time.

llvm-svn: 123806
2011-01-19 02:02:08 +00:00
Johnny Chen d5f66fcbac Add a test case for the SBFrame APIs. In particular, it uses the frame API to
get the argument values of the call stacks when stopped on the breakpoint.

Radar has been filed for the expected failures:
test failure: ./dotest.py -v -w -t -p TestFrames (argument values are wrong)

llvm-svn: 122460
2010-12-23 01:12:19 +00:00
Johnny Chen c12a1893d2 Execute the test case teardown hooks in a LIFO (last in, first out) order.
ALso add a teardown hook to the LoadUnloadTestCase.test_dyld_library_path()
test case to have it restore the DYLD_LIBRARY_PATH environment variable.

llvm-svn: 121802
2010-12-14 23:13:03 +00:00
Johnny Chen 0a3d1ca28e Add TestBreakpointIgnoreCount.py to exercise the breakpoint ignore count features,
with both command line and Python API tests.

llvm-svn: 121717
2010-12-13 21:49:58 +00:00
Johnny Chen de1451ef4b Make sure that @python_api_test is only used to decorate a test method, not the
test class.

llvm-svn: 121538
2010-12-10 21:33:31 +00:00
Johnny Chen f3e22ac3c7 Add a '+a' command line option to the test driver to run only the Python API tests.
Add an attribute __python_api_test__ (set to True) to the @python_api_test decorated
test method to distinguish them from the lldb command line tests.

llvm-svn: 121500
2010-12-10 18:52:10 +00:00
Johnny Chen 7c7d936e48 Add an infrastructure to mark the Python APIs only test using a decorator.
Example:

    @python_api_test
    def test_evaluate_expression_python(self):
        """Test SBFrame.EvaluateExpression() API for evaluating an expression."""
    ...

The opposite of Python APIs only test is an lldb command line test, which sends
commands to the lldb command interpreter.  Add a '-a' option to the test driver
to skip Python API only tests.

Modify TestExprs.py to mark a test as @python_api_test and remove an @expectedFailure
decorator as the bug has been fixed.

llvm-svn: 121442
2010-12-10 00:51:23 +00:00
Johnny Chen 5b3a357dec Add a test/source-manager directory for testing lldb core component SourceManager.
Initial test case test_modify_source_file_while_debugging() in TestSourceManager.py
tests the caching mechanism of the source manager.

llvm-svn: 121389
2010-12-09 18:22:12 +00:00
Johnny Chen 025d1b8f95 For SBTarget.Launch()/LaunchProcess(), there's no need to pass an empty string
as the args and the envs to the launched process.

o lldbtest.py:

  Forgot to check in some assertion messages changes for lldbtest.py.

o dotest.py:

  Also add "api" category to the default lldb log option list.

llvm-svn: 121220
2010-12-08 01:25:21 +00:00
Johnny Chen 224af79e80 Fail fast by raising an exception if the 'string_to_match' cannot be located
within the file 'filename'.

llvm-svn: 121039
2010-12-06 21:48:08 +00:00
Johnny Chen f10af38ba5 Forgot to also check in this file along with the TestBreakpointConditions.py change.
llvm-svn: 120861
2010-12-04 00:07:24 +00:00
Johnny Chen b91cb31490 Add a mechanism of overwriting the default 1.0 second wait time between test
cases when specifying '-w' to the test driver, by using an environment variable
LLDB_TIME_WAIT_BETWEEN_TEST_CASES.

llvm-svn: 120625
2010-12-01 23:35:13 +00:00
Johnny Chen 6bc1366366 TestBase.setUp() needs to make sure lldb.blacklist is not None
before iterating on it.

llvm-svn: 120623
2010-12-01 23:07:36 +00:00
Johnny Chen 75739045d3 Add a '-b blacklistFile' option to the test driver to take a file specifying the
test classes or test cases to be excludued from the test suite.

Check in an example blacklist file: blacklist.py:

"""
'blacklist' is a Python dictionary, it stores the mapping of a string describing
either a testclass or a testcase, i.e, testclass.testmethod, to the reason (a
string) it is blacklisted.

Following is an example which states that test class IntegerTypesExprTestCase
should be skipped because 'This test class crashed' and the test case
FoundationTestCase.test_data_type_and_expr_with_dsym should be skipped because
it is 'Temporarily disabled'.

blacklist = {'IntegerTypesExprTestCase': 'This test class crashed',
             'FoundationTestCase.test_data_type_and_expr_with_dsym': 'Temporarily disabled'
             }
"""

blacklist = {}

An example of invoking the test driver and specifying a blacklist file:

./dotest.py -b blacklist.py -v types

This runs the tests under 'types' directory but excludes the tests specified in
balcklist.py.

llvm-svn: 120620
2010-12-01 22:47:54 +00:00
Johnny Chen c9ea0f8425 Add an expect() statement within breakpoint_conditions() method which tests that
the process status is indeed 'stopped' before invoking 'frame variable -t val'
command to inspect the frame variable.  When I run the test suite, it sometimes
emits an output like:

runCmd: frame variable -t val
runCmd failed!
error: you must be stopped in a valid stack frame to view frame variables.

The expect() statement inserted is to make sure that the process is stopped and
we have a valid frame.

llvm-svn: 120318
2010-11-29 20:49:09 +00:00
Johnny Chen 4921b11320 Change the variable TestBase.timeWait to the more descriptive timeWaitNextLaunch.
Increase the delta between successive test cases, enabled with the '-w' option to
the test driver, from 0.5 to 1.0 second, to arrive at a cleaner state before the
next test case.

llvm-svn: 120315
2010-11-29 20:20:34 +00:00
Johnny Chen 80e6db9c79 Make the string matching for 'frame variable' more stringent with respect to
output from clang and llvm-gcc compiled program; both generate the correct debug
info with respect to the typedef scoped inside a namespace.

Add a TestBase.getCompiler(self) method which returns the compiler in effect the
test suite is now running with.  Subclasses (like TestNamespace) can use it to
distinguish among different compilers.

llvm-svn: 119445
2010-11-17 00:52:41 +00:00
Johnny Chen 285371e32a The session log entry containing the command to rerun the same test now also includes
the architecture and compiler specs.

llvm-svn: 118860
2010-11-11 23:54:12 +00:00
Johnny Chen 5d6c464f46 Some rewordings of the assert messages for process stopped due to breakpoint.
llvm-svn: 118739
2010-11-10 23:46:38 +00:00
Johnny Chen de0338bac9 Add the actual stop reason to an assert message for bitfields_variable_python()
test method when asserting:

    thread.GetStopReason() == lldb.eStopReasonBreakpoint

llvm-svn: 118711
2010-11-10 20:20:06 +00:00
Johnny Chen 07569be70c When dumping the session log for a failed/errored test, also emit the command to
invoke the test driver to rerun the very same test.  Example output:

/Volumes/data/lldb/svn/trunk/test $ tail 2010-11-09-14_51_34/ExpectedFailure-TestSettings.SettingsCommandTestCase.test_set_output_path.log 

Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 136, in test_set_output_path
    "'stdout.txt' exists due to target.process.output-path.")
AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.


To rerun this test, issue the following command from the 'test' directory:

./dotest.py -v -t -f SettingsCommandTestCase.test_set_output_path

llvm-svn: 118646
2010-11-09 22:56:12 +00:00
Johnny Chen c0c67f275d Distinguish between the assert messages for runCmd and expect. The former now
takes the form:

    "Command '%s' returns successfully" % str

and the latter takes the form:

    "'%s' returns expected result" % str
    or
    "'%s' matches expected result" % str

llvm-svn: 118599
2010-11-09 18:42:22 +00:00
Johnny Chen f7f9d9d880 Make the type of session log part of the filename itself. It allows for easier
identification of the test failures/errors by human beings as well as automatic
processings.

The prefix which identifies the type can be: Error, Failure, or ExpectedFailure.

llvm-svn: 118315
2010-11-06 00:07:07 +00:00
Johnny Chen a29c14e40d Fix comments.
llvm-svn: 118285
2010-11-05 18:10:11 +00:00
Johnny Chen 87bb589c4d Add a test for Python API SBValue.GetValueType() inside TestArrayTypes.py for a
local variable and an argument variable.

Add ValueTypeString() utility function into lldbutil.py which converts the enum
into a human readable string.

Modify TestBase.DebugSBValue() to also dump the value type of an SBValue object.

llvm-svn: 118197
2010-11-03 21:37:58 +00:00
Johnny Chen 110bd0f8cf Now that the test suite has very few expected failures remaining (2), it is a good
idea to also dump the session information for expected failures in addition to
failures and errors (tests with unexpected exceptions).

Modify the LLDBTestResult class which derives from unittest2.TextTestResult to
intercept the addExpectedFailure() method in order to mark the test instance as
needing its session information dumped.

llvm-svn: 118185
2010-11-03 18:17:03 +00:00
Johnny Chen b2d4f7f20e Add more comment for dumpSessionInfo(self) method.
llvm-svn: 118129
2010-11-03 00:09:38 +00:00
Johnny Chen ab9c1dd9ba Replace the two call sites of inspect.getsource(obj) with a utility function
getsource_if_available(obj) which also handles the exception if it occurs.

llvm-svn: 117949
2010-11-01 20:35:01 +00:00
Johnny Chen 6713a9b317 Python API changes: SBValue.IsPtrType() -> SBValue.IsPointerType().
SBFrame print representation: idx: 0 -> #0

llvm-svn: 117924
2010-11-01 16:08:47 +00:00
Johnny Chen 62d4f8684f Add comment on passing the assert message to self.runCmd().
llvm-svn: 117604
2010-10-28 21:10:32 +00:00
Johnny Chen ebfff955b6 If TestBase.expect() is passed an assert message, pass it along when delegating to TestBase.runCmd()
to execute the debugger command.  This will result in a more semantic assert message than just the
command failed message like before:

AssertionError: False is not True : Command 'thread list' returns successfully

And now, we will have:

AssertionError: False is not True : Process state is stopped due to breakpoint
llvm-svn: 117569
2010-10-28 18:24:22 +00:00
Johnny Chen 8334dadb62 Add more docstring for the lldbtest.TestBase class.
llvm-svn: 117175
2010-10-22 23:15:46 +00:00
Johnny Chen 5bfb8ee64e Add test case for using SBBreakpointLocation to set break condition.
llvm-svn: 117116
2010-10-22 18:10:25 +00:00
Johnny Chen 096011eebf Add an option '-s session-dir-name' to overwrite the default timestamp-named
directory used to dump the session info for test failures/errors.

Example:

/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -s jason -v array_types

Session info for test errors or failures will go into directory jason
----------------------------------------------------------------------
Collected 4 tests

test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... ok
test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... ok
test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types. ... FAIL

======================================================================
FAIL: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
Test 'frame variable var_name' on some variables with array types.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 27, in test_with_dwarf_and_run_command
    self.array_types()
  File "/Volumes/data/lldb/svn/trunk/test/array_types/TestArrayTypes.py", line 62, in array_types
    'stop reason = breakpoint'])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 594, in expect
    self.runCmd(str, trace = (True if trace else False), check = not error)
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 564, in runCmd
    msg if msg else CMD_MSG(cmd, True))
AssertionError: False is not True : Command 'thread list' returns successfully

----------------------------------------------------------------------
Ran 4 tests in 3.086s

FAILED (failures=1)
/Volumes/data/lldb/svn/trunk/test $ ls jason
TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
/Volumes/data/lldb/svn/trunk/test $ head -10 jason/TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log 
Session info generated @ Thu Oct 21 09:54:15 2010

os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=NO']]
stdout: rm -rf "a.out" "a.out.dSYM"  main.o main.d
cc -arch x86_64 -gdwarf-2 -O0   -c -o main.o main.c
cc -arch x86_64 -gdwarf-2 -O0  main.o -o "a.out"

stderr: None
retcode: 0

/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 117028
2010-10-21 16:55:35 +00:00
Johnny Chen 2e431cea84 Make the breakpoint condition test more robust with regard to checking the correct
parent call frame information.  And comment out the check for stop reason for the
time being.  The stop reason disappeared from the "thread backtrace" output for
breakpoint stop with condition.

llvm-svn: 116939
2010-10-20 18:38:48 +00:00
Johnny Chen bf45719408 Clean up the teardown logic to make it more robust and to record the additions and
invocations of them into session object.

Remove a debug statement.

llvm-svn: 116938
2010-10-20 18:12:58 +00:00
Johnny Chen 707d8228df Restoring the original setting should be done more robustly by adding a hook function
to be run during tearDown() to effect the restore action instead of executing it inline
during the test method, because the test may already fail and bailout before the inline
restore action.

Fix test_set_output_path() and pass_run_args_and_env_vars() to use this mechanism.

llvm-svn: 116881
2010-10-19 23:40:13 +00:00
Johnny Chen 1e4fc3c9be Add Python doc string for setTearDownCleanup() method.
llvm-svn: 116874
2010-10-19 22:45:25 +00:00
Johnny Chen 3343f04bf8 Add a little helper function SETTING_MSG(setting) to be used from TestSettings.test_set_output_path().
llvm-svn: 116847
2010-10-19 19:11:38 +00:00
Johnny Chen 430eb7607d Change the environment variable name to be more descriptive from LLDB_TIME_WAIT to LLDB_TIME_WAIT_NEXT_LAUNCH.
llvm-svn: 116809
2010-10-19 16:00:42 +00:00
Johnny Chen 04d2c5cb0e Modify the test driver and lldbtest.TestBase so that the dumping of session info
now goes into a timestamp-specific directory instead of the previous .session-*
files.

[17:24:34] johnny:/Volumes/data/lldb/svn/trunk $ ls -l test/2010-10-18-16:56:12.935342
total 48
-rw-r--r--  1 johnny  admin  1695 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dsym_and_run_command.log
-rw-r--r--  1 johnny  admin  1652 Oct 18 16:56 TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  2967 Oct 18 16:56 TestBreakpointCommand.BreakpointCommandTestCase.test_with_dsym.log
-rw-r--r--  1 johnny  admin  1648 Oct 18 16:56 TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command.log
-rw-r--r--  1 johnny  admin  1665 Oct 18 16:56 TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase.test_with_dsym_and_python_api.log
-rw-r--r--  1 johnny  admin  3873 Oct 18 16:58 TestFloatTypesExpr.FloatTypesTestCase.test_float_types_with_dsym.log
[17:24:37] johnny:/Volumes/data/lldb/svn/trunk $ 

Also, the dumping happens when a test errored in additioned to when it failed.

llvm-svn: 116778
2010-10-19 00:25:01 +00:00
Johnny Chen 24d6ea25f8 Missed this spot where module.cleanup() can be passed a test instance to
facilitate session recording.  This happens inside an instance method where the
test instance is well known.

llvm-svn: 116649
2010-10-16 00:08:49 +00:00
Johnny Chen 690fcef762 Some re-achitecturing of the plugins interface. The caller is now required to
pass in a 'sender' arg to the buildDefault(), buildDsym(), buildDwarf(), and
cleanup() functions.  The sender arg will be the test instance itself (i.e.,
an instance of TestBase).  This is so that the relevant command execution can be
recorded in the TestBase.session object if sender is available.

The lldbtest.system() command has been modified to pop the 'sender' arg out of
the keyword arguments dictionary and use it as the test instance to facilitate
seesion recordings.  An example is in test/types/AbstractBase.py:

    def generic_type_tester(self, atoms, quotedDisplay=False):
        """Test that variables with basic types are displayed correctly."""

        # First, capture the golden output emitted by the oracle, i.e., the
        # series of printf statements.
        go = system("./a.out", sender=self)

There are cases when sender is None.  This is the case when the @classmethod is
involved in the use of these APIs.  When this happens, there is no recording
into a session object, but printing on the sys.stderr is still honored if the
trace flag is ON.

An example is in test/settings/TestSettings.py:

    @classmethod
    def classCleanup(cls):
        system(["/bin/sh", "-c", "rm -f output.txt"])
        system(["/bin/sh", "-c", "rm -f stdout.txt"])

llvm-svn: 116648
2010-10-15 23:55:05 +00:00
Johnny Chen 27b107b800 Add a test case for exercising breakpoint conditions using the lldb command:
breakpoint modify -c 'val == 3' 1

after:

    breakpoint set -n c

which sets a breakpoint on function 'c'.  The breakpoint should only stop if
expression 'val == 3' evaluates to true.

llvm-svn: 116607
2010-10-15 18:52:22 +00:00
Johnny Chen 164f1e18b1 Add a const string for assert message. Remove extra printing of newlines from
the session recordings.

llvm-svn: 116596
2010-10-15 18:07:09 +00:00
Johnny Chen fd64df4f92 Add more comments.
llvm-svn: 116583
2010-10-15 16:22:10 +00:00
Johnny Chen ab254f5da4 Be more specific about cases whenthe runCmd() check flag is False, meaning there
is no need to check the return status of the command execution, and an error
status is not deemed a failure in the test.

llvm-svn: 116582
2010-10-15 16:13:00 +00:00
Johnny Chen 56f7939cdc Simply use the TestBase.markFailure() callback method to set the __failed__ flag.
llvm-svn: 116559
2010-10-15 02:28:13 +00:00
Johnny Chen 150c3cc825 This is an initial version of test driver enhanceent to be able to dump the
session info after a test case failure, allowing more direct inspection of
debugger session which leads to the test failure.

For a simple usage scenario:

[18:06:26] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log 

...

[18:14:43] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l .session-*
-rw-r--r--  1 johnny  admin  1359 Oct 14 18:06 .session-TestArrayTypes.ArrayTypesTestCase.test_with_dwarf_and_run_command
-rw-r--r--  1 johnny  admin  2054 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser
-rw-r--r--  1 johnny  admin  2055 Oct 14 18:07 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser
-rw-r--r--  1 johnny  admin  1351 Oct 14 17:57 .session-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_run_command
[18:14:51] johnny:/Volumes/data/lldb/svn/trunk/test $ 

The test case which failed will have its recorded session info dumped to a
.session-* file in the current working directory.  For test suite using
relocated directory, expect to find the .session-* files there.

In this checkin, I also add @skip decorator to the two test methods in
test/foundation/TestObjCMethods.py as it looks like the test suite is
deadlocking when running the tests.  More investigations are needed.

llvm-svn: 116552
2010-10-15 01:18:29 +00:00
Johnny Chen c066ab4385 Add an initial version of test that exercise the lldb commands: 'process signal'
and 'process handle'.  The test suite would like to control the asynch/sync
execution of the interpreter during the middle of the test method, so the
CommandInterpreter::SetSynchronous(bool value) is modified to allow the mode to
be changed more than once.

In practice, it would be advisable to control the process and to set the
async/sync mode from a single thread, too.

llvm-svn: 116467
2010-10-14 01:22:03 +00:00
Johnny Chen cd9b7779f3 Avoid using hardcoded line number to break on. Use the line_number() utility
function to get the line number to break on during setUp().

llvm-svn: 116275
2010-10-12 00:09:25 +00:00
Johnny Chen 47ceb03210 Add a utility function to lldbtest.py to return the line number of a matched string
within a file.  This is to be used within the test case to avoid hardcoded line number.

array_types/TestArrayTypes.py is modified first to use this pattern.  Other test modules
to follow.

rdar://problem/8537816
Testsuite: don't set breakpoints by exact file & line number

llvm-svn: 116270
2010-10-11 23:52:19 +00:00
Johnny Chen 707b3c9c73 Added the capability for the test driver to relocate the tests and the intermediate
files to a different top level directory than those specified on the command line.

When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all.  This allows for an easier
postmortem analysis of test failures.

Example:

./dotest.py -v -t -r /tmp/lldbtest types

will create a /tmp/lldbtest directory which houses the types directory and its supported
files.

Files modified:

o dotest.py, lldbtest.py:

  Add logic to process '-r dir' option to support relocating the tests to a different
  top level directory instead of exected in place.

o darwin.py, test/types/Makefile:

  The 'make clean' should only clean the minimum .o and .d files.

llvm-svn: 116255
2010-10-11 22:25:46 +00:00
Johnny Chen 65045f2168 o TestStdCXXDisassembly.py:
Update the expected match string.

o lldbtest.py:

  Indicate when a command fails, even if there is nothing in the error stream.

o TestHelp.py:

  Add a regression test case for 'help image dump symtab'.

o CommandObjectHelp.cpp:

  Some of the logic branches with successful help command results were not tagged
  with a Success Status.  They are fixed now.  This is important for Python
  interaction.

llvm-svn: 116062
2010-10-08 17:21:27 +00:00
Johnny Chen 28ae29420a Move the enum to string utility functions from lldbtest.py to lldbuti.py and
update the affected API clients.

llvm-svn: 115995
2010-10-07 22:15:58 +00:00
Johnny Chen c5b06ebe82 Clean up the StateType and StopReason enums now that they reside in the lldb module.
llvm-svn: 115922
2010-10-07 16:51:56 +00:00
Johnny Chen 0ed37c9615 o SBtarget.cpp/.h:
Temporarily commenting out the deprecated LaunchProcess() method.
  SWIG is not able to handle the overloaded functions.

o dotest.py/lldbtest.py:

  Add an '-w' option to insert some wait time between consecutive test cases.

o TestClassTypes.py:

  Make the breakpoint_creation_by_filespec_python() test method more robust and
  more descriptive by printing out a more insightful assert message.

o lldb.swig: Coaches swig to treat StateType as an int type, instead of a C++ class.

llvm-svn: 115899
2010-10-07 02:04:14 +00:00
Johnny Chen 67af43f04f Make the test module with unique name instead of just the generic name TestDisassembly.
Add a utility function to lldbtest.py to get the native pointer size of the host os.

llvm-svn: 115652
2010-10-05 19:27:32 +00:00
Johnny Chen da88434b39 o Added a new feature to the test framework to skip long running tests conditionally.
To not skip long running tests, pass '-l' to the test driver (dotest.py).

  An example:

    @unittest2.skipIf(TestBase.skipLongRunningTest(), "Skip this long running test")
    def test_foundation_disasm(self):
        ...

o Added a long running disassemble test to the foundation directory, which iterates
  the code symbols from Foundation.framework and kicks off a disassemble command for
  for the named function symbol.  Found a crasher: rdar://problem/8504895.

o Plus added/updated some comments for the TestBase class.

llvm-svn: 115368
2010-10-01 22:59:49 +00:00
Johnny Chen 703dbd049a These prints should go to stderr as well. Plus add a global assert string.
llvm-svn: 115166
2010-09-30 17:06:24 +00:00
Johnny Chen b052f6c595 Added a generic_type_tester() to the TestBasicTypes class to be used for
testing various combinations of displaying variales of basic types.

The generic_type_tester() works by first capturing the golden output produced
by the printf stmts of ./a.out, creating a list of (var, value) pairs, and then
running the a.out to a stop point, and comparing the 'frame variable var' output
against the list of (var, value) pairs.

Modified int_type() and added long_type() to use generic_type_tester().

Also modified TestBase.expect() such that substring matching also return ok if
the substring starts at the 0-th position.

llvm-svn: 114708
2010-09-23 23:35:28 +00:00
Johnny Chen c70b02a324 Checked in an initial template for the types directory. Right now, it doesn't
actually test-and-compare anything yet.  The lldbtest.TestBase has an added
method setTearDownCleanup(dictionary=None) to facilitate running the cleanup
right after each data type test is run.  The test case can pass a dictionary
object when registering the test case cleanup.

There is currently only int_type test in the repository.

llvm-svn: 114600
2010-09-22 23:00:20 +00:00
Johnny Chen d64c3e7913 Trivial change of a trace output string.
llvm-svn: 114505
2010-09-21 23:47:01 +00:00
Johnny Chen 9c48b8d79c Added a subtest to exercise the capability of lldb Python objects to print
themselves.  Right now, it tests a breakpoint both before and after it has been
resolved.

Updated lldbtest.TestBase.expect() with an additional keyword argument 'exe' (
default to True), which if set to False, will treat the mandatory first argument
as just the string to be matched/or not-matched against the golden input.

llvm-svn: 114501
2010-09-21 23:33:30 +00:00
Johnny Chen 90312a8492 Wrapped the subclass-specific cleanup call within a try:except: clause.
And removed the informational output from the conditional_break test case.

llvm-svn: 114493
2010-09-21 22:34:45 +00:00
Johnny Chen ea88e94318 Added a more complex test case of breakpoint commands, which executes a sequence
of 'breakpoint command add/list/remove' commands to set breakpoint callbacks,
list them, and then remove one.

Modified the lldbtest.TestBase.expect() method to add two additional keyword
arguments:

o matching (default to True), which, if set to False, reverses the semantics of
  'expect' to 'expect not'

o patterns (default to None), which specifies a list of regexp patterns to match
  against the output from running the command

TestBreakpointCommand.py uses the matching=False and the patterns=[...] expect()
API.

llvm-svn: 114480
2010-09-21 21:08:53 +00:00
Johnny Chen aa90292126 Fixed a typo and supplied a more appropriate assert message.
llvm-svn: 114232
2010-09-17 22:45:27 +00:00
Johnny Chen b330786d1f Added test cases to FoundationTestCase to exercise lookup of objc data types and
to evaluate expressions.  Marked with @expectedFailure decorators for the time
being.

Enhanced the lldbtest.TestBase.expect() API to allow an additional keyword arg
named "error".  If the client passes "error=True", it signifies that an error
while running the command is expected.  The golden input is then compared
against the return object's error output.

llvm-svn: 114228
2010-09-17 22:28:51 +00:00
Johnny Chen 2fcc0e5555 Added a test case for the settings command which sets process.output-path and
checks that the launched process writes its standard output there.

llvm-svn: 114102
2010-09-16 18:26:06 +00:00
Johnny Chen 1a9f4dd5a0 Provided a mechanism for the test class to cleanup after itself once it's done.
This will remove the confusion experienced when previous test runs left some
files (both intermediate or by-product as a result of the test).

lldbtest.TestBase defines a classmethod tearDownClass(cls) which invokes the
platform-specific cleanup() function as defined by the plugin; after that, it
invokes a subclass-specific function classCleanup(cls) if defined; and, finally,
it restores the old working directory.

An example of classCleanup(cls) is in settings/TestSettings.py:

    @classmethod
    def classCleanup(cls):
        system(["/bin/sh", "-c", "rm output.txt"])

where it deletes the by-product "output.txt" as a result of running a.out.

llvm-svn: 114058
2010-09-16 01:53:04 +00:00
Johnny Chen 9fa032173a Changed to a more informative message.
llvm-svn: 113989
2010-09-15 18:00:19 +00:00
Johnny Chen f7f0cabe9f We believed the intermittent process launch failure while running the test suite
has been fixed by r113947.  But just in case it still fails, emits a prominent
message always.

llvm-svn: 113986
2010-09-15 17:33:57 +00:00
Johnny Chen ae74556ff6 Added two test cases to exercise 'expr this->m_c_int' with @expectedFailure
decorators for the time being.

llvm-svn: 113905
2010-09-15 00:00:54 +00:00
Johnny Chen e049051e68 More appropriate headings for the traceoutput: stdout & stderr.
llvm-svn: 113879
2010-09-14 22:39:02 +00:00
Johnny Chen 1aad5c61f8 Removed the expectedFailure decorator from test_with_dwarf_and_run_command() test case
as it now passes.  Added some extra tests to breakpoint_creation_by_filespec_python().

More clarification for the "os command" output and error as defined in
lldbtest.system() function.

Cleaned up the option processing of the test driver (dotest.py) and fixed the comment
about enabling gdb-remote logging.  Example:

$ GDB_REMOTE_LOG=/tmp/log.txt ./dotest.py -v -t enum_types

llvm-svn: 113868
2010-09-14 22:01:40 +00:00
Johnny Chen 1394a4b715 Extend the build mechanism to allow for specifying the compiler used to build
the binaries.

If the build* function is passed the compiler argument, for example, 'llvm-gcc',
it is passed as a make variable to the make command.  Otherwise, we check the
LLDB_CC environment variable; if it is defined, it is passed as a make variable
to the make command.

If neither the compiler keyword argument nor the LLDB_CC environment variable is
specified, no CC make variable is passed to the make command.  The Makefile gets
to define the default CC being used.

--------------------------------------------------------------------------------
Example usage follows:

o Via the keyword argument:

class ArrayTypesTestCase(TestBase):

    mydir = "array_types"

    @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
    def test_with_dsym_and_run_command(self):
        """Test 'frame variable var_name' on some variables with array types."""
        self.buildDsym(compiler='llvm-gcc')
        self.array_types()
...

o Via LLDB_CC environment variable:

$ LLDB_CC=llvm-gcc ./dotest.py -v -t array_types
----------------------------------------------------------------------
Collected 4 tests

test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
Use Python APIs to inspect variables with array types. ... 
os command: [['/bin/sh', '-c', 'make clean; make MAKE_DSYM=YES CC=llvm-gcc']]
output: rm -rf "a.out" "a.out.dSYM"  main.o main.d
llvm-gcc -arch x86_64 -gdwarf-2 -O0 -arch x86_64 -gdwarf-2 -O0  -c -o main.o main.c
llvm-gcc -arch x86_64 -gdwarf-2 -O0  main.o -o "a.out"
/usr/bin/dsymutil  -o "a.out.dSYM" "a.out"

...

llvm-svn: 113781
2010-09-13 20:54:30 +00:00
Johnny Chen ea772bbba0 The output for term-width setting has single quotes around the (int) value.
And added a trace output for the stop function name to breakAfterLaunch() method.

llvm-svn: 113251
2010-09-07 18:55:50 +00:00
Johnny Chen 1b1b9accd1 Marked test_process_launch_for_universal() test case as requiring 'darwin' and 'i386'
in order to be run.  And added a default build phase at the beginning of the method.

llvm-svn: 113037
2010-09-03 23:49:16 +00:00
Johnny Chen 725945d568 Fixed an lldb infrastructure bug, where the debugger should reaaly update its
execution context only when the process is still alive.  When running the test
suite, the debugger is launching and killing processes constantly.

This might be the cause of the test hang as reported in rdar://problem/8377854,
where the debugger was looping infinitely trying to update a supposedly stale
thread list.

llvm-svn: 113022
2010-09-03 22:35:47 +00:00
Johnny Chen 57b47384cc Added comments regarding the two mechanisms of process cleanup to lldbtest.py.
Also changed the expected strings to be matched since "thread list" changed its
output format. 

llvm-svn: 112880
2010-09-02 22:25:47 +00:00
Johnny Chen 7d1d7537cf Moved the process cleanup of Script-Bridge-based APIs into TestBase.tearDown()
method where they belong.  Also fixed a logic error in maintaining the command
interface flag (runStarted) indicating whether the lldb "run"/"process launch"
command has been issued.  It was erroneously cleared.

Modified the test cases to take advantage of the refactoring.

llvm-svn: 112863
2010-09-02 21:23:12 +00:00
Johnny Chen 13639089c6 Put the little dances done after SBTarget.LaunchProcess() into the base class.
llvm-svn: 112749
2010-09-01 22:08:51 +00:00
Johnny Chen 63dfb27647 Avoid killing the inferior process twice by passing a setCookie=False keyword
argument when issuing a "run" lldb command within the test case meant to
exercise the Python APIs, but is using the command interface due to certain
reason (such as target.LaunchProcess() does not reliably bring up the inferior).

llvm-svn: 112682
2010-09-01 00:15:19 +00:00
Johnny Chen 8d55a34a34 Changed the buildDsym()/buildDwarf() TestBase methods to use a plugin framework
to delegate the building of binaries to a sys.platform-sepcific plugin.

Modified the dotest.py test driver to add the "plugins" directory to the
PYTHONPATH as well.

darwin.py is the Mac OS X plugin module.

llvm-svn: 112606
2010-08-31 17:42:54 +00:00
Johnny Chen 324355b9a1 Added doc strings to the array_types test cases. And terminate the current
process being debugged in the TestBase.tearDown() instead of letting it continue
and finish.

llvm-svn: 112556
2010-08-30 23:44:39 +00:00
Johnny Chen a33a93cbde Converted TestArrayTypes.py to Dsym/Dwarf combination, and added verbose output
of os command to lldbtest.TestBase.system() method.

llvm-svn: 112547
2010-08-30 23:08:52 +00:00
Johnny Chen 2f1ad5e2bd Added buildDsym() and buildDwarf() methods to lldbtest.TestBase class, and call
them from test cases instead of issuing "make clean; make ..." os command.

llvm-svn: 112542
2010-08-30 22:26:48 +00:00
Johnny Chen 8952a2d5b6 Added a system() method to the TestBase class of lldbtest.py, which is actually
taken from Python 2.7's subprocess.check_output() convenience function.  The
purpose of this method is to run the os command with arguments and return its
output as a byte string.

Modified hello_world/TestHelloWorld.py to have two test cases:

o test_with_dsym_and_run_command
o test_with_dwarf_and_process_launch_api

with the dsym case conditioned on sys.platform.startswith("darwin") being true.
The two cases utilize the system() method to invoke "make clean; make MAKE_DYSM=YES/NO"
to prepare for the appropriate debugging format before running the test logic.

llvm-svn: 112530
2010-08-30 21:35:00 +00:00
Johnny Chen 5ee881948a Added a test case test_breakpoint_creation_by_filespec_python() which creates a
breakpoint by FileSpec and line number and exercises some FileSpec APIs.

Also, RUN_STOPPED is a bad assert name, RUN_SUCCEEDED is better.

llvm-svn: 112327
2010-08-27 23:47:36 +00:00
Johnny Chen 82d404c886 Added TestHelloWorld.py which exercises the Python APIs for target, breakpoint,
and process.  Added comment within the file about issues of using LaunchProcess
of SBTarget to launch a process (rdar://problem/8364687).

llvm-svn: 112276
2010-08-27 18:08:58 +00:00
Johnny Chen 827edffafd Added a test case to bitfields which uses the Python APIs from lldb.py.
Added a utility method to TestBase class to debug print an SBValue object.

llvm-svn: 112247
2010-08-27 00:15:48 +00:00
Johnny Chen 27c412320e Added a utility function EnvArray() to lldbtest.py.
llvm-svn: 112223
2010-08-26 21:49:29 +00:00
Johnny Chen 5fca8ca8cd o Added a test case for array_types which uses the Python APIs from lldb.py,
with the only exception of launching the process from SBTarget which is under
  investigation.

o build-swig-Python.sh should also checks the timestamp of ${swig_input_file}
  for update eligibility.  Also, once an update is in order, there's no need
  to check the remaining header files for timestamps.

o Coaches swig to treat StopReason as an int type, instead of a C++ class.

llvm-svn: 112210
2010-08-26 20:04:17 +00:00
Johnny Chen 61703c96f2 Minor doc string modification.
llvm-svn: 112103
2010-08-25 22:56:10 +00:00
Johnny Chen f3c59231b3 Added logic to TestUniversal.py to exercise the python APIs:
o SBDebugger.GetCurrentTarget()
o SBTarget.GetProcess()
o SBProcess.GetAddressByteSize()

in order to make sure that, indeed, 64-bit, followed by 32-bit processes have
been launched.

Added invoke() method to TestBase to factor in the tracing logic in one place.
This method allows an object to call a method with no arg reflectively.

llvm-svn: 112102
2010-08-25 22:52:45 +00:00
Johnny Chen 981463d670 Fixed a logic error in the expect() method. If the start string does not match,
there's no point matching further sub strings; the expect() already fails.

Also cleaned up the assert message for VARIABLES_DISPLAYED_CORRECTLY.

llvm-svn: 112074
2010-08-25 19:00:04 +00:00
Johnny Chen f2b70237e6 Allow command retries in case of process launch failures. This recovery
mechanism seems to work fine on my MacBook Pro in some limited test cases.

The default maxLaunchCount and timeWait variables used in the scheme can be
overridden by the env variables LLDB_MAX_LAUNCH_COUNT and LLDB_TIME_WAIT.

llvm-svn: 112071
2010-08-25 18:49:48 +00:00
Johnny Chen c7c9fcfbd6 More descriptive trace messages for the matchings of start and sub strings.
llvm-svn: 112000
2010-08-24 23:48:10 +00:00
Johnny Chen 3c884a014c Added a test for launching a universal binary. Launch of i386 architecture
currently fails: rdar://problem/8349784.

Forgot to check in lldbtest.py in the previous commit r111958.

llvm-svn: 111975
2010-08-24 22:07:56 +00:00
Johnny Chen d0190a6182 Changed the keyword argument for runCmd()/expect() from 'verbose' to 'trace',
which, defaults to False, and if set to True, will trace lldb command execution
and result.

Added "-t" command option to the test driver dotest.py which sets the
LLDB_COMMAND_TRACE environment variable to "YES" and as a result always turns on
command tracing regardless of the 'trace' keyword argument to runCmd()/expect().

llvm-svn: 111811
2010-08-23 17:10:44 +00:00
Johnny Chen ff3d01d0b7 Print the verbose output of runCmd()/expect() to stderr instead of stdout.
And converted the rest of the test cases to runCmd()/expect().

llvm-svn: 111677
2010-08-20 21:03:09 +00:00
Johnny Chen 74f26b8188 Changed TestBase.expect() to allow default 'msg' arg. Converted TestHelp.py.
llvm-svn: 111671
2010-08-20 19:17:39 +00:00
Johnny Chen b145bbaf4b Added more verbose output when string match fails. Converted TestGlobalVariables.py.
llvm-svn: 111666
2010-08-20 18:25:15 +00:00
Johnny Chen 5bbb88ff97 Added verbose option to runCmd()/expect() in lldbtest.py. Converted TestFunctionTypes.py.
llvm-svn: 111658
2010-08-20 17:57:32 +00:00
Johnny Chen 617cca957e Converted some more test cases to use runCmd()/expect().
llvm-svn: 111652
2010-08-20 17:04:20 +00:00
Johnny Chen a6480c124e Use cmd.startswith("run") instead of string equivalence test.
llvm-svn: 111587
2010-08-19 23:53:55 +00:00
Johnny Chen 27f212d1e6 Abstracted the running of command through the command interpreter and checking
its return status into lldbtest.TestBase.runCmd(); and runCmd() in combination
with checking the output against matching substrings (including startswith) into
lldbtest.TestBase.expect().

TestUnsignedTypes.py is refactored to use the abstracted APIs.  Other test cases
to be modified later.

llvm-svn: 111572
2010-08-19 23:26:59 +00:00
Johnny Chen e76896c987 Specify a more meaningful assert message for TestDeadStrip.py.
llvm-svn: 111294
2010-08-17 21:33:31 +00:00
Johnny Chen 6ca006c7c1 Factored the "continue" command execution of the inferior process as part of the
cleanup before finish into the test fixture in lldbtest.TestBase.tearDown().

Derivatives of TestBase is responsible for setting self.runStarted to True if an
inferior process has been started.

llvm-svn: 111188
2010-08-16 21:28:10 +00:00
Johnny Chen 1794184ada Modified the remaining test cases to display more meaningful assert messages.
Added a generic message generator to the lldbtest.py base module.

llvm-svn: 110625
2010-08-09 23:44:24 +00:00
Johnny Chen 0077809bc2 Added some commonly used assert messages to the lldbtest.py module which houses
the TestBase.  Modified TestArrayTypes.py to use the assert messages.  Other
files to follow.

llvm-svn: 110611
2010-08-09 22:01:17 +00:00
Johnny Chen 9289a65f75 Commented out "import traceback".
llvm-svn: 110497
2010-08-07 01:13:18 +00:00
Johnny Chen 73258830f3 o Added unittest2 which has added the new features in unittest for Python 2.7
backported to Python 2.3+.  Some of the features desired include better
  verbose reporting in unittest2.TextTestRunner and decorator support for
  skipping tests and expected failures.

  http://pypi.python.org/pypi/unittest2

o Modified the existing .py tests to use unittest2 and decorated
  TestSTL.test_step_into_stl(), which is known to always fail currently, with
  @unittest2.expectedFailure.

llvm-svn: 110397
2010-08-05 23:42:46 +00:00
Johnny Chen a21249597e When running a single test case, lldb.SBDebugger.Terminate() is not being called
because unittest.main() calls sys.exit() before returning.  Fixed by registering
an exit handler for this situation.

llvm-svn: 110379
2010-08-05 21:23:45 +00:00
Johnny Chen f02ec12796 Make it fail fast if 'mydir' attribute is not overridden by subclasses of
lldbtest.TestBase.

Also removed some debug prints.

llvm-svn: 107575
2010-07-03 20:41:42 +00:00
Johnny Chen bf6ffa3b89 Abstracted the lldb-specific unittest.TestCase.setUp()/tearDown() in a separate
module lldbtest.py and refactored the existing test cases to derive from the
abstract base class lldbtest.TestBase.

MOdified the test driver (dotest.py and dotest.pl) to set up additional
PYTHONPATH component for locating the lldbtest module, which sits in the same
directory.

llvm-svn: 107563
2010-07-03 03:41:59 +00:00