Commit Graph

67 Commits

Author SHA1 Message Date
Greg Clayton 3bcdfc0ec1 <rdar://problem/12798131>
Cleaned up the option parsing code to always pass around the short options as integers. Previously we cast this down to "char" and lost some information. I recently added an assert that would detect duplicate short character options which was firing during the test suite.

This fix does the following:
- make sure all short options are treated as "int"
- make sure that short options can be non-printable values when a short option is not required or when an option group is mixed into many commands and a short option is not desired
- fix the help printing to "do the right thing" in all cases. Previously if there were duplicate short character options, it would just not emit help for the duplicates
- fix option parsing when there are duplicates to parse options correctly. Previously the option parsing, when done for an OptionGroup, would just start parsing options incorrectly by omitting table entries and it would end up setting the wrong option value

llvm-svn: 169189
2012-12-04 00:32:51 +00:00
Daniel Malea 1b8b392889 Clear auto-confirm setting on teardown for tests that use it
llvm-svn: 168382
2012-11-20 19:46:54 +00:00
Daniel Malea 096925233b Use auto-confirm setting to avoid prompts causing some tests to appear to hang
- this commit addresses bugzilla #14385

llvm-svn: 168375
2012-11-20 18:53:21 +00:00
Daniel Malea 218894788d Temporarily skip tests known to hang on Linux
- opened bugzilla defects #14384 and #14385

llvm-svn: 168323
2012-11-19 20:55:52 +00:00
Jim Ingham 63dfc725a7 Fix all the test case breakages caused by folks writing tests all over the place that depended explicitly
on the output of "break set".  Please don't do this sort of thing!!!!!

llvm-svn: 164433
2012-09-22 00:05:11 +00:00
Greg Clayton 1f7460716b <rdar://problem/11757916>
Make breakpoint setting by file and line much more efficient by only looking for inlined breakpoint locations if we are setting a breakpoint in anything but a source implementation file. Implementing this complex for a many reasons. Turns out that parsing compile units lazily had some issues with respect to how we need to do things with DWARF in .o files. So the fixes in the checkin for this makes these changes:
- Add a new setting called "target.inline-breakpoint-strategy" which can be set to "never", "always", or "headers". "never" will never try and set any inlined breakpoints (fastest). "always" always looks for inlined breakpoint locations (slowest, but most accurate). "headers", which is the default setting, will only look for inlined breakpoint locations if the breakpoint is set in what are consudered to be header files, which is realy defined as "not in an implementation source file". 
- modify the breakpoint setting by file and line to check the current "target.inline-breakpoint-strategy" setting and act accordingly
- Modify compile units to be able to get their language and other info lazily. This allows us to create compile units from the debug map and not have to fill all of the details in, and then lazily discover this information as we go on debuggging. This is needed to avoid parsing all .o files when setting breakpoints in implementation only files (no inlines). Otherwise we would need to parse the .o file, the object file (mach-o in our case) and the symbol file (DWARF in the object file) just to see what the compile unit was.
- modify the "SymbolFileDWARFDebugMap" to subclass lldb_private::Module so that the virtual "GetObjectFile()" and "GetSymbolVendor()" functions can be intercepted when the .o file contenst are later lazilly needed. Prior to this fix, when we first instantiated the "SymbolFileDWARFDebugMap" class, we would also make modules, object files and symbol files for every .o file in the debug map because we needed to fix up the sections in the .o files with information that is in the executable debug map. Now we lazily do this in the DebugMapModule::GetObjectFile()

Cleaned up header includes a bit as well.

llvm-svn: 162860
2012-08-29 21:13:06 +00:00
Johnny Chen 9a27713a74 First step to make the test suite runnable for remote platforms.
For the types directory, we were running lldbtest.system() to execute the compiled program
on the test host to collect golden output in order to compare with the output of various
lldb debugger commands as performed later.  This won't work for the remote platform
scenario where the architecture of the target and host platforms are different.

Modify the AbstractBase class to use lldb to launch the inferior while specifying the
output file, from which the golden output is collected and grokked.  How to bootstrap and
to connect to the remote platform is still being worked at.

llvm-svn: 154699
2012-04-13 20:40:52 +00:00
Johnny Chen 7bba2c6d64 The final batch of adding @dsym_test/@dwarf_test decorators.
llvm-svn: 154211
2012-04-06 21:33:58 +00:00
Johnny Chen c260232dfa Add comment for build_and_run_with_source_atoms_expr() and remove redundant #include from basic_type.cpp.
llvm-svn: 147895
2012-01-10 23:36:06 +00:00
Johnny Chen 0d81fe7a77 Add documentation for the generic functions build_and_run() and build_and_run_expr() called from
the Test*Types*.py test cases.

llvm-svn: 147893
2012-01-10 23:22:57 +00:00
Johnny Chen c256b99a17 Refactor the test/types directory to reduce some stupid reduant code.
Also add test cases to the test suite to exercise displaying of variables captured inside a block (Darwin-only).

llvm-svn: 147828
2012-01-10 02:04:04 +00:00
Johnny Chen c2a64b9b90 Fix comment.
llvm-svn: 147789
2012-01-09 19:19:23 +00:00
Johnny Chen b7373c92e6 o lldbtest.py:
Assign the test method name to self.testMethodName.  This can be useful for the
test directory (see test/types for a good example) which houses a bunch of executables
compiled from different source files.  The default build action is to create a.out as
the binary executable, which can confuse the module cacheing mechanism and result in
the debugger getting a stale image as the target to be debugged, and chaos ensues.

o AbstractBase.py, TestThreadAPI.py:

Use self.testMethodName to our advantage.

o TestLoadUnload.py:

Add expected failure marker to test case test_modules_search_paths().

llvm-svn: 133768
2011-06-23 22:11:20 +00:00
Johnny Chen 4acd7b4936 Refactorings of the test/types directory to more elegantly express the uniqueness of
executable names given to each test method.

llvm-svn: 133749
2011-06-23 20:10:23 +00:00
Johnny Chen bfae99cf29 Modify the test cases (there are a bunch of them) under test/types so that they
employ different executable names when creating the target for lldb to debug.
Comparing the log files for the same test case with success/failure results show
that the failed one was setting the breakpoint on some different address than the
succeeded one, leading us to suspect that the file system maybe think it knows better
and caches the wrong executable file (they were all named 'a.out' before) which lldb
subsequently reads from.

Now './dotest.py -v types' passes without specifying the '-w' option.

rdar://problem/9644488
rdar://problem/9649573

llvm-svn: 133649
2011-06-22 21:15:10 +00:00
Johnny Chen 554332d3a6 This newly introduced test file (for debugging purpose) needs to handle the case
where the environment variables for log files (DEBUG_LLDB_LOG and DEBUG_GDB_REMOTE_LOG)
are not defined.

llvm-svn: 133603
2011-06-22 02:49:56 +00:00
Johnny Chen 0ebe5c539f I messed up the test case id to turn logging on and then off. It is "test_long_type_with_dsym",
not "test_long_long_type_with_dsym".

llvm-svn: 133562
2011-06-21 22:40:34 +00:00
Johnny Chen 97d774076e Add logging of 'commands' category, too.
llvm-svn: 133556
2011-06-21 22:16:07 +00:00
Johnny Chen e1ba3123c1 rdar://problem/9649573 has captured the log files when running test_long_long_type_with_dsym() (which luckily failed) and the test session files for test failures.
llvm-svn: 133538
2011-06-21 19:59:45 +00:00
Johnny Chen fce70bb43b Add a test file "TestFailures.py" to try to capture test failures which occur when running
with no delay between successive test cases.  This one happend to capture a test failure where
the inferior shoud stop at a breakpoint but did not (radar will be filed to capture the log files
as well as the test session files):

[12:40:37] johnny:/Volumes/data/lldb/svn/trunk/test $ DEBUG_LLDB_LOG=/tmp/lldb.log DEBUG_GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v -p TestFailures.py
LLDB-63
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 133529
Node Kind: directory
Schedule: normal
Last Changed Author: jmolenda
Last Changed Rev: 133500
Last Changed Date: 2011-06-20 19:57:15 -0700 (Mon, 20 Jun 2011)



Session logs for test failures/errors/unexpected successes will go into directory '2011-06-21-12_40_42'
Command invoked: python ./dotest.py -v -p TestFailures.py
----------------------------------------------------------------------
Collected 10 tests

 1: test_char_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that char-type variables are displayed correctly. ... ok
 2: test_char_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that char-type variables are displayed correctly. ... ok
 3: test_int_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that int-type variables are displayed correctly. ... ok
 4: test_int_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that int-type variables are displayed correctly. ... ok
 5: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that 'long long'-type variables are displayed correctly. ... FAIL
 6: test_long_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that 'long long'-type variables are displayed correctly. ... ok
 7: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that long-type variables are displayed correctly. ... TestFailures.DebugIntegerTypesFailures.test_long_type_with_dsym
FAIL
 8: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that long-type variables are displayed correctly. ... FAIL
 9: test_short_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that short-type variables are displayed correctly. ... ok
10: test_short_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that short-type variables are displayed correctly. ... ok

======================================================================
FAIL: test_long_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that 'long long'-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 102, in test_long_long_type_with_dsym
    self.long_long_type()
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 129, in long_long_type
    self.generic_type_tester(set(['long long']))
  File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
    "stop reason = breakpoint"])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
    msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint

======================================================================
FAIL: test_long_type_with_dsym (TestFailures.DebugIntegerTypesFailures)
    Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 87, in test_long_type_with_dsym
    self.long_type()
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
    self.generic_type_tester(set(['long']))
  File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
    "stop reason = breakpoint"])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
    msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint

======================================================================
FAIL: test_long_type_with_dwarf (TestFailures.DebugIntegerTypesFailures)
    Test that long-type variables are displayed correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 94, in test_long_type_with_dwarf
    self.long_type()
  File "/Volumes/data/lldb/svn/trunk/test/types/TestFailures.py", line 125, in long_type
    self.generic_type_tester(set(['long']))
  File "/Volumes/data/lldb/svn/trunk/test/types/AbstractBase.py", line 67, in generic_type_tester
    "stop reason = breakpoint"])
  File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 863, in expect
    msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Process should be stopped due to breakpoint

----------------------------------------------------------------------
Ran 10 tests in 5.617s

FAILED (failures=3)
Session logs for test failures/errors/unexpected successes can be found in directory '2011-06-21-12_40_42'
[12:40:47] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 133537
2011-06-21 19:47:11 +00:00
Johnny Chen ed492022f4 For now, insert some delays between succesive test cases in the test/types directory
if not already specified by the test driver (via ./dotest -w).  Remove the AbstractBase.setUp()
method definition when/if we find out the cause of the failures if no delays are inserted
between these test cases.

llvm-svn: 133495
2011-06-21 00:53:00 +00:00
Johnny Chen e5cc4b6330 Simplify the the generic AbstractBase.py test base for testing variable types.
We still have the the issue where running:

    ./dotest.py -v types

we have test failures where the inferior either runs to exited with status 0 or
the inferior stops but not because of breakpoint (for example):

runCmd: process status
output: Process 90060 stopped
* thread #1: tid = 0x2d03, 0x000000010000e2ca, stop reason = EXC_BAD_ACCESS (code=2, address=0x10000e2ca)
  frame #0: 0x000000010000e2ca

There are two cases where the inferior stops for the breakpoint (good), but the expression parser
prints out the wrong information.  The two failures are:

Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dsym.log
Failure-TestFloatTypesExpr.FloatTypesExprTestCase.test_double_type_with_dwarf.log

I'll file a radar on the two expression parser misbehave, while continue investigating why the
inferior stops for the wrong reason or does not stop at all.

For now, you'll need to do:

    ./dotest.py -v -w types

llvm-svn: 133488
2011-06-21 00:13:15 +00:00
Jim Ingham f72ce3a216 Use the dyld_mode, image_infos & image_infos_count passed into the shared library notification function
to update libraries rather than reading the whole all_imaage_infos structure every time we get notified.

llvm-svn: 133448
2011-06-20 17:32:44 +00:00
Jim Ingham 2837b766f5 Change "frame var" over to using OptionGroups (and thus the OptionGroupVariableObjectDisplay).
Change the boolean "use_dynamic" over to a tri-state, no-dynamic, dynamic-w/o running target,
and dynamic with running target.

llvm-svn: 130832
2011-05-04 03:43:18 +00:00
Johnny Chen ac77f3b2ac Turns out that the test failure wrt:
rdar://problem/9173060 lldb hangs while running unique-types

disappears if running with clang version >= 3.  Modify the TestUniqueTypes.py
to detect if we are running with clang version < 3 and, if true, skip the test.

Update the lldbtest.system() function to return a tuple of (stdoutdata, stderrdata)
since we need the stderr data from "clang -v" command.  Modify existing clients of
lldbtest.system() to now use, for example:

         # First, capture the golden output emitted by the oracle, i.e., the
         # series of printf statements.
-        go = system("./a.out", sender=self)
+        go = system("./a.out", sender=self)[0]
         # This golden list contains a list of (variable, value) pairs extracted
         # from the golden output.
         gl = []

And add two utility functions to lldbutil.py.

llvm-svn: 128162
2011-03-23 20:28:59 +00:00
Johnny Chen 5f46b89df2 On second thought, inserting a delay of 1.0 second slows down the test suite
for the types directory quite a bit and masking out potential problems.  Enable
the delay via an environment variable, instead.

llvm-svn: 120719
2010-12-02 18:03:37 +00:00
Johnny Chen 560befce38 GenericTester.generic_type_expr_tester():
Don't overwhelm the expression mechanism.  Previously, there was no delay
    between successive invocations of 'expression' command.  The test suite
    run has experienced 'Resume timed out' or similar error messages.  Now we
    currently add a sleep of 1.0 second before issuing the next 'expression'
    command.

llvm-svn: 120626
2010-12-01 23:46:33 +00:00
Johnny Chen 1922a8dbea Remove @skip decorators for fixed crashes.
llvm-svn: 120479
2010-11-30 22:34:19 +00:00
Johnny Chen 876e6b31f5 Skip two test classes due to:
rdar://problem/8710994 Assertion failed: (reg_info) GDBRemoteRegisterContext.cpp, line 199

llvm-svn: 120462
2010-11-30 21:09:06 +00:00
Johnny Chen 94ce9711c1 We should make the class name of test cases which emphasize expression command
be different than the class name which uses frame variable command.

llvm-svn: 118602
2010-11-09 18:49:57 +00:00
Johnny Chen d2ed5dbf83 Clean up the test case by removing the filtering code related to fixed radars.
Also modify the:

    self.runCmd("expr %s" % var)

to be:

    self.runCmd("expression %s" % var)

since 'expr' is an alias of expression.  And fix the assert message to be either
'frame variable -t %s' or 'expression %s'.

llvm-svn: 118085
2010-11-02 22:01:37 +00:00
Johnny Chen fa3fa41d09 The remaining reference-related variable display failures have been fixed.
Comment out the filtering code.  The next check in will clean up the test case
by wiping off the filtering related logic.

llvm-svn: 118081
2010-11-02 21:43:50 +00:00
Johnny Chen 9a4c23635b Remove the reference-related variable displays that now pass from the 'notnow' set.
The failures that remain are:

        # rdar://problem/8620735 test/types: frame variable -t a_class_ref.m_b fails
        # The reference type related failures that remain are:
        notnow = set(['a_class_ref.m_b',
                      'a_struct_ref.b',
                      'a_union_nonzero_ref.u.a'])

llvm-svn: 118021
2010-11-02 17:06:08 +00:00
Johnny Chen 321ab7a1c8 Remove 8 @expectedFailure decorators, the following bug has been fixed:
# rdar://problem/8482903
    # test suite failure for types dir -- "long long" and "unsigned long long"

llvm-svn: 118017
2010-11-02 16:04:18 +00:00
Johnny Chen de7cd9229e Mark these test methods to be eligible for running only under the 'darwin' platform.
llvm-svn: 117071
2010-10-21 21:58:02 +00:00
Johnny Chen a295fd8045 Remove the two @skip decorators now that we are not seeing the hang while running the test suite.
llvm-svn: 116844
2010-10-19 18:43:57 +00:00
Johnny Chen 5517f64dae Uncomment the @skip decorator to reproduce the hang while running the test suite.
llvm-svn: 116763
2010-10-18 22:54:25 +00:00
Johnny Chen 690fcef762 Some re-achitecturing of the plugins interface. The caller is now required to
pass in a 'sender' arg to the buildDefault(), buildDsym(), buildDwarf(), and
cleanup() functions.  The sender arg will be the test instance itself (i.e.,
an instance of TestBase).  This is so that the relevant command execution can be
recorded in the TestBase.session object if sender is available.

The lldbtest.system() command has been modified to pop the 'sender' arg out of
the keyword arguments dictionary and use it as the test instance to facilitate
seesion recordings.  An example is in test/types/AbstractBase.py:

    def generic_type_tester(self, atoms, quotedDisplay=False):
        """Test that variables with basic types are displayed correctly."""

        # First, capture the golden output emitted by the oracle, i.e., the
        # series of printf statements.
        go = system("./a.out", sender=self)

There are cases when sender is None.  This is the case when the @classmethod is
involved in the use of these APIs.  When this happens, there is no recording
into a session object, but printing on the sys.stderr is still honored if the
trace flag is ON.

An example is in test/settings/TestSettings.py:

    @classmethod
    def classCleanup(cls):
        system(["/bin/sh", "-c", "rm -f output.txt"])
        system(["/bin/sh", "-c", "rm -f stdout.txt"])

llvm-svn: 116648
2010-10-15 23:55:05 +00:00
Johnny Chen 6e8ab6496a Fix these comments and the commented out code about 'frame variable -t', too.
llvm-svn: 116420
2010-10-13 19:25:42 +00:00
Johnny Chen 456c9c343e Apply (query-replace "frame variable" "frame variable -t") and fix a comment about 'expr var',
not 'frame variable var'.

llvm-svn: 116419
2010-10-13 19:22:50 +00:00
Johnny Chen 707b3c9c73 Added the capability for the test driver to relocate the tests and the intermediate
files to a different top level directory than those specified on the command line.

When relocated, the test clanups normally performed afterwards after each test method
and after each test class will not be exercised at all.  This allows for an easier
postmortem analysis of test failures.

Example:

./dotest.py -v -t -r /tmp/lldbtest types

will create a /tmp/lldbtest directory which houses the types directory and its supported
files.

Files modified:

o dotest.py, lldbtest.py:

  Add logic to process '-r dir' option to support relocating the tests to a different
  top level directory instead of exected in place.

o darwin.py, test/types/Makefile:

  The 'make clean' should only clean the minimum .o and .d files.

llvm-svn: 116255
2010-10-11 22:25:46 +00:00
Johnny Chen 2c2d69dff5 Change the regular expression used to grok the data type associated the output of
expression parser in light of the recent check ins.

Example:
    runCmd: expr a
    output: (double) $0 = 1100.12

llvm-svn: 115821
2010-10-06 20:00:48 +00:00
Johnny Chen 2f71831ceb Remove the @skip() and @expectedFaiure decorators, the previously affected tests
have now all passed.

rdar://problem/8502549 and rdar://problem/8493023 are fixed

llvm-svn: 115399
2010-10-02 01:46:20 +00:00
Johnny Chen b5051dc529 Added bug number to the @skip decorator.
llvm-svn: 115317
2010-10-01 16:34:53 +00:00
Johnny Chen 179543890e These currently crash (from running 'expr' command). Disable the entire class for the time being.
llvm-svn: 115316
2010-10-01 16:26:42 +00:00
Johnny Chen 9ea6e21f91 Updated to remove most of the @expectedFailure decorators for:
# rdar://problem/8493023
    # test/types failures for Test*TypesExpr.py: element offset computed wrong and sign error?

Two failures remain for test_short* test cases.

llvm-svn: 115229
2010-09-30 22:49:07 +00:00
Johnny Chen d8ed43cde0 Simple refactoring.
llvm-svn: 115075
2010-09-29 19:36:25 +00:00
Johnny Chen b19cd0fff4 Wrapped the regexp grokking of data type within a try:expect: instead of letting
the Python runtime take over in case there isn't a match.

llvm-svn: 115073
2010-09-29 19:29:42 +00:00
Johnny Chen 2c48dbec96 Added an example 'frame variable' output to demonstrate how the generic_type_tester
groks the data type of the variable.

llvm-svn: 115070
2010-09-29 19:12:10 +00:00
Johnny Chen c0ba019a19 Added @expectedFailure decorators for test suite failures:
# rdar://problem/8493023
    # test/types failures for Test*TypesExpr.py: element offset computed wrong?

llvm-svn: 115065
2010-09-29 19:02:20 +00:00