Commit Graph

12 Commits

Author SHA1 Message Date
Douglas Gregor c998409cce Remove an overly-eager assertion when replacing tokens with an
annotation token, because some of the tokens we're annotating might
not be in the set of cached tokens (we could have consumed them
unconditionally).

Also, move the tentative parsing from ParseTemplateTemplateArgument
into the one caller that needs it, improving recovery.

llvm-svn: 86904
2009-11-12 00:03:40 +00:00
Mike Stump 11289f4280 Remove tabs, and whitespace cleanups.
llvm-svn: 81346
2009-09-09 15:08:12 +00:00
Nuno Lopes bd2cd92907 fix segfault (because of erasing after the vector boundaries) when the cached token position is at the end
llvm-svn: 77159
2009-07-26 16:36:45 +00:00
Chris Lattner 5a7971e0c3 This change refactors some of the low-level lexer interfaces a bit.
Token now has a class of kinds for "literals", which include 
numeric constants, strings, etc.  These tokens can optionally have
a pointer to the start of the token in the lexer buffer.  This 
makes it faster to get spelling and do other gymnastics, because we
don't have to go through source locations.

This change is performance neutral, but will make other changes
more feasible down the road.

llvm-svn: 63028
2009-01-26 19:29:26 +00:00
Argyrios Kyrtzidis f5e2812e69 Remove Preprocessor::CacheTokens boolean data member. The same functionality can be provided by using Preprocessor::isBacktrackEnabled().
llvm-svn: 59631
2008-11-19 14:23:14 +00:00
Ted Kremenek 50b4f48225 Use PushIncludeMacroStack() instead of manually manipulating the include stack.
llvm-svn: 59181
2008-11-12 22:21:57 +00:00
Argyrios Kyrtzidis c7e67a04c3 Introduce annotation tokens, a special kind of token, created and used only by the parser to replace a group of tokens with a single token encoding semantic information.
Will be fully utilized later for C++ nested-name-specifiers.

llvm-svn: 58911
2008-11-08 16:17:04 +00:00
Argyrios Kyrtzidis 91c3f526dc Line endings: CRLF -> LF
llvm-svn: 55829
2008-09-05 08:53:53 +00:00
Argyrios Kyrtzidis 75b34536d0 Rename Preprocessor::DisableBacktrack -> Preprocessor::CommitBacktrackedTokens.
llvm-svn: 55281
2008-08-24 12:29:43 +00:00
Argyrios Kyrtzidis bd024c7fdb Change line endings: CRLF -> LF
llvm-svn: 55235
2008-08-23 12:05:53 +00:00
Argyrios Kyrtzidis a65490c5df Allow nested backtracks.
llvm-svn: 55204
2008-08-22 21:27:50 +00:00
Argyrios Kyrtzidis b3dd1e0889 Allow the preprocessor to cache the lexed tokens, so that we can do efficient lookahead and backtracking.
1) New public methods added:
  -EnableBacktrackAtThisPos
  -DisableBacktrack
  -Backtrack
  -isBacktrackEnabled

2) LookAhead() implementation is replaced with a more efficient one.
3) LookNext() is removed.

llvm-svn: 54611
2008-08-10 13:15:22 +00:00