replica
/
hanchenye-llvm-project
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
hanchenye-llvm-project/clang/test/Lexer/utf8-invalid.c

16 lines
393 B
C
Raw Normal View History

Handle universal character names and Unicode characters outside of literals. This is a missing piece for C99 conformance. This patch handles UCNs by adding a '\\' case to LexTokenInternal and LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN. If the UCN is not syntactically well-formed, we fall back to the old treatment: a backslash followed by an identifier beginning with 'u' (or 'U'). Because the spelling of an identifier with UCNs still has the UCN in it, we need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo. Of course, valid code that does *not* use UCNs will see only a very minimal performance hit (checks after each identifier for non-ASCII characters, checks when converting raw_identifiers to identifiers that they do not contain UCNs, and checks when getting the spelling of an identifier that it does not contain a UCN). This patch also adds basic support for actual UTF-8 in the source. This is treated almost exactly the same as UCNs except that we consider stray Unicode characters to be mistakes and offer a fixit to remove them. llvm-svn: 173369
2013-01-25 04:50:46 +08:00
// RUN: %clang_cc1 -fsyntax-only -verify %s
Fix r173881 to properly skip invalid UTF-8 characters in raw lexing and -E. This caused hangs as we processed the same invalid byte over and over. <rdar://problem/13115651> llvm-svn: 173959
2013-01-31 03:21:12 +08:00
// RUN: %clang_cc1 -E %s -o /dev/null
Handle universal character names and Unicode characters outside of literals. This is a missing piece for C99 conformance. This patch handles UCNs by adding a '\\' case to LexTokenInternal and LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN. If the UCN is not syntactically well-formed, we fall back to the old treatment: a backslash followed by an identifier beginning with 'u' (or 'U'). Because the spelling of an identifier with UCNs still has the UCN in it, we need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo. Of course, valid code that does *not* use UCNs will see only a very minimal performance hit (checks after each identifier for non-ASCII characters, checks when converting raw_identifiers to identifiers that they do not contain UCNs, and checks when getting the spelling of an identifier that it does not contain a UCN). This patch also adds basic support for actual UTF-8 in the source. This is treated almost exactly the same as UCNs except that we consider stray Unicode characters to be mistakes and offer a fixit to remove them. llvm-svn: 173369
2013-01-25 04:50:46 +08:00
Fix comment in test/Lexer/utf8-invalid.c for updates in r173959. llvm-svn: 173961
2013-01-31 03:29:14 +08:00
// Note: This file deliberately contains invalid UTF-8. Please do not fix!
Handle universal character names and Unicode characters outside of literals. This is a missing piece for C99 conformance. This patch handles UCNs by adding a '\\' case to LexTokenInternal and LexIdentifier -- if we see a backslash, we tentatively try to read in a UCN. If the UCN is not syntactically well-formed, we fall back to the old treatment: a backslash followed by an identifier beginning with 'u' (or 'U'). Because the spelling of an identifier with UCNs still has the UCN in it, we need to convert that to UTF-8 in Preprocessor::LookUpIdentifierInfo. Of course, valid code that does *not* use UCNs will see only a very minimal performance hit (checks after each identifier for non-ASCII characters, checks when converting raw_identifiers to identifiers that they do not contain UCNs, and checks when getting the spelling of an identifier that it does not contain a UCN). This patch also adds basic support for actual UTF-8 in the source. This is treated almost exactly the same as UCNs except that we consider stray Unicode characters to be mistakes and offer a fixit to remove them. llvm-svn: 173369
2013-01-25 04:50:46 +08:00
extern int <EFBFBD>x; // expected-error{{source file is not valid UTF-8}}
Fix r173881 to properly skip invalid UTF-8 characters in raw lexing and -E. This caused hangs as we processed the same invalid byte over and over. <rdar://problem/13115651> llvm-svn: 173959
2013-01-31 03:21:12 +08:00
#if 0
// Don't warn about bad UTF-8 in raw lexing mode.
extern int <EFBFBD>x;
#endif
Lexer: Don't warn about Unicode in preprocessor directives. This allows people to use Unicode in their #pragma mark and in macros that exist only to be string-ized. <rdar://problem/13107323&13121362> llvm-svn: 174081
2013-02-01 03:48:48 +08:00
// Don't warn about bad UTF-8 in preprocessor directives.
#define x82 <20>
#pragma mark <20>