AST file more lazy, so that we don't eagerly load that information for
all known identifiers each time a new AST file is loaded. The eager
reloading made some sense in the context of precompiled headers, since
very few identifiers were defined before PCH load time. With modules,
however, a huge amount of code can get parsed before we see an
@import, so laziness becomes important here.
The approach taken to make this information lazy is fairly simple:
when we load a new AST file, we mark all of the existing identifiers
as being out-of-date. Whenever we want to access information that may
come from an AST (e.g., whether the identifier has a macro definition,
or what top-level declarations have that name), we check the
out-of-date bit and, if it's set, ask the AST reader to update the
IdentifierInfo from the AST files. The update is a merge, and we now
take care to merge declarations before/after imports with declarations
from multiple imports.
The results of this optimization are fairly dramatic. On a small
application that brings in 14 non-trivial modules, this takes modules
from being > 3x slower than a "perfect" PCH file down to 30% slower
for a full rebuild. A partial rebuild (where the PCH file or modules
can be re-used) is down to 7% slower. Making the PCH file just a
little imperfect (e.g., adding two smallish modules used by a bunch of
.m files that aren't in the PCH file) tips the scales in favor of the
modules approach, with 24% faster partial rebuilds.
This is just a first step; the lazy scheme could possibly be improved
by adding versioning, so we don't search into modules we already
searched. Moreover, we'll need similar lazy schemes for all of the
other lookup data structures, such as DeclContexts.
llvm-svn: 143100
as part of the hash rather than ignoring them. This means we'll end up
building more module variants (overall), but it allows configuration
macros such as NDEBUG to work so long as they're specified via command
line. More to come in this space.
llvm-svn: 142187
the AST reader), merge that header file information with whatever
header file information we already have. Otherwise, we might forget
something we already knew (e.g., that the header was #import'd already).
llvm-svn: 139979
arbitrary amount of code. This forces us to stage the AST writer more
strictly, ensuring that we don't assign a declaration ID to a
declaration until after we're certain that no more modules will get
loaded.
llvm-svn: 139974
target triple to separate modules built under different
conditions. The hash is used to create a subdirectory in the module
cache path where other invocations of the compiler (with the same
version, language options, etc.) can find the precompiled modules.
llvm-svn: 139662
but there is a corresponding umbrella header in a framework, build the
module on-the-fly so it can be immediately loaded at the import
statement. This is very much proof-of-concept code, with details to be
fleshed out over time.
llvm-svn: 139558
where the compiler will look for module files. Eliminates the
egregious hack where we looked into the header search paths for
modules.
llvm-svn: 139538
include guards don't show up as macro definitions in every translation
unit that imports a module. Macro definitions can, however, be
exported with the intentionally-ugly #__export_macro__
directive. Implement this feature by not even bothering to serialize
non-exported macros to a module, because clients of that module need
not (should not) know that these macros even exist.
llvm-svn: 138943
The initial incentive was to fix a crash when PCH chaining categories
to an interface, but the fix was done in the "modules way" that I hear
is popular with the kids these days.
Each module stores the local chain of categories and we combine them
when the interface is loaded. We also warn if non-dependent modules
introduce duplicate named categories.
llvm-svn: 138926
existing practice with Python extension modules. Not that Python
extension modules should be using a double-underscored identifier
anyway, but...
llvm-svn: 138870
__import__ within the preprocessor, since the prior one foolishly
assumed that Preprocessor::Lex() was re-entrant. We now handle
__import__ at the top level (only), after macro expansion. This should
fix the buildbot failures.
llvm-svn: 138704
loads the named module. The syntax itself is intentionally hideous and
will be replaced at some later point with something more
palatable. For now, we're focusing on the semantics:
- Module imports are handled first by the preprocessor (to get macro
definitions) and then the same tokens are also handled by the parser
(to get declarations). If both happen (as in normal compilation),
the second one is redundant, because we currently have no way to
hide macros or declarations when loading a module. Chris gets credit
for this mad-but-workable scheme.
- The Preprocessor now holds on to a reference to a module loader,
which is responsible for loading named modules. CompilerInstance is
the only important module loader: it now knows how to create and
wire up an AST reader on demand to actually perform the module load.
- We search for modules in the include path, using the module name
with the suffix ".pcm" (precompiled module) for the file name. This
is a temporary hack; we hope to improve the situation in the
future.
llvm-svn: 138679
from the given source. -emit-module behaves similarly to -emit-pch,
except that Sema is somewhat more strict about the contents of
-emit-module. In the future, there are likely to be more interesting
differences.
llvm-svn: 138595
given selector, rather than walking the chain backwards. Teach its
visitor how to merge multiple result sets into a single result set,
combining the results of selector lookup in several different modules
into a single result set.
llvm-svn: 138556
which supports both pre-order and post-order traversal via a visitor
mechanism. Use this depth-first search with a post-order traversal to
give predictable ordering semantics when walking all of the lexical
declarations in the translation unit.
Eventually, module imports will occur in the source code rather than
at the beginning, and we'll have to revisit this walk.
llvm-svn: 138490
module DAG-based lookup scheme. This required some reshuffling, so
that each module stores its own mapping from DeclContexts to their
lexical and visible sets for those DeclContexts (rather than one big
"chain").
Overall, this allows simple qualified name lookup into the translation
unit to gather results from multiple modules, with the lookup results
in module B shadowing the lookup results in module A when B imports A.
Walking all of the lexical declarations in a module DAG is still a
mess; we'll end up walking the loaded module list backwards, which
works fine for chained PCH but doesn't make sense in a DAG. I'll
tackle this issue as a separate commit.
llvm-svn: 138463
different modules) more robust. It already handled (simple) merges of
the set of declarations attached to that identifier, so add a test
case that shows us getting two different declarations for the same
identifier (one struct, one function) from different modules, and are
able to use both of them.
llvm-svn: 138189
modules (those that no other module depends on) and performs a search
over all of the modules, visiting a new module only when all of the
modules that depend on it have already been visited. The visitor can
abort the search for all modules that a module depends on, which
allows us to minimize the number of lookups necessary when performing
a search.
Switch identifier lookup from a linear walk over the set of modules to
this module visitation operation. The behavior is the same for simple
PCH and chained PCH, but provides the proper search order for
modules. Verified with printf debugging, since we don't have enough in
place to actually test this.
llvm-svn: 138187
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
Note that this version moves the file-opening logic into the module
manager, rather than splitting it between the module manager and the
AST reader. More importantly, it properly handles the
weird-but-possibly-useful case of loading an AST file from "-".
llvm-svn: 138030
Teach ModuleManager::addModule() to check whether a particular module
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
llvm-svn: 137971
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
llvm-svn: 137925
Douglas Gregor