Summary: The COFF linker previously implemented link-time optimization using an API which has now been marked as legacy. This change refactors the COFF linker to use the new LTO API, which is also used by the ELF linker.
Reviewers: pcc, ruiu
Reviewed By: pcc
Subscribers: mgorny, mehdi_amini
Differential Revision: https://reviews.llvm.org/D29059
llvm-svn: 293967
Profiling revealed that the majority of lld's execution time on Windows was
spent opening and mapping input files. We can reduce this cost significantly
by performing these operations asynchronously.
This change introduces a queue for all operations on input file data. When
we discover that we need to load a file (for example, when we find a lazy
archive for an undefined symbol, or when we read a linker directive to
load a file from disk), the file operation is launched using a future and
the symbol resolution operation is enqueued. This implies another change
to symbol resolution semantics, but it seems to be harmless ("ninja All"
in Chromium still succeeds).
To measure the perf impact of this change I linked Chromium's chrome_child.dll
with both thin and fat archives.
Thin archives:
Before (median of 5 runs): 19.50s
After: 10.93s
Fat archives:
Before: 12.00s
After: 9.90s
On Linux I found that doing this asynchronously had a negative effect on
performance, probably because the cost of mapping a file is small enough that
it becomes outweighed by the cost of managing the futures. So on non-Windows
platforms I use the deferred execution strategy.
Differential Revision: https://reviews.llvm.org/D27768
llvm-svn: 289760
This ports the ELF linker's symbol table design, introduced in r268178,
to the COFF linker.
Differential Revision: http://reviews.llvm.org/D21166
llvm-svn: 289280
Previously, we had different way to stringize SymbolBody and InputFile
to construct error messages. This patch defines overloaded function
toString() so that we don't need to memorize all these different
function names.
With that change, it is now easy to include demangled names in error
messages. Now, if there is a symbol name conflict, we'll print out
both mangled and demangled names.
llvm-svn: 288992
Previously, we discarded .debug$ sections. This patch adds them to
files so that PDB.cpp can access them.
This patch also adds a debug option, /dumppdb, to dump debug info
fed to createPDB so that we can verify that valid data has been passed.
llvm-svn: 287555
LLD type-punned an integral type and a pointer type using a pointer
field. This is problematic because the pointer type has alignment
greater than some of the integral values.
This would be less problematic if a union was used but it turns out the
integral values are only present for a short, transient, amount of time.
Let's remove this undefined behavior by skipping the punning altogether
by storing the state in a separate memory location: a vector which
informs us which symbols to process for weak externs.
llvm-svn: 263918
There are some DLLs whose initializers depends on other DLLs'
initializers. The initialization order matters for them.
MSVC linker uses the order of the libraries from the command line.
LLD used ASCII-betical order. So they were incompatible.
This patch makes LLD compatible with MSVC.
llvm-svn: 245201
This has a few advantages
* Less C++ code (about 300 lines less).
* Less machine code (about 14 KB of text on a linux x86_64 build).
* It is more debugger friendly. Just set a breakpoint on the exit function and
you get the complete lld stack trace of when the error was found.
* It is a more robust API. The errors are handled early and we don't get a
std::error_code hot potato being passed around.
* In most cases the error function in a better position to print diagnostics
(it has more context).
llvm-svn: 244215
An object file compatible with Safe SEH contains a .sxdata section.
The section contains a list of symbol table indices, each of which
is an exception handler function. A safe SEH-enabled executable
contains a list of exception handler RVAs. So, what the linker has
to do to support Safe SEH is basically to read the .sxdata section,
interpret the contents as a list of symbol indices, unique-fy and
sort their RVAs, and then emit that list to .rdata. This patch
implements that feature.
llvm-svn: 243182
I am adding support for thin archives. On those, getting the buffer
involves reading another file.
Since we only need an id in here, use the member offset in the archive.
llvm-svn: 242205
clang-cl doesn't compile std::atomic_flag correctly (PR24101). Since the COFF
linker doesn't use threads yet, just revert r241420 and r241481 for now to
work around this clang-cl bug.
llvm-svn: 242006
Previously, we infer machine type at the very end of linking after
all symbols are resolved. That's actually too late because machine
type affects how we mangle symbols (whether or not we need to
add "_").
For example, /entry:foo adds "_foo" to the symbol table if x86 but
"foo" if x64.
This patch moves the code to infer machine type, so that machine
type is inferred based on input files given via the command line
(but not based on .directives files).
llvm-svn: 241843
The previous code was not even safe with MSVC 2013 because the compiler
doesn't guarantee that static variables (in this case, a mutex) are
initialized in a thread-safe manner.
llvm-svn: 241481
Previously, SymbolBody::compare(A, B) didn't satisfy weak ordering.
There was a case that A < B and B < A could have been true.
This is because we just pick LHS if A and B are consisdered equivalent.
This patch is to make symbols being weakly ordered. If A and B are
not tie, one of A < B && B > A or A > B && B < A is true.
This is not an improtant property for a single-threaded environment
because everything is deterministic anyways. However, in a multi-
threaded environment, this property becomes important.
If a symbol is defined or lazy, ties are resolved by its file index.
For simple types that we don't really care about their identities,
symbols are compared by their addresses.
llvm-svn: 241294
Previously, the order of adding symbols to the symbol table was simple.
We have a list of all input files. We read each file from beginning of
the list and add all symbols in it to the symbol table.
This patch changes that order. Now all archive files are added to the
symbol table first, and then all the other object files are added.
This shouldn't change the behavior in single-threading, and make room
to parallelize in multi-threading.
In the first step, only lazy symbols are added to the symbol table
because archives contain only Lazy symbols. Member object files
found to be necessary are queued. In the second step, defined and
undefined symbols are added from object files. Adding an undefined
symbol to the symbol table may cause more member files to be added
to the queue. We simply continue reading all object files until the
queue is empty.
Finally, new archive or object files may be added to the queues by
object files' directive sections (which contain new command line
options).
The above process is repeated until we get no new files.
Symbols defined both in object files and in archives can make results
undeterministic. If an archive is read before an object, a new member
file gets linked, while in the other way, no new file would be added.
That is the most popular cause of an undeterministic result or linking
failure as I observed. Separating phases of adding lazy symbols and
undefined symbols makes that deterministic. Adding symbols in each
phase should be parallelizable.
llvm-svn: 241107
method to get a SymbolBody and into the callers, and kill now dead
includes.
This removes the need to have the SymbolBody definition when we're
defining the inline method and makes it a better inline method. That was
the only reason for a lot of header includes here. Removing these and
using forward declarations actually uncovers a bunch of cross-header
dependencies that I've fixed while I'm here, and will allow me to
introduce some *important* inline code into Chunks.h that requires the
definition of ObjectFile.
No functionality changed at this point.
Differential Revision: http://reviews.llvm.org/D10789
llvm-svn: 240982
This function is actually *very* hot. It is hard to see currently
because the call graph is very recursive, but I'm working to remove that
and when I do this function becomes significantly higher on the profile
(up to 5%!) and so worth avoiding the call overhead.
No specific perf gain I can measure yet (below the noise), but likely to
have more impact as we stop cluttering the call graph.
Differential Revision: http://reviews.llvm.org/D10788
llvm-svn: 240873
This change seems to make the linker about 10% faster.
Reading symbol name is not very cheap because it needs strlen()
on the string table. We were wasting time on reading non-external
symbol names that would never be used by the linker.
llvm-svn: 239332
In r238690, I made all files have only MemoryBufferRefs. This change
is to do the same thing for the bitcode file reader. Also updated
a few variable names to match with other code.
llvm-svn: 238782
Previously, a MemoryBuffer of a file was owned by each InputFile object.
This patch makes the Driver own all of them. InputFiles now have only
MemoryBufferRefs. This change simplifies ownership managment
(particularly for ObjectFile -- the object owned a MemoryBuffer only when
it's not created from an archive file, because in that case a parent
archive file owned the entire buffer. Now it owns nothing unconditionally.)
llvm-svn: 238690
This is an initial patch for a section-based COFF linker.
The patch has 2300 lines of code including comments and blank lines.
Before diving into details, you want to start from reading README
because it should give you an overview of the design.
All important things are written in the README file, so I write
summary here.
- The linker is already able to self-link on Windows.
- It's significantly faster than the existing implementation.
The existing one takes 5 seconds to link LLD on my machine,
while the new one only takes 1.2 seconds, even though the new
one is not multi-threaded yet. (And a proof-of-concept multi-
threaded version was able to link it in 0.5 seconds.)
- It uses much less memory (250MB vs. 2GB virtual memory space
to self-host).
- IMHO the new code is much simpler and easier to read than
the existing PE/COFF port.
http://reviews.llvm.org/D10036
llvm-svn: 238458
Bob Haarman