All readers except PE/COFF reader create layout-after edges to preserve
the original symbol order. PE/COFF uses layout-before edges as primary
edges for no reason.
This patch makes PE/COFF reader to create layout-after edges.
Resolver is updated to recognize reverse edges of layout-after edges
in the garbage collection pass.
Now we can retire layout-before edges. I don't do that in this patch
because if I do, I would have updated many tests to replace all
occurrrences of "layout-before" with "layout-after". So that's a TODO.
llvm-svn: 231615
Atoms with fallback atoms are never be added to the symbol table.
However, we added such atoms to _undefines array. We had to call
isCoalescedAway to identify and skip them. We should just stop
adding them in the first place.
This seems to make the linker ~1% faster in my test case.
llvm-svn: 231552
If an undefined symbol is added to the symbol table by the previous
call of SymbolTable::add, SymbolTable::isDefined will always return
false for the same symbol.
llvm-svn: 231551
This is yet another optimization patch. Previously we called
SymbolTable::isDefined() and SymbolTable::findByName() from a very
frequently executed function. Because isDefined calls findByName,
findByName is called twice on each iteration.
findByName is not a cheap function. It computes a hash value for a
given symbol name. When linking C++ programs, it can be expensive
because of C++ mangled long symbols.
This patch reduces the number of call from 2 to 1. Performance
improvements by this patch was larger than I expected. Linking time
of chrome.dll gets almost 5% shorter.
llvm-svn: 231549
In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
find() function to lookup a symbol.
Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.
find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.
We can eliminate the bottleneck.
Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).
In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.
This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result of _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.
The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 70 seconds. Now it takes (only?) 30 seconds!
http://reviews.llvm.org/D8091
llvm-svn: 231434
_reverseRef is a multimap from atoms to atoms. The map contains
reverse edges of "layout-before" and "group" edges for dead-stripping.
The type of the variable was DenseMap<Atom *, DenseSet<Atom *>>.
This patch changes that to std::unordered_multimap<Atom *, Atom *>.
A DenseMap with a value type of DenseSet was not fast. Inserting 900k
items to the map took about 1.6 seconds on my workstation.
unordered_multimap on the other hand took only 0.6 seconds.
Use of the map also got faster -- originally markLive took 1.3 seconds
in the same test case, and it now took 1.0 seconds. In total we shove
off 1.3 seconds out of 27 seconds in that test case.
llvm-svn: 231432
We maintain a map from symbols to archive files for the archive file
pre-loading. That map is created at the beginning of the resolve()
and is never updated. However, the input file list may be updated by
File::beforeLink(). This is a patch to update the map after beforeLink.
llvm-svn: 231395
File objects are not really const in the resolver. We set ordinals to
them and call beforeLink hooks. Also, File's member functions marked
as const are not really const. ArchiveFile never returns the same
member file twice, so it remembers files returned before. find() has
side effects.
In order to deal with the inconsistencies, we sprinkled const_casts
and marked member varaibles as mutable.
This patch removes const from there to reflect the reality.
llvm-svn: 231212
std::promise and std::future in old version of libstdc++ are buggy.
I think that's the reason why LLD tests were flaky on Ubuntu 13
buildbots until we disabled file preloading.
In this patch, I implemented very simple future and used that in
FileArchive. Compared to std::promise and std::future, it lacks
many features, but should serve our purpose.
http://reviews.llvm.org/D8025
llvm-svn: 231153
Previously we didn't call the hook on a file in an archive, which
let the PE/COFF port fail to link files in archives. It was a
simple mistake. Added a call to the hook and also added a test to
catch that error.
const_cast is an unfortunate hack. Files in the resolver are usually
const, but they are not actually const objects, since they are
mutated if either a file is taken from an archive (an archive file
does never return the same file twice) or the beforeLink hook is
called. Maybe we should just remove const from there -- because they
are not const.
llvm-svn: 230808
It is observed that the function throws std::future_error on a few buildbots.
That cannot be easily reproducible on local machines. Kill the feature
temporarily to see if this is going to fix the buildbot issue.
llvm-svn: 230735
This fixes a linker crash (found out while testing --gc-sections,
testcase provided by Rafael Avila de Espindola).
While this behaviour was found while testing ELF, it' not necessarily
ELF specific and this change is (apparently) harmless on all the
other drivers.
Differential Revision: D7823
Reviewed by: ruiu
llvm-svn: 230614
I believe the original code is valid, but on Windows it failed with an
assertion error saying "Expression: vector iterator is not decrementable."
Don't use rbegin and rend to workaround that error.
llvm-svn: 226706
We used to manage the state whether we are in a group or not
using a counter. The counter is incremented by one if we jump from
end-group to start-group, and decremented by one if we don't.
The counter was assumed to be either zero or one, but obviously it
could be negative (if there's a group which is not repeated at all).
This is a fix for that issue.
llvm-svn: 226632
LLD parses archive file index table only at first. When it finds a symbol
it is looking for is defined in a member file in an archive file, it actually
reads the member from the archive file. That's done in the core linker.
That's a single-thread process since the core linker is single threaded.
If your command line contains a few object files and a lot of archive files
(which is quite often the case), LLD hardly utilizes hardware parallelism.
This patch improves parallelism by speculatively instantiating archive
file members. At the beginning of the core linking, we first create a map
containing all symbols defined in all members, and each time we find a
new undefined symbol, we instantiate a member file containing the
symbol (if such file exists). File instantiation is side effect free, so this
should not affect correctness.
This is a quick benchmark result. Time to link self-link LLD executable:
Linux 9.78s -> 8.50s (0.86x)
Windows 6.18s -> 4.51s (0.73x)
http://reviews.llvm.org/D7015
llvm-svn: 226336
This patch makes File::parse() multi-thread safe. If one thread is running
File::parse(), other threads will block if they try to call the same method.
File::parse() is idempotent, so you can safely call multiple times.
With this change, we don't have to wait for all worker threads to finish
in Driver::link(). Previously, Driver::link() calls TaskGroup::sync() to
wait for all threads running File::parse(). This was not ideal because
we couldn't start the resolver until we parse all files.
This patch increase parallelism by making Driver::link() to not wait for
worker threads. The resolver calls parse() to make sure that the file
being read has been parsed, and then uses the file. In this approach,
the resolver can run with the parser threads in parallel.
http://reviews.llvm.org/D6994
llvm-svn: 226281
InputElement was named that because it's an element of an InputGraph.
It's losing the origin because the InputGraph is now being removed.
InputElement's subclass is FileNode, that naming inconsistency needed
to be fixed.
llvm-svn: 226147
These changes depended on r225674 and had been rolled back in r225859.
Because r225674 has been re-submitted, it's safe to re-submit them.
llvm-svn: 226132
r225764 broke a basic functionality on Mac OS. This change reverts
r225764, r225766, r225767, r225769, r225814, r225816, r225829, and r225832.
llvm-svn: 225859
getNextFile used to have a complex logic to determine which file
should be processed by the Resolver on next iteration.
Now, it is just a sequential accessor to the internal array and
provides no sensible feature.
This patch also removes InputGraph::getGroupSize and InputGraph::
skipGroup to simplify the code.
llvm-svn: 225832
These member functions returns either no_more_files error or a File object.
We could simply return a nullptr instead of a no_more_files.
This function will be removed soon as a part of InputGraph cleanup.
I had to do that step by step.
llvm-svn: 224208
This reverts commit r223330 because it broke Darwin and ELF
linkers in a way that we couldn't have caught with the existing
test cases.
llvm-svn: 223373
The aim of this patch is to reduce the excessive abstraction from
the InputGraph. We found that even a simple thing, such as sorting
input files (Mach-O) or adding a new file to the input file list
(PE/COFF), is nearly impossible with the InputGraph abstraction,
because it hides too much information behind it. As a result,
we invented complex interactions between components (e.g.
notifyProgress() mechanism) and tricky code to work around that
limitation. There were many occasions that we needed to write
awkward code.
This patch is a first step to make it cleaner. As a first step,
this removes Group class from the InputGraph. The grouping feature
is now directly handled by the Resolver. notifyProgress is removed
since we no longer need that. I could have cleaned it up even more,
but in order to keep the patch minimum, I focused on Group.
SimpleFileNode class, a container of File objects, is now limited
to have only one File. We shold have done this earlier.
We used to allow putting multiple File objects to FileNode.
Although SimpleFileNode usually has only one file, the Driver class
actually used that capability. I modified the Driver class a bit,
so that one FileNode is created for each input File.
We should now probably remove SimpleFileNode and directly store
File objects to the InputGraph in some way, because a container
that can contain only one object is useless. This is a TODO.
Mach-O input files are now sorted before they are passe to the
Resolver. DarwinInputGraph class is no longer needed, so removed.
PECOFF still has hacky code to add a new file to the input file list.
This will be cleaned up in another patch.
llvm-svn: 223330
The job of the CompactUnwind pass is to turn __compact_unwind data (and
__eh_frame) into the compressed final form in __unwind_info. After it's done,
the original atoms are no longer relevant and should be deleted (they cause
problems during actual execution, quite apart from the fact that they're not
needed).
llvm-svn: 221301
The darwin linker has the -demangle option which directs it to demangle C++
(and soon Swift) mangled symbol names. Long term we need some Diagnostics object
for formatting errors and warnings. But for now we have the Core linker just
writing messages to llvm::errs(). So, to enable demangling, I changed the
Resolver to call a LinkingContext method on the symbol name.
To make this more interesting, the demangling code is done via __cxa_demangle()
which is part of the C++ ABI, which is only supported on some platforms, so I
had to conditionalize the code with the config generated HAVE_CXXABI_H.
llvm-svn: 218718
COFF supports a feature similar to ELF's section groups. This
patch implements it.
In ELF, section groups are identified by their names, and they are
treated somewhat differently from regular symbols. In COFF, the
feature is realized in a more straightforward way. A section can
have an annotation saying "if Nth section is linked, link this
section too."
I added a new reference type, kindAssociate. If a target atom is
coalesced away, the referring atom is removed by Resolver, so that
they are treated as a group.
Differential Revision: http://reviews.llvm.org/D4028
llvm-svn: 211106
isCoalescedAway(x) is faster than replacement(x) != x as the former
does not follow the replacement atom chain. Also it's easier to use.
llvm-svn: 210242
Previously section groups are doubly linked to their children.
That is, an atom representing a group has group-child references
to its group contents, and content atoms also have group-parent
references to the group atom. That relationship was invariant;
if X has a group-child edge to Y, Y must have a group-parent
edge to X.
However we were not using group-parent references at all. The
resolver only needs group-child edges.
This patch simplifies the section group by removing the unused
reverse edge. No functionality change intended.
Differential Revision: http://reviews.llvm.org/D3945
llvm-svn: 210066
Layout-before edges are no longer used for layout, but they are
still there for dead-stripping. If we would just remove them
from code, LLD would wrongly remove live atoms that were
referenced by layout-befores.
This patch fixes the issue. Before dead-stripping, it scans all
atoms to construct a reverse map for layout-after edges. Dead-
stripping pass uses the map to traverse the graph.
Differential Revision: http://reviews.llvm.org/D3986
llvm-svn: 210057
Reference::target() never returns a nullptr, so NULL check
is not needed and is more harmful than doing nothing.
No functionality change.
llvm-svn: 210008
In r205566, I made a change to Resolver so that Resolver revisit
only archive files in --start-group and --end-group pair. That's
not correct, as it also has to revisit DSO files.
This patch is to fix the issue.
Added a test to demonstrate the fix. I confirmed that it succeeded
before r205566, failed after r205566, and is ok with this patch.
Differential Revision: http://reviews.llvm.org/D3734
llvm-svn: 208797
Rui Ueyama