Auto merge of #89819 - davidtwco:issue-81024-multiple-crates-multiple-dwarves, r=nagisa

cg: split dwarf for crate dependencies

Fixes #81024.

- In #79570, `-Z split-dwarf-kind={none,single,split}` was replaced by `-C split-debuginfo={off,packed,unpacked}`. `-C split-debuginfo`'s packed and unpacked aren't exact parallels to single and split, respectively.

  On Unix, `-C split-debuginfo=packed` will put debuginfo in object files and package debuginfo into a DWARF package file (`.dwp`) and `-C split-debuginfo=unpacked` will put debuginfo in dwarf object files and won't package it.

  In the initial implementation of Split DWARF, split mode wrote sections which did not require relocation into a DWARF object (`.dwo`) file which was ignored by the linker and then packaged those DWARF objects into DWARF packages (`.dwp`). In single mode, sections which did not require relocation were written into object files but ignored by the linker and were not packaged. However, both split and single modes could be packaged or not, the primary difference in behaviour was where the debuginfo sections that did not require link-time relocation were written (in a DWARF object or the object file).

  In the first commit of this PR, I re-introduce a `-Z split-dwarf-kind` flag, which can be used to pick between split and single modes when `-C split-debuginfo` is used to enable Split DWARF (either packed or unpacked).
- Split DWARF packaging requires all of the object files to exist, including those in dependencies. ~~Therefore, the second commit of this PR makes rustc keep all objects or dwarf objects for unpacked mode and if the crate is a dependency in packed mode (determined by heuristic: if no linking is taking place), then objects or dwarf objects are kept. Objects are kept if `-Z split-dwarf-kind` is `SplitDwarfKind::Single`, and dwarf objects if `SplitDwarfKind::Split`.~~

  ~~There are other approaches that could be taken to supporting packed Split DWARF with crate dependencies but this seemed like the least complicated and was contained to only rustc. Other potential approaches are described in https://github.com/rust-lang/rust/issues/81024#issuecomment-760478223, I'm happy to change the approach I've taken here if it isn't what we're looking for.~~ See https://github.com/rust-lang/rust/pull/89819#issuecomment-985671867 for the current approach.
- ~~There's still a dependency on `llvm-dwp` after this change, which [we probably want to move away from](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/llvm-dwp.20is.20not.20recommended) but that seems out-of-scope for this PR. Ideally, Split DWARF (in packed or unpacked modes) will be usable on nightly after this lands. If there aren't any bugs reported then it's possible we could allow Split DWARF to be used on stable after this change, it depends whether or not switching away from `llvm-dwp` later would break any guarantees, or whether we'd want to change how we handle this cross-crate case in future.~~ See https://github.com/rust-lang/rust/pull/89819#issuecomment-985671867.

r? `@nagisa`
cc `@alexcrichton`
This commit is contained in:
bors 2022-01-06 12:21:10 +00:00
commit a77cc64af4
18 changed files with 443 additions and 162 deletions

View File

@ -9,7 +9,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e61f2b7f93d2c7d2b08263acaa4a363b3e276806c68af6134c44f523bf1aacd"
dependencies = [
"compiler_builtins",
"gimli",
"gimli 0.25.0",
"rustc-std-workspace-alloc",
"rustc-std-workspace-core",
]
@ -87,9 +87,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.34"
version = "1.0.51"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf8dcb5b4bbaa28653b647d8c77bd4ed40183b48882e130c1f1ffb73de069fd7"
checksum = "8b26702f315f53b6071259e15dd9d64528213b44d61de1ec926eca7715d62203"
[[package]]
name = "array_tool"
@ -1158,6 +1158,12 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e88a8acf291dafb59c2d96e8f59828f3838bb1a70398823ade51a84de6a6deed"
[[package]]
name = "fallible-iterator"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
[[package]]
name = "filetime"
version = "0.2.14"
@ -1446,6 +1452,17 @@ dependencies = [
"rustc-std-workspace-core",
]
[[package]]
name = "gimli"
version = "0.26.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78cc372d058dcf6d5ecd98510e7fbc9e5aec4d21de70f65fea8fecebcd881bd4"
dependencies = [
"fallible-iterator",
"indexmap",
"stable_deref_trait",
]
[[package]]
name = "git2"
version = "0.13.23"
@ -2339,6 +2356,18 @@ dependencies = [
"rustc-std-workspace-core",
]
[[package]]
name = "object"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67ac1d3f9a1d3616fd9a60c8d74296f22406a238b6a72f5cc1e6f314df4ffbf9"
dependencies = [
"crc32fast",
"flate2",
"indexmap",
"memchr",
]
[[package]]
name = "odht"
version = "0.3.1"
@ -3725,10 +3754,11 @@ dependencies = [
"itertools 0.9.0",
"jobserver",
"libc",
"object",
"object 0.26.2",
"pathdiff",
"regex",
"rustc_apfloat",
"rustc_arena",
"rustc_ast",
"rustc_attr",
"rustc_data_structures",
@ -3749,6 +3779,7 @@ dependencies = [
"smallvec",
"snap",
"tempfile",
"thorin-dwp",
"tracing",
]
@ -4993,7 +5024,7 @@ dependencies = [
"hermit-abi",
"libc",
"miniz_oxide",
"object",
"object 0.26.2",
"panic_abort",
"panic_unwind",
"profiler_builtins",
@ -5057,9 +5088,9 @@ checksum = "8ea5119cdb4c55b55d432abb513a0429384878c15dde60cc77b1c99de1a95a6a"
[[package]]
name = "structopt"
version = "0.3.16"
version = "0.3.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de5472fb24d7e80ae84a7801b7978f95a19ec32cb1876faea59ab711eb901976"
checksum = "40b9788f4202aa75c240ecc9c15c65185e6a39ccdeb0fd5d008b98825464c87c"
dependencies = [
"clap",
"lazy_static",
@ -5068,9 +5099,9 @@ dependencies = [
[[package]]
name = "structopt-derive"
version = "0.4.9"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e0eb37335aeeebe51be42e2dc07f031163fbabfa6ac67d7ea68b5c2f68d5f99"
checksum = "dcb5ae327f9cc13b68763b5749770cb9e048a99bd9dfdfa58d0cf05d5f64afe0"
dependencies = [
"heck",
"proc-macro-error",
@ -5249,24 +5280,36 @@ dependencies = [
[[package]]
name = "thiserror"
version = "1.0.20"
version = "1.0.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dfdd070ccd8ccb78f4ad66bf1982dc37f620ef696c6b5028fe2ed83dd3d0d08"
checksum = "854babe52e4df1653706b98fcfc05843010039b406875930a70e4d9644e5c417"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "1.0.20"
version = "1.0.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd80fc12f73063ac132ac92aceea36734f04a1d93c1240c6944e23a3b8841793"
checksum = "aa32fd3f627f367fe16f893e2597ae3c05020f8bba2666a4e6ea73d377e5714b"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "thorin-dwp"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "039d1fc0bfdb73910c2702893515580e38c192f47a987bc98ddd38a36f2d953a"
dependencies = [
"gimli 0.26.1",
"indexmap",
"object 0.27.1",
"tracing",
]
[[package]]
name = "thread_local"
version = "1.0.1"
@ -5394,9 +5437,9 @@ checksum = "360dfd1d6d30e05fda32ace2c8c70e9c0a9da713275777f5a4dbb8a1893930c6"
[[package]]
name = "tracing"
version = "0.1.28"
version = "0.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84f96e095c0c82419687c20ddf5cb3eadb61f4e1405923c9dc8e53a1adacbda8"
checksum = "375a639232caf30edfc78e8d89b2d4c375515393e7af7e16f01cd96917fb2105"
dependencies = [
"cfg-if 1.0.0",
"pin-project-lite",
@ -5406,9 +5449,9 @@ dependencies = [
[[package]]
name = "tracing-attributes"
version = "0.1.17"
version = "0.1.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4f915eb6abf914599c200260efced9203504c4c37380af10cdf3b7d36970650"
checksum = "f4f480b8f81512e825f337ad51e94c1eb5d3bbdf2b363dcd01e2b19a9ffe3f8e"
dependencies = [
"proc-macro2",
"quote",

View File

@ -23,7 +23,7 @@ use rustc_errors::{FatalError, Handler, Level};
use rustc_fs_util::{link_or_copy, path_to_c_string};
use rustc_middle::bug;
use rustc_middle::ty::TyCtxt;
use rustc_session::config::{self, Lto, OutputType, Passes, SwitchWithOptPath};
use rustc_session::config::{self, Lto, OutputType, Passes, SplitDwarfKind, SwitchWithOptPath};
use rustc_session::Session;
use rustc_span::symbol::sym;
use rustc_span::InnerSpan;
@ -106,7 +106,11 @@ pub fn create_informational_target_machine(sess: &Session) -> &'static mut llvm:
pub fn create_target_machine(tcx: TyCtxt<'_>, mod_name: &str) -> &'static mut llvm::TargetMachine {
let split_dwarf_file = if tcx.sess.target_can_use_split_dwarf() {
tcx.output_filenames(()).split_dwarf_path(tcx.sess.split_debuginfo(), Some(mod_name))
tcx.output_filenames(()).split_dwarf_path(
tcx.sess.split_debuginfo(),
tcx.sess.opts.debugging_opts.split_dwarf_kind,
Some(mod_name),
)
} else {
None
};
@ -892,17 +896,18 @@ pub(crate) unsafe fn codegen(
.generic_activity_with_arg("LLVM_module_codegen_emit_obj", &*module.name);
let dwo_out = cgcx.output_filenames.temp_path_dwo(module_name);
let dwo_out = match cgcx.split_debuginfo {
// Don't change how DWARF is emitted in single mode (or when disabled).
SplitDebuginfo::Off | SplitDebuginfo::Packed => None,
// Emit (a subset of the) DWARF into a separate file in split mode.
SplitDebuginfo::Unpacked => {
if cgcx.target_can_use_split_dwarf {
Some(dwo_out.as_path())
} else {
None
}
}
let dwo_out = match (cgcx.split_debuginfo, cgcx.split_dwarf_kind) {
// Don't change how DWARF is emitted when disabled.
(SplitDebuginfo::Off, _) => None,
// Don't provide a DWARF object path if split debuginfo is enabled but this is
// a platform that doesn't support Split DWARF.
_ if !cgcx.target_can_use_split_dwarf => None,
// Don't provide a DWARF object path in single mode, sections will be written
// into the object as normal but ignored by linker.
(_, SplitDwarfKind::Single) => None,
// Emit (a subset of the) DWARF into a separate dwarf object file in split
// mode.
(_, SplitDwarfKind::Split) => Some(dwo_out.as_path()),
};
with_codegen(tm, llmod, config.no_builtins, |cpm| {
@ -939,7 +944,9 @@ pub(crate) unsafe fn codegen(
Ok(module.into_compiled_module(
config.emit_obj != EmitObj::None,
cgcx.target_can_use_split_dwarf && cgcx.split_debuginfo == SplitDebuginfo::Unpacked,
cgcx.target_can_use_split_dwarf
&& cgcx.split_debuginfo != SplitDebuginfo::Off
&& cgcx.split_dwarf_kind == SplitDwarfKind::Split,
config.emit_bc,
&cgcx.output_filenames,
))

View File

@ -1072,7 +1072,11 @@ pub fn compile_unit_metadata<'ll, 'tcx>(
let output_filenames = tcx.output_filenames(());
let split_name = if tcx.sess.target_can_use_split_dwarf() {
output_filenames
.split_dwarf_path(tcx.sess.split_debuginfo(), Some(codegen_unit_name))
.split_dwarf_path(
tcx.sess.split_debuginfo(),
tcx.sess.opts.debugging_opts.split_dwarf_kind,
Some(codegen_unit_name),
)
// We get a path relative to the working directory from split_dwarf_path
.map(|f| tcx.sess.source_map().path_mapping().map_prefix(f).0)
} else {

View File

@ -14,12 +14,14 @@ tracing = "0.1"
libc = "0.2.50"
jobserver = "0.1.22"
tempfile = "3.2"
thorin-dwp = "0.1.1"
pathdiff = "0.2.0"
snap = "1"
smallvec = { version = "1.6.1", features = ["union", "may_dangle"] }
regex = "1.4"
rustc_serialize = { path = "../rustc_serialize" }
rustc_arena = { path = "../rustc_arena" }
rustc_ast = { path = "../rustc_ast" }
rustc_span = { path = "../rustc_span" }
rustc_middle = { path = "../rustc_middle" }

View File

@ -1,11 +1,13 @@
use rustc_arena::TypedArena;
use rustc_data_structures::fx::{FxHashSet, FxIndexMap};
use rustc_data_structures::memmap::Mmap;
use rustc_data_structures::temp_dir::MaybeTempDir;
use rustc_errors::{ErrorReported, Handler};
use rustc_fs_util::fix_windows_verbatim_for_gcc;
use rustc_hir::def_id::CrateNum;
use rustc_middle::middle::dependency_format::Linkage;
use rustc_session::config::{self, CFGuard, CrateType, DebugInfo, LdImpl, Strip};
use rustc_session::config::{OutputFilenames, OutputType, PrintRequest};
use rustc_session::config::{OutputFilenames, OutputType, PrintRequest, SplitDwarfKind};
use rustc_session::cstore::DllImport;
use rustc_session::output::{check_file_is_writeable, invalid_output_for_target, out_filename};
use rustc_session::search_paths::PathKind;
@ -32,7 +34,10 @@ use cc::windows_registry;
use regex::Regex;
use tempfile::Builder as TempFileBuilder;
use std::ffi::{OsStr, OsString};
use std::borrow::Borrow;
use std::ffi::OsString;
use std::fs::{File, OpenOptions};
use std::io::{BufWriter, Write};
use std::lazy::OnceCell;
use std::path::{Path, PathBuf};
use std::process::{ExitStatus, Output, Stdio};
@ -134,25 +139,20 @@ pub fn link_binary<'a, B: ArchiveBuilder<'a>>(
}
}
// Remove the temporary object file and metadata if we aren't saving temps
// Remove the temporary object file and metadata if we aren't saving temps.
sess.time("link_binary_remove_temps", || {
if !sess.opts.cg.save_temps {
// If the user requests that temporaries are saved, don't delete any.
if sess.opts.cg.save_temps {
return;
}
let remove_temps_from_module = |module: &CompiledModule| {
if let Some(ref obj) = module.object {
ensure_removed(sess.diagnostic(), obj);
}
if let Some(ref obj) = module.dwarf_object {
ensure_removed(sess.diagnostic(), obj);
}
};
if sess.opts.output_types.should_link() && !preserve_objects_for_their_debuginfo(sess) {
for module in &codegen_results.modules {
remove_temps_from_module(module);
}
}
// Otherwise, always remove the metadata and allocator module temporaries.
if let Some(ref metadata_module) = codegen_results.metadata_module {
remove_temps_from_module(metadata_module);
}
@ -160,6 +160,27 @@ pub fn link_binary<'a, B: ArchiveBuilder<'a>>(
if let Some(ref allocator_module) = codegen_results.allocator_module {
remove_temps_from_module(allocator_module);
}
// If no requested outputs require linking, then the object temporaries should
// be kept.
if !sess.opts.output_types.should_link() {
return;
}
// Potentially keep objects for their debuginfo.
let (preserve_objects, preserve_dwarf_objects) = preserve_objects_for_their_debuginfo(sess);
debug!(?preserve_objects, ?preserve_dwarf_objects);
for module in &codegen_results.modules {
if !preserve_objects {
remove_temps_from_module(module);
}
if !preserve_dwarf_objects {
if let Some(ref obj) = module.dwarf_object {
ensure_removed(sess.diagnostic(), obj);
}
}
}
});
@ -245,10 +266,16 @@ fn link_rlib<'a, B: ArchiveBuilder<'a>>(
let mut ab = <B as ArchiveBuilder>::new(sess, out_filename, None);
for obj in codegen_results.modules.iter().filter_map(|m| m.object.as_ref()) {
for m in &codegen_results.modules {
if let Some(obj) = m.object.as_ref() {
ab.add_file(obj);
}
if let Some(dwarf_obj) = m.dwarf_object.as_ref() {
ab.add_file(dwarf_obj);
}
}
// Note that in this loop we are ignoring the value of `lib.cfg`. That is,
// we may not be configured to actually include a static library if we're
// adding it here. That's because later when we consume this rlib we'll
@ -502,59 +529,108 @@ fn escape_stdout_stderr_string(s: &[u8]) -> String {
})
}
const LLVM_DWP_EXECUTABLE: &'static str = "rust-llvm-dwp";
/// Invoke `llvm-dwp` (shipped alongside rustc) to link debuginfo in object files into a `dwp`
/// file.
fn link_dwarf_object<'a, I>(sess: &'a Session, executable_out_filename: &Path, object_files: I)
where
I: IntoIterator<Item: AsRef<OsStr>>,
{
info!("preparing dwp to {}.dwp", executable_out_filename.to_str().unwrap());
/// Use `thorin` (rust implementation of a dwarf packaging utility) to link DWARF objects into a
/// DWARF package.
fn link_dwarf_object<'a>(
sess: &'a Session,
cg_results: &CodegenResults,
executable_out_filename: &Path,
) {
let dwp_out_filename = executable_out_filename.with_extension("dwp");
let mut cmd = Command::new(LLVM_DWP_EXECUTABLE);
cmd.arg("-o");
cmd.arg(&dwp_out_filename);
cmd.args(object_files);
debug!(?dwp_out_filename, ?executable_out_filename);
let mut new_path = sess.get_tools_search_paths(false);
if let Some(path) = env::var_os("PATH") {
new_path.extend(env::split_paths(&path));
#[derive(Default)]
struct ThorinSession<Relocations> {
arena_data: TypedArena<Vec<u8>>,
arena_mmap: TypedArena<Mmap>,
arena_relocations: TypedArena<Relocations>,
}
let new_path = env::join_paths(new_path).unwrap();
cmd.env("PATH", new_path);
info!("{:?}", &cmd);
match sess.time("run_dwp", || cmd.output()) {
Ok(prog) if !prog.status.success() => {
sess.struct_err(&format!(
"linking dwarf objects with `{}` failed: {}",
LLVM_DWP_EXECUTABLE, prog.status
))
.note(&format!("{:?}", &cmd))
.note(&escape_stdout_stderr_string(&prog.stdout))
.note(&escape_stdout_stderr_string(&prog.stderr))
.emit();
info!("linker stderr:\n{}", escape_stdout_stderr_string(&prog.stderr));
info!("linker stdout:\n{}", escape_stdout_stderr_string(&prog.stdout));
impl<Relocations> ThorinSession<Relocations> {
fn alloc_mmap<'arena>(&'arena self, data: Mmap) -> &'arena Mmap {
(*self.arena_mmap.alloc(data)).borrow()
}
Ok(_) => {}
}
impl<Relocations> thorin::Session<Relocations> for ThorinSession<Relocations> {
fn alloc_data<'arena>(&'arena self, data: Vec<u8>) -> &'arena [u8] {
(*self.arena_data.alloc(data)).borrow()
}
fn alloc_relocation<'arena>(&'arena self, data: Relocations) -> &'arena Relocations {
(*self.arena_relocations.alloc(data)).borrow()
}
fn read_input<'arena>(&'arena self, path: &Path) -> std::io::Result<&'arena [u8]> {
let file = File::open(&path)?;
let mmap = (unsafe { Mmap::map(file) })?;
Ok(self.alloc_mmap(mmap))
}
}
match sess.time("run_thorin", || -> Result<(), thorin::Error> {
let thorin_sess = ThorinSession::default();
let mut package = thorin::DwarfPackage::new(&thorin_sess);
// Input objs contain .o/.dwo files from the current crate.
match sess.opts.debugging_opts.split_dwarf_kind {
SplitDwarfKind::Single => {
for input_obj in cg_results.modules.iter().filter_map(|m| m.object.as_ref()) {
package.add_input_object(input_obj)?;
}
}
SplitDwarfKind::Split => {
for input_obj in cg_results.modules.iter().filter_map(|m| m.dwarf_object.as_ref()) {
package.add_input_object(input_obj)?;
}
}
}
// Input rlibs contain .o/.dwo files from dependencies.
let input_rlibs = cg_results
.crate_info
.used_crate_source
.values()
.filter_map(|csource| csource.rlib.as_ref())
.map(|(path, _)| path);
for input_rlib in input_rlibs {
debug!(?input_rlib);
package.add_input_object(input_rlib)?;
}
// Failing to read the referenced objects is expected for dependencies where the path in the
// executable will have been cleaned by Cargo, but the referenced objects will be contained
// within rlibs provided as inputs.
//
// If paths have been remapped, then .o/.dwo files from the current crate also won't be
// found, but are provided explicitly above.
//
// Adding an executable is primarily done to make `thorin` check that all the referenced
// dwarf objects are found in the end.
package.add_executable(
&executable_out_filename,
thorin::MissingReferencedObjectBehaviour::Skip,
)?;
let output = package.finish()?.write()?;
let mut output_stream = BufWriter::new(
OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(true)
.open(dwp_out_filename)?,
);
output_stream.write_all(&output)?;
output_stream.flush()?;
Ok(())
}) {
Ok(()) => {}
Err(e) => {
let dwp_not_found = e.kind() == io::ErrorKind::NotFound;
let mut err = if dwp_not_found {
sess.struct_err(&format!("linker `{}` not found", LLVM_DWP_EXECUTABLE))
} else {
sess.struct_err(&format!("could not exec the linker `{}`", LLVM_DWP_EXECUTABLE))
};
err.note(&e.to_string());
if !dwp_not_found {
err.note(&format!("{:?}", &cmd));
}
err.emit();
sess.struct_err("linking dwarf objects with thorin failed")
.note(&format!("{:?}", e))
.emit();
}
}
}
@ -900,14 +976,11 @@ fn link_natively<'a, B: ArchiveBuilder<'a>>(
SplitDebuginfo::Packed if sess.target.is_like_msvc => {}
// ... and otherwise we're processing a `*.dwp` packed dwarf file.
//
// We cannot rely on the .o paths in the exectuable because they may have been
// remapped by --remap-path-prefix and therefore invalid. So we need to provide
// the .o paths explicitly
SplitDebuginfo::Packed => link_dwarf_object(
sess,
&out_filename,
codegen_results.modules.iter().filter_map(|m| m.object.as_ref()),
),
// remapped by --remap-path-prefix and therefore invalid, so we need to provide
// the .o/.dwo paths explicitly.
SplitDebuginfo::Packed => link_dwarf_object(sess, codegen_results, out_filename),
}
let strip = strip_value(sess);
@ -1138,26 +1211,36 @@ pub fn linker_and_flavor(sess: &Session) -> (PathBuf, LinkerFlavor) {
bug!("Not enough information provided to determine how to invoke the linker");
}
/// Returns a boolean indicating whether we should preserve the object files on
/// the filesystem for their debug information. This is often useful with
/// split-dwarf like schemes.
fn preserve_objects_for_their_debuginfo(sess: &Session) -> bool {
/// Returns a pair of boolean indicating whether we should preserve the object and
/// dwarf object files on the filesystem for their debug information. This is often
/// useful with split-dwarf like schemes.
fn preserve_objects_for_their_debuginfo(sess: &Session) -> (bool, bool) {
// If the objects don't have debuginfo there's nothing to preserve.
if sess.opts.debuginfo == config::DebugInfo::None {
return false;
return (false, false);
}
// If we're only producing artifacts that are archives, no need to preserve
// the objects as they're losslessly contained inside the archives.
let output_linked =
sess.crate_types().iter().any(|&x| x != CrateType::Rlib && x != CrateType::Staticlib);
if !output_linked {
return false;
if sess.crate_types().iter().all(|&x| x.is_archive()) {
return (false, false);
}
// "unpacked" split debuginfo means that we leave object files as the
// debuginfo is found in the original object files themselves
sess.split_debuginfo() == SplitDebuginfo::Unpacked
match (sess.split_debuginfo(), sess.opts.debugging_opts.split_dwarf_kind) {
// If there is no split debuginfo then do not preserve objects.
(SplitDebuginfo::Off, _) => (false, false),
// If there is packed split debuginfo, then the debuginfo in the objects
// has been packaged and the objects can be deleted.
(SplitDebuginfo::Packed, _) => (false, false),
// If there is unpacked split debuginfo and the current target can not use
// split dwarf, then keep objects.
(SplitDebuginfo::Unpacked, _) if !sess.target_can_use_split_dwarf() => (true, false),
// If there is unpacked split debuginfo and the target can use split dwarf, then
// keep the object containing that debuginfo (whether that is an object file or
// dwarf object file depends on the split dwarf kind).
(SplitDebuginfo::Unpacked, SplitDwarfKind::Single) => (true, false),
(SplitDebuginfo::Unpacked, SplitDwarfKind::Split) => (false, true),
}
}
fn archive_search_paths(sess: &Session) -> Vec<PathBuf> {

View File

@ -286,7 +286,11 @@ impl TargetMachineFactoryConfig {
module_name: &str,
) -> TargetMachineFactoryConfig {
let split_dwarf_file = if cgcx.target_can_use_split_dwarf {
cgcx.output_filenames.split_dwarf_path(cgcx.split_debuginfo, Some(module_name))
cgcx.output_filenames.split_dwarf_path(
cgcx.split_debuginfo,
cgcx.split_dwarf_kind,
Some(module_name),
)
} else {
None
};
@ -329,6 +333,7 @@ pub struct CodegenContext<B: WriteBackendMethods> {
pub target_arch: String,
pub debuginfo: config::DebugInfo,
pub split_debuginfo: rustc_target::spec::SplitDebuginfo,
pub split_dwarf_kind: rustc_session::config::SplitDwarfKind,
// Number of cgus excluding the allocator/metadata modules
pub total_cgus: usize,
@ -1060,6 +1065,7 @@ fn start_executing_work<B: ExtraBackendMethods>(
target_arch: tcx.sess.target.arch.clone(),
debuginfo: tcx.sess.opts.debuginfo,
split_debuginfo: tcx.sess.split_debuginfo(),
split_dwarf_kind: tcx.sess.opts.debugging_opts.split_dwarf_kind,
};
// This is the "main loop" of parallel work happening for parallel codegen.

View File

@ -231,6 +231,37 @@ pub enum DebugInfo {
Full,
}
/// Split debug-information is enabled by `-C split-debuginfo`, this enum is only used if split
/// debug-information is enabled (in either `Packed` or `Unpacked` modes), and the platform
/// uses DWARF for debug-information.
///
/// Some debug-information requires link-time relocation and some does not. LLVM can partition
/// the debuginfo into sections depending on whether or not it requires link-time relocation. Split
/// DWARF provides a mechanism which allows the linker to skip the sections which don't require
/// link-time relocation - either by putting those sections in DWARF object files, or by keeping
/// them in the object file in such a way that the linker will skip them.
#[derive(Clone, Copy, Debug, PartialEq, Hash)]
pub enum SplitDwarfKind {
/// Sections which do not require relocation are written into object file but ignored by the
/// linker.
Single,
/// Sections which do not require relocation are written into a DWARF object (`.dwo`) file
/// which is ignored by the linker.
Split,
}
impl FromStr for SplitDwarfKind {
type Err = ();
fn from_str(s: &str) -> Result<Self, ()> {
Ok(match s {
"single" => SplitDwarfKind::Single,
"split" => SplitDwarfKind::Split,
_ => return Err(()),
})
}
}
#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug, PartialOrd, Ord)]
#[derive(Encodable, Decodable)]
pub enum OutputType {
@ -378,7 +409,7 @@ impl OutputTypes {
self.0.len()
}
// Returns `true` if any of the output types require codegen or linking.
/// Returns `true` if any of the output types require codegen or linking.
pub fn should_codegen(&self) -> bool {
self.0.keys().any(|k| match *k {
OutputType::Bitcode
@ -391,7 +422,7 @@ impl OutputTypes {
})
}
// Returns `true` if any of the output types require linking.
/// Returns `true` if any of the output types require linking.
pub fn should_link(&self) -> bool {
self.0.keys().any(|k| match *k {
OutputType::Bitcode
@ -681,18 +712,23 @@ impl OutputFilenames {
pub fn split_dwarf_path(
&self,
split_debuginfo_kind: SplitDebuginfo,
split_dwarf_kind: SplitDwarfKind,
cgu_name: Option<&str>,
) -> Option<PathBuf> {
let obj_out = self.temp_path(OutputType::Object, cgu_name);
let dwo_out = self.temp_path_dwo(cgu_name);
match split_debuginfo_kind {
SplitDebuginfo::Off => None,
match (split_debuginfo_kind, split_dwarf_kind) {
(SplitDebuginfo::Off, SplitDwarfKind::Single | SplitDwarfKind::Split) => None,
// Single mode doesn't change how DWARF is emitted, but does add Split DWARF attributes
// (pointing at the path which is being determined here). Use the path to the current
// object file.
SplitDebuginfo::Packed => Some(obj_out),
(SplitDebuginfo::Packed | SplitDebuginfo::Unpacked, SplitDwarfKind::Single) => {
Some(obj_out)
}
// Split mode emits the DWARF into a different file, use that path.
SplitDebuginfo::Unpacked => Some(dwo_out),
(SplitDebuginfo::Packed | SplitDebuginfo::Unpacked, SplitDwarfKind::Split) => {
Some(dwo_out)
}
}
}
}
@ -821,6 +857,18 @@ pub enum CrateType {
impl_stable_hash_via_hash!(CrateType);
impl CrateType {
/// When generated, is this crate type an archive?
pub fn is_archive(&self) -> bool {
match *self {
CrateType::Rlib | CrateType::Staticlib => true,
CrateType::Executable | CrateType::Dylib | CrateType::Cdylib | CrateType::ProcMacro => {
false
}
}
}
}
#[derive(Clone, Hash, Debug, PartialEq, Eq)]
pub enum Passes {
Some(Vec<String>),

View File

@ -412,6 +412,8 @@ mod desc {
pub const parse_wasi_exec_model: &str = "either `command` or `reactor`";
pub const parse_split_debuginfo: &str =
"one of supported split-debuginfo modes (`off`, `packed`, or `unpacked`)";
pub const parse_split_dwarf_kind: &str =
"one of supported split dwarf modes (`split` or `single`)";
pub const parse_gcc_ld: &str = "one of: no value, `lld`";
pub const parse_stack_protector: &str =
"one of (`none` (default), `basic`, `strong`, or `all`)";
@ -941,6 +943,14 @@ mod parse {
true
}
crate fn parse_split_dwarf_kind(slot: &mut SplitDwarfKind, v: Option<&str>) -> bool {
match v.and_then(|s| SplitDwarfKind::from_str(s).ok()) {
Some(e) => *slot = e,
_ => return false,
}
true
}
crate fn parse_gcc_ld(slot: &mut Option<LdImpl>, v: Option<&str>) -> bool {
match v {
None => *slot = None,
@ -1403,6 +1413,14 @@ options! {
"control stack smash protection strategy (`rustc --print stack-protector-strategies` for details)"),
strip: Strip = (Strip::None, parse_strip, [UNTRACKED],
"tell the linker which information to strip (`none` (default), `debuginfo` or `symbols`)"),
split_dwarf_kind: SplitDwarfKind = (SplitDwarfKind::Split, parse_split_dwarf_kind, [UNTRACKED],
"split dwarf variant (only if -Csplit-debuginfo is enabled and on relevant platform)
(default: `split`)
`split`: sections which do not require relocation are written into a DWARF object (`.dwo`)
file which is ignored by the linker
`single`: sections which do not require relocation are written into object file but ignored
by the linker"),
split_dwarf_inlining: bool = (true, parse_bool, [UNTRACKED],
"provide minimal debug info in the object/executable to facilitate online \
symbolication/stack traces in the absence of .dwo/.dwp files when using Split DWARF"),

View File

@ -479,7 +479,7 @@ pub enum SplitDebuginfo {
///
/// * Windows - not supported
/// * macOS - supported, scattered object files
/// * ELF - supported, scattered `*.dwo` files
/// * ELF - supported, scattered `*.dwo` or `*.o` files (see `SplitDwarfKind`)
Unpacked,
}

View File

@ -1143,7 +1143,6 @@ impl Step for Assemble {
let libdir = builder.sysroot_libdir(target_compiler, target_compiler.host);
let libdir_bin = libdir.parent().unwrap().join("bin");
t!(fs::create_dir_all(&libdir_bin));
if let Some(lld_install) = lld_install {
let src_exe = exe("lld", target_compiler.host);
let dst_exe = exe("rust-lld", target_compiler.host);
@ -1161,17 +1160,11 @@ impl Step for Assemble {
}
}
// Similarly, copy `llvm-dwp` into libdir for Split DWARF. Only copy it when the LLVM
// backend is used to avoid unnecessarily building LLVM and because LLVM is not checked
// out by default when the LLVM backend is not enabled.
if builder.config.rust_codegen_backends.contains(&INTERNER.intern_str("llvm")) {
let src_exe = exe("llvm-dwp", target_compiler.host);
let dst_exe = exe("rust-llvm-dwp", target_compiler.host);
let llvm_config_bin = builder.ensure(native::Llvm { target: target_compiler.host });
if !builder.config.dry_run {
let llvm_bin_dir = output(Command::new(llvm_config_bin).arg("--bindir"));
let llvm_bin_dir = Path::new(llvm_bin_dir.trim());
builder.copy(&llvm_bin_dir.join(&src_exe), &libdir_bin.join(&dst_exe));
// Since we've already built the LLVM tools, install them to the sysroot.
// This is the equivalent of installing the `llvm-tools-preview` component via

View File

@ -398,12 +398,12 @@ impl Step for Rustc {
// component for now.
maybe_install_llvm_runtime(builder, host, image);
let src_dir = builder.sysroot_libdir(compiler, host).parent().unwrap().join("bin");
let dst_dir = image.join("lib/rustlib").join(&*host.triple).join("bin");
t!(fs::create_dir_all(&dst_dir));
// Copy over lld if it's there
if builder.config.lld_enabled {
let src_dir = builder.sysroot_libdir(compiler, host).parent().unwrap().join("bin");
let rust_lld = exe("rust-lld", compiler.host);
builder.copy(&src_dir.join(&rust_lld), &dst_dir.join(&rust_lld));
// for `-Z gcc-ld=lld`
@ -417,10 +417,6 @@ impl Step for Rustc {
}
}
// Copy over llvm-dwp if it's there
let exe = exe("rust-llvm-dwp", compiler.host);
builder.copy(&src_dir.join(&exe), &dst_dir.join(&exe));
// Man pages
t!(fs::create_dir_all(image.join("share/man/man1")));
let man_src = builder.src.join("src/doc/man");

View File

@ -44,16 +44,78 @@ off:
[ ! -f $(TMPDIR)/*.dwp ]
[ ! -f $(TMPDIR)/*.dwo ]
packed:
$(RUSTC) foo.rs -g -C split-debuginfo=packed -Z unstable-options
packed: packed-split packed-single
packed-split:
$(RUSTC) foo.rs -g -C split-debuginfo=packed -Z unstable-options -Zsplit-dwarf-kind=split
ls $(TMPDIR)/*.dwp
rm -rf $(TMPDIR)/*.dwp $(TMPDIR)/*.dwo
packed-single:
$(RUSTC) foo.rs -g -C split-debuginfo=packed -Z unstable-options -Zsplit-dwarf-kind=single
ls $(TMPDIR)/*.dwp
ls $(TMPDIR)/*.dwo && exit 1 || exit 0
rm -rf $(TMPDIR)/*.dwp
unpacked:
$(RUSTC) foo.rs -g -C split-debuginfo=unpacked -Z unstable-options
packed-remapped: packed-remapped-split packed-remapped-single
packed-remapped-split:
$(RUSTC) -Z unstable-options -C split-debuginfo=packed -C debuginfo=2 \
-Z split-dwarf-kind=split --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
packed-remapped-single:
$(RUSTC) -Z unstable-options -C split-debuginfo=packed -C debuginfo=2 \
-Z split-dwarf-kind=single --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
packed-crosscrate: packed-crosscrate-split packed-crosscrate-single
packed-crosscrate-split:
$(RUSTC) --crate-type lib -Z unstable-options -C split-debuginfo=packed \
-Zsplit-dwarf-kind=split -C debuginfo=2 -g bar.rs
ls $(TMPDIR)/*.rlib
ls $(TMPDIR)/*.dwo && exit 1 || exit 0
ls $(TMPDIR)/*.dwp && exit 1 || exit 0
$(RUSTC) --extern bar=$(TMPDIR)/libbar.rlib -Z unstable-options -C split-debuginfo=packed \
-Zsplit-dwarf-kind=split -C debuginfo=2 -g main.rs
rm $(TMPDIR)/*.dwo
rm $(TMPDIR)/main.dwp
rm $(TMPDIR)/$(call BIN,main)
packed-crosscrate-single:
$(RUSTC) --crate-type lib -Z unstable-options -C split-debuginfo=packed \
-Zsplit-dwarf-kind=single -C debuginfo=2 -g bar.rs
ls $(TMPDIR)/*.rlib
ls $(TMPDIR)/*.dwo && exit 1 || exit 0
ls $(TMPDIR)/*.dwp && exit 1 || exit 0
$(RUSTC) --extern bar=$(TMPDIR)/libbar.rlib -Z unstable-options -C split-debuginfo=packed \
-Zsplit-dwarf-kind=single -C debuginfo=2 -g main.rs
ls $(TMPDIR)/*.dwo && exit 1 || exit 0
rm $(TMPDIR)/main.dwp
rm $(TMPDIR)/$(call BIN,main)
unpacked: unpacked-split unpacked-single unpacked-remapped-split unpacked-remapped-single
unpacked-split:
$(RUSTC) foo.rs -g -C split-debuginfo=unpacked -Z unstable-options -Zsplit-dwarf-kind=split
ls $(TMPDIR)/*.dwp && exit 1 || exit 0
ls $(TMPDIR)/*.dwo
rm -rf $(TMPDIR)/*.dwo
rm -rf $(TMPDIR)/*.dwp $(TMPDIR)/*.dwo
unpacked-single:
$(RUSTC) foo.rs -g -C split-debuginfo=unpacked -Z unstable-options -Zsplit-dwarf-kind=single
ls $(TMPDIR)/*.dwp && exit 1 || exit 0
ls $(TMPDIR)/*.dwo && exit 1 || exit 0
unpacked-remapped-split:
$(RUSTC) -Z unstable-options -C split-debuginfo=unpacked -C debuginfo=2 \
-Z split-dwarf-kind=split --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
unpacked-remapped-single:
$(RUSTC) -Z unstable-options -C split-debuginfo=unpacked -C debuginfo=2 \
-Z split-dwarf-kind=single --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
endif
endif

View File

@ -0,0 +1,13 @@
pub struct Bar {
x: u32,
}
impl Bar {
pub fn print(&self) {
println!("{}", self.x);
}
}
pub fn make_bar(x: u32) -> Bar {
Bar { x }
}

View File

@ -1 +1,15 @@
pub struct Foo {
x: u32,
}
impl Foo {
pub fn print(&self) {
println!("{}", self.x);
}
}
pub fn make_foo(x: u32) -> Foo {
Foo { x }
}
fn main() {}

View File

@ -0,0 +1,8 @@
extern crate bar;
use bar::{Bar, make_bar};
fn main() {
let b = make_bar(3);
b.print();
}

View File

@ -1,17 +0,0 @@
-include ../tools.mk
# only-linux
all: packed remapped
remapped:
$(RUSTC) -Z unstable-options -C split-debuginfo=packed -C debuginfo=2 --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
$(RUSTC) -Z unstable-options -C split-debuginfo=unpacked -C debuginfo=2 --remap-path-prefix $(TMPDIR)=/a foo.rs -g
objdump -Wi $(TMPDIR)/foo | grep DW_AT_GNU_dwo_name | (! grep $(TMPDIR)) || exit 1
packed:
$(RUSTC) -Z unstable-options -C split-debuginfo=packed -C debuginfo=2 foo.rs -g
rm $(TMPDIR)/foo.dwp
rm $(TMPDIR)/$(call BIN,foo)

View File

@ -1 +0,0 @@
fn main() {}

View File

@ -109,6 +109,7 @@ const PERMITTED_DEPENDENCIES: &[&str] = &[
"env_logger",
"expect-test",
"fake-simd",
"fallible-iterator", // dependency of `thorin`
"filetime",
"fixedbitset",
"flate2",
@ -201,6 +202,7 @@ const PERMITTED_DEPENDENCIES: &[&str] = &[
"tempfile",
"termcolor",
"termize",
"thorin-dwp",
"thread_local",
"time",
"tinyvec",