Commit Graph

950 Commits

Author SHA1 Message Date
Brian Behlendorf 9b2af9a097 Configure --with-spl{-obj} auto-detect cleanup
Because the install location for the spl/zfs-devel headers was
changed we need to refresh the auto-detect code.  Note that
for packaging which already explicitly calls --with-spl{-obj}
nothing has changed.

The updated code is now structured like that in ZFS_AC_KERNEL
and should be cleaner and easier to maintain.  In addition,
it's stricter about detecting a valid source and object
directory.  It requires:

* The source directory contains the file 'spl.release'
* The object directory contains the file 'spl_config.h'
* The following paths will be checked.  Notice the /var/lib/
  and /usr/src paths require that the spl and zfs version be
  matched.  This is done to prevent accidentally mixing releases.

        dnl # 1) /var/lib/dkms/spl/<version>/build
        dnl # 2) /usr/src/spl-<version>/<kernel-version>
        dnl # 3) /usr/src/spl-<version>
        dnl # 4) ../spl
        dnl # 5) /usr/src/kernels/<kernel-version>

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-13 13:42:16 -07:00
Brian Behlendorf 775f2d34a3 Change zfs-kmod-devel install path
Install the common zfs kernel development headers under
/usr/src/zfs-<version>/ rather than in a kernel specific
directory.  The kernel specific build products such as
zfs_config.h and Modules.symvers are left installed under
/usr/src/zfs-<version>/<kernel>.

This was done to be consistent with where dkms expects
kernel module source to be packaged.  It also allows for
a common zfs-kmod-devel package which includes the headers,
and per-kernel zfs-kmod-devel-<kernel> packages.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-13 13:42:16 -07:00
Brian Behlendorf d69b2dee71 Remove COPYING
There's no reason to keep this file around it's redundant with
the COPYRIGHT and OPENSOLARIS.LICENSE files and causes lintian
to emit an extra-license-file warning.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-13 13:42:16 -07:00
Darik Horn cf2129e6dc Create mount.zfs, zinject, and zpios man pages.
And update the automake templates to install them.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #518
2013-03-13 13:41:22 -07:00
Michael Gebetsroither 5cd0add214 Import ztest.1 man page.
Create a nroff man page for ZoL from:

  http://grml.org/online-docs/ztest.1.html

Which itself was derived from:

  http://opensolaris.org/os/community/zfs/ztest/

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #518
2013-03-13 13:40:04 -07:00
Richard Yao 1c24b699b0 Linux 3.9 compat: Undefine GCC_VERSION
The mainline kernel started defining GCC_VERSION with commit
torvalds/linux@3f3f8d2f48.
Unfortunately, LZ4 also defines this macro, but the two
defintions are incompatible. We undefine GCC_VERSION in lz4.c
to handle this.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1339
2013-03-06 15:48:48 -08:00
Brian Behlendorf 7c72f8e391 Merge branch 'build-system' 2013-03-06 15:47:38 -08:00
Brian Behlendorf 2016ff96d1 Fix zdb.8 macro warning
Detected by rpmlint the 'rpool/export/home' section was being
interpretted by troff as an undefined macro.  This resulted
in the 'rpool/export/home' output being omitted from 'man zdb'.

This was caused by starting the line with a ' character.  By
moving the 'in' down to the next line we're able to fix it.

  zfs.x86_64: W: manual-page-warning /usr/share/man/man8/zdb.8.gz
  450: warning: macro `rpool/export/home'' not defined

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Ned Bass 92db59ca3b Refresh links to web site
A few files still refer to @behlendorf's private fork on
github.  Use the primary web site URL instead.  Two typos
are also corrected.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Brian Behlendorf e07b579b1b Retire ZFS.RELEASE file
The ZFS.RELEASE file was originally added to document which
version of OpenSolaris the ZoL code was based on.  However,
that's no longer particularly important or useful.  We'll
likely never see a new onnv_* drop from Solaris, and even
if we do the ZoL changes are now extensive enough they
could not be easily applied.  We now treat Illumos as the
official upstream and cherry pick the patches we need.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Brian Behlendorf 0da31cd6ca Remove ARCH packaging
The kernel modules are now available in the Arch User Repository
(AUR) via zfs.  Since their packaging is maintained and superior
to ours it is being removed from the tree.

  https://wiki.archlinux.org/index.php/ZFS

Now that various distributions are picking up the packages we
should eventually be able to remove most of this infrastructure.
Packaging belongs with the distributions not upstream.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Brian Behlendorf ffb21118ad Add --with-dracutdir configure option
The standard dracut directory has moved from /usr/share/dracut to
/usr/lib/dracut.  To ensure the dracut modules get installed in
the correct location provide a --with-dracutdir configure option
to set the path.

The default install location has been updated to /usr/lib/dracut
which is used by more current versions of Fedora.  However, this
default is overriden by the RPM packaging for consistency.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Brian Behlendorf d09f98a9a6 Add KMODDIR to install target
Provide a mechanism to control the directory name the modules
are installed in.  The kernel privdes INSTALL_MOD_DIR for
this but it was hardcoded to be 'addon/zfs'.

Add a KMODDIR variable which can be passed to 'make install'
to override the default directory name.  While we're here
change the default from 'addon/zfs' to 'extra' which is the
kernel.org default.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:40 -08:00
Brian Behlendorf fd2b4aa71a Fix zfs_config.h install permissions
The default permissions used by install are 755.  Since this
file isn't executable 644 is more appropriate.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:40 -08:00
Brian Behlendorf 48c028f5a5 Replace libexecdir with datadir
According to the FHS.  Testing scripts and examples which are all
architecture independent should be installed in a subdirectory
under /usr/share.

  http://www.pathname.com/fhs/2.2/fhs-4.11.html

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:40 -08:00
Eric Dillmann 0b4d1b5853 Add snapdev=[hidden|visible] dataset property
The new snapdev dataset property may be set to control the
visibility of zvol snapshot devices.  By default this value
is set to 'hidden' which will prevent zvol snapshots from
appearing under /dev/zvol/ and /dev/<dataset>/.  When set to
'visible' all zvol snapshots for the dataset will be visible.

This functionality was largely added because when automatic
snapshoting is enabled large numbers of read-only zvol snapshots
will be created.  When creating these devices the kernel will
attempt to read their partition tables, and blkid will attempt
to identify any filesystems on those partitions.  This leads
to a variety of issues:

1) The zvol partition tables will be read in the context of
   the `modprobe zfs` for automatically imported pools.  This
   is undesirable and should be done asynchronously, but for
   now reducing the number of visible devices helps.

2) Udev expects to be able to complete its work for a new
   block devices fairly quickly.  When many zvol devices are
   added at the same time this is no longer be true.  It can
   lead to udev timeouts and missing /dev/zvol links.

3) Simply having lots of devices in /dev/ can be aukward from
   a management standpoint.  Hidding the devices your unlikely
   to ever use helps with this.  Any snapshot device which is
   needed can be made visible by changing the snapdev property.

NOTE: This patch changes the default behavior for zvols which
      was effectively 'snapdev=visible'.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1235
Closes #945
Issue #956
Issue #756
2013-03-05 12:37:54 -08:00
George Wilson a4430fce69 Merge zvol.c changes from PSARC 2010/306 Read-only ZFS pools
The changes to zvol.c were never merged from the last onnv_147
bulk update.  This was because zvol.c was largely rewritten
for Linux making it fairly easy to miss these sorts of changes.

This causes a regression when importing a zpool with zvols
read-only.  This does not impact pool which only contain
filesystem datasets.

References:
  illumos/illumos-gate@f9af39b

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1332
Closes #1333
2013-03-04 09:56:13 -08:00
Richard Yao b01615d5ac Constify structures containing function pointers
The PaX team modified the kernel's modpost to report writeable function
pointers as section mismatches because they are potential exploit
targets. We could ignore the warnings, but their presence can obscure
actual issues. Proper const correctness can also catch programming
mistakes.

Building the kernel modules against a PaX/GrSecurity patched Linux 3.4.2
kernel reports 133 section mismatches prior to this patch. This patch
eliminates 130 of them. The quantity of writeable function pointers
eliminated by constifying each structure is as follows:

vdev_opts_t             52
zil_replay_func_t       24
zio_compress_info_t     24
zio_checksum_info_t     9
space_map_ops_t         7
arc_byteswap_func_t     5

The remaining 3 writeable function pointers cannot be addressed by this
patch. 2 of them are in zpl_fs_type. The kernel's sget function requires
that this be non-const. The final writeable function pointer is created
by SPL_SHRINKER_DECLARE. The kernel's set_shrinker() and
remove_shrinker() functions also require that this be non-const.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1300
2013-03-04 08:49:32 -08:00
Richard Yao c38367c73f Eliminate runtime function pointer mods in autotools checks
PaX/GrSecurity patched kernels implement a dialect of C that relies on a
GCC plugin for enforcement. A basic idea in this dialect is that
function pointers in structures should not change during runtime.
This causes code that modifies function pointers at runtime to fail to
compile in many instances. The autotools checks rely on whether or
not small test cases compile against a given kernel. Some
autotools checks assume some default case if other cases fail. When one
of these autotools checks tests a PaX/GrSecurity patched kernel by
modifying a function pointer at runtime, the default case will be used.

Early detection of such situations is possible by relying on compiler
warnings, which are compiler errors when --enable-debug is used.
Unfortunately, very few people build ZFS with --enable-debug. The more
common situation is that these issues manifest themselves as runtime
failures in the form of NULL pointer exceptions.

Previous patches that addressed such issues with PaX/GrSecurity
compatibility largely relied on rewriting autotools checks to avoid
runtime function pointer modification or the addition of PaX/GrSecurity
specific checks. This patch takes the previous work to its logical
conclusion by eliminating the use of runtime function pointer
modification. This permits the removal of PaX-specific autotools checks
in favor of ones that work across all supported kernels.

This should resolve issues that were reported to occur with
PaX/GrSecurity-patched Linux 3.7.5 kernels on Gentoo Linux.

https://bugs.gentoo.org/show_bug.cgi?id=457176

We should be able to prevent future regressions in PaX/GrSecurity
compatibility by ensuring that all changes to ZFSOnLinux avoid runtime
function pointer modification. At the same time, this does not solve the
issue of silent failures triggering default cases in the autotools
check, which is what permitted these regressions to become runtime
failures in the first place. This will need to be addressed in a future
patch.

Reported-by: Marcin Mirosław <bug@mejor.pl>
Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1300
2013-03-04 08:49:17 -08:00
Brian Behlendorf 8128bd89fb Fix hot spares
The issue with hot spares in ZoL is because it opens all leaf
vdevs exclusively (O_EXCL).  On Linux, exclusive opens cause
subsequent exclusive opens to fail with EBUSY.

This could be resolved by not opening any of the devices
exclusively, which is what Illumos does, but the additional
protection offered by exclusive opens is desirable.  It cleanly
prevents you from accidentally adding an in-use non-ZFS device
to your pool.

To fix this we very slightly relaxed the usage of O_EXCL in
the following ways.

1) Functions which open the device but only read had the
   O_EXCL flag removed and were updated to use O_RDONLY.

2) A common holder was added to the vdev disk code.  This
   allow the ZFS code to internally open the device multiple
   times but non-ZFS callers may not.

3) An exception was added to make_disks() for hot spare when
   creating partition tables.  For hot spare devices which
   are already opened exclusively we skip creating the partition
   table because this must already have been done when the disk
   was originally added as a hot spare.

Additional minor changes include fixing check_in_use() to use
a partition instead of a slice suffix.  And is_spare() was moved
above make_disks() to avoid adding a forward reference.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #250
2013-03-01 13:31:02 -08:00
Brian Behlendorf bd99a7584a Remove wholedisk check from vdev_disk_open()
As described by the comment and enforced the by assertion the
v->vdev_wholedisk will never be -1.  The wholedisk handling
is performed by the user space utilities.  To prevent confusion
this dead code is being removed.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-02-28 12:02:59 -08:00
Brian Behlendorf 0d8103d956 Leaf vdevs should not be reopened
When vdev_disk.c was implemented for Linux we failed to handle the
reopen case.  According to the vdev_reopen() comment leaf vdevs should
not be closed or opened when v->vdev_reopening is set.  Under Linux
we would always close and open the device.

This issue was only noticed when a 'zpool scrub' command was run while
the leaf vdev device names in /dev/disk/by-vdev were missing.  The
scrub command calls vdev_reopen() which caused the vdevs to be closed
but they couldn't be reopened due to the missing links.  The result
was that all the vdevs were marked unavailable and the pool was
halted due to failmode=wait.

This patch adds the missing functionality in a similiar fashion to
to the Illumos code.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-02-28 12:02:59 -08:00
Tim Connors c5b247f335 -x shouldn't warn about old on-disk format or unavailable features
`zpool status -x` should only flag errors or where the pool is
unavailable.  If it imported fine but isn't using the latest features
available in the code, that's not an error.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1319
2013-02-28 09:17:09 -08:00
Etienne Dechamps d9b0ebbe82 Remove the bio_empty_barrier() check.
To determine whether the kernel is capable of handling empty barrier
BIOs, we check for the presence of the bio_empty_barrier() macro,
which was introduced in 2.6.24. If this macro is defined, then we can
flush disk vdevs; if it isn't, then flushing is disabled.

Unfortunately, the bio_empty_barrier() macro was removed in 2.6.37,
even though the kernel is still capable of handling empty barrier BIOs.

As a result, flushing is effectively disabled on kernels >= 2.6.37,
meaning that starting from this kernel version, zfs doesn't use
barriers to guarantee on-disk data consistency. This is quite bad and
can lead to potential data corruption on power failures.

This patch fixes the issue by removing the configure check for
bio_empty_barrier(), as we don't support kernels <= 2.6.24 anymore.

Thanks to Richard Kojedzinszky for catching this nasty bug.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1318
2013-02-24 10:22:34 -08:00
Etienne Dechamps d75af3c0eb Use -Werror for all kernel configure tests.
As a matter of fact, we're already using -Werror for most tests because
of a bug in kernel-bio-empty-barrier.m4 which sets -Werror without
reverting it afterwards. This meant that all tests which ran after this
one was using -Werror.

This patch simply makes it clear that we're using -Werror and makes
the code more readable and more predictable.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1317
2013-02-24 10:20:28 -08:00
Brian Behlendorf 546c978bbd Enable zfs_arc_memory_throttle_disable by default
The zfs_arc_memory_throttle_disable module option was introduced
by commit 0c5493d470 to resolve a
memory miscalculation which could result in the txg_sync thread
spinning.

When this was first introduced the default behavior was left
unchanged until enough real world usage confirmed there were no
unexpected issues.  We've now reached that point.  Linux's
direct reclaim is working as expected so we're enabling this
behavior by default.

This helps pave the way to retire the spl_kmem_availrmem()
functionality in the SPL layer.  This was the only caller.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #938
2013-02-21 13:38:24 -08:00
Brian Behlendorf 73a046cc8c Fix broken RPATH in spec file
Rather then setting _prefix=/ and having to override all the
default install locations.  It's cleaner, and more understandable,
to leave prefix=/usr and only override _sbindir and _libdir.  This
fixes three issues:

* The commands no longer get built with an incorrect rpath for
  the libraries.   This is good because fixing this sort of
  thing is required by the Fedora packaging guidelines.

    http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath

* The various AUTHORS, COPYRIGHT, etc files are now correctly
  installed under /usr/share/doc instead of /share/doc.

* _libexecdir is now handled properly for each distribution.
  Fedora/RHEL=/usr/libexec, OpenSUSE/SLES=/usr/lib, Debian=/usr/lib/rpm

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1058
2013-02-12 13:59:09 -08:00
Richard Yao 8dca0a9a38 Make spa.c assertions catch unsupported pre-feature flag pool versions
A couple of assertions in spa.c were designed to prevent the use of
invalid pool versions. They were written under the assumption
that all valid pools are less than SPA_VERSION. Since feature flags
jumped from 28 to 5000, any numbers in the range 28 to 5000
non-inclusive will fail to trigger them.  We switch to the new
SPA_VERSION_IS_SUPPORTED macro to correct this.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1282
2013-02-12 10:27:44 -08:00
Brian Behlendorf 9878a89d7a Add explicit MAXNAMELEN check
It turns out that the Linux VFS doesn't strictly handle all cases
where a component path name exceeds MAXNAMELEN.  It does however
appear to correctly handle MAXPATHLEN for us.

The right way to handle this appears to be to add an explicit
check to the zpl_lookup() function.  Several in-tree filesystems
handle this case the same way.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1279
2013-02-12 10:27:39 -08:00
Brian Behlendorf 52768784e6 Update the zfs.8 "ZFS Volumes as Swap" section
As of 0.6.0-rc11 using ZFS volumes as Linux swap devices is
supported.  Swapping to files in ZFS filesystems is not.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1189
2013-02-07 14:19:40 -08:00
Ned Bass ed2e157605 Switch KM_SLEEP to KM_PUSHPAGE
Two more locations where KM_SLEEP was used in a call which must
use KM_PUSHPAGE were found while using the zpool upgrade command.
See commit b8d06fc for additional details.

Also make a small correction to the comment block above
dsl_dir_open_spa().

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1268
2013-02-06 11:19:58 -08:00
Brian Behlendorf 02a946ce10 Remove unused machelf.h header
The machelf.h header is never included by anything in the zfs
build process.  It is all effectively dead code which can be
safely removed.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1265
2013-02-05 15:34:50 -08:00
Richard Yao 399f60c8b4 Fix function relocations in libzpool
binutils 2.23.1 fails in situations that generate function relocations
on PowerPC and possibly other architectures. This causes linking of
libzpool to fail because it depends on libnvpair. We add a dependency on
libnvpair to lib/libzpool/Makefile.am to correct that.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1267
2013-02-05 15:33:32 -08:00
Brian Behlendorf dd26aa535b Cast 'zfs bad bloc' to ULL for x86
Explicitly case this value to an unsigned long long for 32-bit
systems to inform the compiler that a long type should not be
used.  Otherwise we get the following compiler error:

  dmu_send.c:376: error: integer constant is too large for
  ‘long’ type

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-02-04 16:39:08 -08:00
Brian Behlendorf a31770283a Fix 1M references in zpool-features.5
The zpool-features(5) man page should reference the Linux zfs(8)
and zpool(8) man pages.  The 1M convention isn't used on Linux.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1184
2013-02-04 12:37:06 -08:00
Brian Behlendorf 3795ae5ed7 Add zpool-features(5) man page
The zpool-features(5) man page was accidentally omitted from the
build target when feature flags was merged.  As a result it doesn't
get installed as part of 'make install' so none of the packages
include this man page.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1262
2013-02-04 12:10:11 -08:00
Brian Behlendorf a997dfbaf0 ZFS 0.6.0-rc14 2013-02-01 11:26:55 -08:00
Brian Behlendorf 0c5493d470 Add zfs_arc_memory_throttle_disable module option
The way in which virtual box ab(uses) memory can throw off the
free memory calculation in arc_memory_throttle().  The result is
the txg_sync thread will effectively spin waiting for memory to
be released even though there's lots of memory on the system.

To handle this case I'm adding a zfs_arc_memory_throttle_disable
module option largely for virtual box users.  Setting this option
disables free memory checks which allows the txg_sync thread to
make progress.

By default this option is disabled to preserve the current
behavior.  However, because Linux supports direct memory reclaim
it's doubtful throttling due to perceived memory pressure is ever
a good idea.  We should enable this option by default once we've
done enough real world testing to convince ourselve there aren't
any unexpected side effects.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #938
2013-02-01 11:17:14 -08:00
Brian Behlendorf 1f7c30df8f Add zfs_disable_dup_eviction module option
Commit 1eb5bfa introduced a new zfs_disable_dup_eviction tunable.
It should have been made available as a module option in the
original patch but was overlooked.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-02-01 09:57:57 -08:00
Brian Behlendorf f52b31eaf0 Honor 80 character limit in 'zpool status'
This is a minor nit, but the second line of the 'action' message
when you need to upgrade your pool to support feature flags exceeds
the standard 80 character limit.  Fix it by moving the word
'feature' on to the third line.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-31 11:06:57 -08:00
Ned Bass 36f86f73f6 Fix mismatch between SA header size and layout
When a system attribute layout is created an inconsistency may occur
between the system attribute header (sa_hdr_phys_t) size and the
variable-sized attribute count stored in the layout.  The inconsistency
results in the following failed assertion when SA_HDR_SIZE_MATCH_LAYOUT
returns false:

SPLError: 11315:0:(sa.c:1541:sa_find_idx_tab())
ASSERTION((IS_SA_BONUSTYPE(bonustype) && SA_HDR_SIZE_MATCH_LAYOUT(hdr,
tb)) || !IS_SA_BONUSTYPE(bonustype) || (IS_SA_BONUSTYPE(bonustype) &&
hdr->sa_layout_info == 0)) failed

The bug originates in this snippet from sa_find_sizes().

    if (is_var_sz && var_size > 1) {
            if (P2ROUNDUP(hdrsize + sizeof (uint16_t),
                *total < full_space) {
                    hdrsize += sizeof (uint16_t);

This assumes that the current variable-sized attribute will be stored in
the current buffer and accounts for the space needed to store its size
in the sa_hdr_phys_t. However if the next attribute spills over we need
to store a blkptr_t at the end of the bonus buffer to point to the spill
block. If the current attribute is in the way of the blkptr_t then it
too will be relocated into the spill block. But since we've already
accounted for it in the header size we get the inconsistency described
above.

To avoid this, record the index of the last variable-sized attribute
that prompted a hdrsize increase, and reverse the increase if we later
determine that that attribute will be relocated to the spill block.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1250
2013-01-31 10:31:19 -08:00
Ned Bass 67629d0f08 Fix rounding discrepancy in sa_find_sizes()
A rounding discrepancy exists between how sa_build_layouts() and
sa_find_sizes() calculate when the spill block needs to be kicked in.
This results in a narrow size range where sa_build_layouts() believes
there must be a spill block allocated but due to the discrepancy there
isn't.  A panic then occurs when the hdl->sa_spill NULL pointer is
dereferenced.

The following reproducer for this bug was isolated:

    truncate -s 128m /tmp/tank
    zpool create tank /tmp/tank
    zfs create -o xattr=sa tank/fish
    ln -s `perl -e 'print "z" x 41'` /tank/fish/z
    setfattr -hn trusted.foo -v`perl -e 'print "z"x45'` /tank/fish/z

This test results in roughly the following system attribute (SA)
layout:

  176 bytes - "standard" SA's
   41 bytes - name of symbolic link target
  100 bytes - XDR encoded nvlist for xattr
  ---
  317 bytes - total

Because 317 is less than DN_MAX_BONUSLEN (320), sa_find_sizes()
decides no spill block is needed. But sa_build_layouts() rounds 41 up
to 48 when computing the space requirements so it tries to switch to
the spill block.

Note that we were only able to reproduce this bug using a combination
of symbolic links and the Linux-specific xattr=sa dataset property.
So while this issue is not technically Linux-specific, it may be
difficult or impossible to hit the narrow size range needed to
reproduce it on other platforms.

To fix the discrepancy, round the running total in sa_find_sizes() up
to an 8-byte boundary before accounting for each SA, since this is how
they will be stored in the bonus and (possibly) spill buffers.

To make the intent of the code more clear, explicitly assert key
assumptions about expected alignment of data and whether spill-over
will occur.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1240
2013-01-31 10:31:13 -08:00
Adam H. Leventhal 89103a2643 Illumos #3447 improve the comment in txg.c
3447 improve the comment in txg.c

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Richard Elling <richard.elling@dey-sys.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  illumos/illumos-gate@adbbcfface
  https://www.illumos.org/issues/3447

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-30 08:55:20 -08:00
Brian Behlendorf dbf763b39b Retire zpool_id infrastructure
In the interest of maintaining only one udev helper to give vdevs
user friendly names, the zpool_id and zpool_layout infrastructure
is being retired.  They are superseded by vdev_id which incorporates
all the previous functionality.

Documentation for the new vdev_id(8) helper and its configuration
file, vdev_id.conf(5), can be found in their respective man pages.
Several useful example files are installed under /etc/zfs/.

  /etc/zfs/vdev_id.conf.alias.example
  /etc/zfs/vdev_id.conf.multipath.example
  /etc/zfs/vdev_id.conf.sas_direct.example
  /etc/zfs/vdev_id.conf.sas_switch.example

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #981
2013-01-29 12:23:17 -08:00
Brian Behlendorf 79c6e4c445 Remove NPTL_GUARD_WITHIN_STACK
Commit 4b2f65b253 increased the user
space stack by 4x to resolve certain stack overflows.  As such it
no longer makes sense to worry about a single extra page which
might or might not be part of the process stack.  There is now
ample headroom for normal usage.

By eliminating this configure check we are also resolving the
following segfault which intentionally occurs at configure time
and may be logged in dmesg.

  conftest[22156]: segfault at 7fbf18a47e48 ip 00000000004007fe
  sp 00007fbf18a4be50 error 6 in conftest[400000+1000]

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-29 10:58:20 -08:00
Eric Dillmann 9759c60f1a Illumos #3035 LZ4 compression support in ZFS and GRUB
3035 LZ4 compression support in ZFS and GRUB

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <csiden@delphix.com>

References:
  illumos/illumos-gate@a6f561b4ae
  https://www.illumos.org/issues/3035
  http://wiki.illumos.org/display/illumos/LZ4+Compression+In+ZFS

This patch has been slightly modified from the upstream Illumos
version to be compatible with Linux.  Due to the very limited
stack space in the kernel a lz4 workspace kmem cache is used.
Since we are using gcc we are also able to take advantage of the
gcc optimized __builtin_ctz functions.

Support for GRUB has been dropped from this patch.  That code
is available but those changes will need to made to the upstream
GRUB package.

Lastly, several hunks of dead code were dropped for clarity.  They
include the functions real_LZ4_uncompress(), LZ4_compressBound()
and the Visual Studio specific hunks wrapped in _MSC_VER.

Ported-by: Eric Dillmann <eric@jave.fr>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1217
2013-01-29 09:28:20 -08:00
Brian Behlendorf ff5b1c8065 Quiet mkfs.ext2 output
The -q option should quiet the mkfs.ext2 output but certain
versions of e2fsprogs appear to ignore it.  This can result in
an extra 'done' message in the test output.  To keep this noise
from distracting just direct stdout to /dev/null.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-28 15:36:02 -08:00
Brian Behlendorf 2b7ab9d4d9 Linux 2.6.26 compat, lookup_bdev()
It's doubtful many people were impacted by this but commit 6c28567
accidentally broke ZFS builds for 2.6.26 and earlier kernels.  This
commit depends on the lookup_bdev() function which exists in 2.6.26
but wasn't exported until 2.6.27.

The availability of the function isn't critical so a wrapper is
introduced which returns ERR_PTR(-ENOTSUP) when the function isn't
defined.  This will have the effect of causing zvol_is_zvol() to
always fail for 2.6.26 kernels.  This in turn means vdevs will
always get opened concurrently which is good for normal usage.
This will only become an issue if your using a zvol as a vdev in
another pool.  In which case you really should be using a newer
kernel anyway.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1205
2013-01-28 15:35:00 -08:00
Brian Behlendorf 930b6fec21 Stop using /bin/ as a source in zconfig.sh
Test 5, 6, 7, and 7 in zconfig.sh use /bin/ as a source of random
directories and files for their test.  This has lead to unexpected
tests failures because the total size of /bin/ on the test system
isn't checked and it is entirely possible for it to be larger than
the target filesystem.

To resolve this issue we create a somewhat random collection of
files and directories in /var/tmp to use.  On average we expect
about 5MB of data with the worst case being 20MB.  This is large
enough to be interesting and small enough to always fit in the
default test datasets.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1113
2013-01-28 14:51:26 -08:00
Brian Behlendorf 14ee71efbc Use strerror() not strerror_r()
The differ() function used strerror_r() instead of strerror() because
it allowed the error message to be directly copied in to a buffer.
This causes two issues under Linux.

* There are two versions of strerror_r() available an XSI-compliant
  version which returns an 'int' error code.  And a GNU-specific
  version which return a 'char *' to the resulting error string.

    int strerror_r(int errnum, char *buf, size_t buflen);   /* XSI */
    char *strerror_r(int errnum, char *buf, size_t buflen); /* GNU */

* The most recent versions of strerror_r() are annotated with the
  warn_unused_result attribute.  This causes the following warning
  since the upstream implementation casts the result to void.

    warning: ignoring return value of 'strerror_r', declared with
    attribute warn_unused_result [-Wunused-result]

The cleanest way to resolve both of these problems is just to use
strerror() and make a copy of the result in to the buffer.  This
resolves both issues and this is the only instance of strerror_r()
in the code base.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1231
2013-01-28 10:02:38 -08:00