GitLab

If the user has not provided an OSFAMILY, snf-image will try to detect
it. On Windows, check the registry key:
'HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion' to determine if the
correct OSFAMILY is 'windows' of 'windows-legacy'.

Otherwise the log file gets filled with control characters and progress
info that is useless.

Add a new OSFAMILY for Windows OSes prior to Windows Vista that use the
old Windows NT sysprep format. For now, this is only tested in Windows
Server 2003 and Windows XP

There is no reason to treat it different from any other Configuration
Tasks that needs to run after InstallUnattend and before
EnforcePersonality.

Fix the code for OFFLINE_NTFSRESIZE and OFFLINE_NTFSRESIZE_NOCHECK
image properties.

The main goal of this patch set is to add support for Windows XP/Server
2003 Images.  In the process, it also adds:

 * The ability for a simple debug shell under KVM, so a developer or the
   administrator can access the helper VM directly for simpler debugging,
 * Robust helper shutdown and mounting/un-mounting of NTFS, fixing the
   remote possibility of data loss.

Conflicts:
	docs/interface.rst
	snf-image-helper/common.sh.in
	snf-image-helper/tasks/50AssignHostname.in
	snf-image-host/kvm-common.sh

Add a mechanism were scripts from within the image can be executed
instead of the original Configuration Tasks.

The ComputerName in Windows has strong restrictions:
    https://technet.microsoft.com/en-us/library/ff715676.aspx
If the name we set in the unattend.xml is not valid, then sysprep
will fail during the first boot.

This resolves #81

Make them comply with the rest of the variable names

Though this image property a user may define a base64 encoded executable
that will run as configuration task during the image deployment. In
order to support this, a new RunCustomTask configuration task has been
created.

Allow making a whole disk as SWAP. To do this, you need to define the
SWAP image property with a letter, denoting a hard disk numbering. If
you use: SWAP=b then snf-image will configure the second disk of the
image as swap.

This resolves #63

If parted supports the resizepart command, use it to enlarge a
partition in favor of removing and recreating it.

If this option is specified, the resulting helper image will not be
shrank to reduce its size. This is useful when debugging. If the image
is shrank to his minimal size, then altering files inside the image
after mounting is not possible.

Check if the provided file exists and is an snf-image-helper package

 * Add an kernel configuration file that can be used to construct the
   helper image
 * Add instructions under the "Development" section on how to build a
   stripped down kernel for snf-image-helper

 * Use jessie as suite in "Debian" and "GRNet" sections
 * Add upstart and python-bcrypt packages from jessie
 * Use kernel from jessie-helper

Fix a bug in snf-image-create-helper where the script would fail if
the multistrap file did not contain an snf-image-helper package,
although this is allowed if the user has provided one from the
command-line.

The -s option is deprecated and removed in newer versions

Credits: psomas@grnet.gr

Attach all the instance's disks to the helper VM.

* Support ED25519 keys
* Use dpkg-reconfigure openssh-server to recreate the host keys in
  Debian/Ubuntu

This resolves #79

Use check_yes_no() to test if a boolean property is set.

Although the documentation states that to set a boolean image property
you need to assign the "yes" value to it, this function will accept
"yes", "true", 1, "on" and "set" in a case-insensitive way and reject
"no", "false", 0, "off" and "unset".

An empty or not-set variable is treated as false.

An unknown value will raise a warning but will be accepted. This is done
to protect the users because prior to this commit, in some cases we only
tested if an image property had a non-empty value.

This resolves #80

Enhance the DisableRemoteDesktopConnections task,
so disabling and re-enabling RDP is more robust,
and respects Image-specific policy.

Previously, snf-image would disable RDP unconditionally
inside DisableRemoteDesktopConnections and assume there would be
an appropriate <RunSynchronousCommand> entry in unattend.xml
so SYSPREP would enable RDP unconditionally during the specialize
pass of the Windows Setup.

This has two main problems:
   * It assumes a specific answer file, with snf-image specific content.
     However, the answer file is Image-specific policy, and ideally
     snf-image should not make any assumption on its contents.
   * It enables RDP unconditionally, even though it may have been
     disabled inside a specific Image by the Administrator, on purpose,
     thus introducing a potential security risk.

To solve this problem, make DisableRemoteDesktopConnections
self-standing:
   * Note whether RDP was initially disabled or not,
   * Disable it unconditionally via a direct edit of the Registry,
     so no incoming RDP connections are allowed while SYSPREP is running,
   * Insert a command to set it to its original state when Setup is
     complete, without depending on the contents of unattend.xml
     or other answer file.

Introduce support for Windows XP / Server 2003 Images.

To do this:
    * Extend common.sh and 40InstallUnattend so they can also detect
      Windows XP / Server 2003 SYSPREP.INF answer files.
    * Extend 50AssignHostname so it can set the hostname inside
      SYSPREP.INF, based on a small handle-ini-file.py utility.
    * Remove the seemingly unnecessary addition of /LOGONPASSWORDCHG:NO,
      which is unsupported under XP / Server 2003. More on this below.
    * Warn the user about Windows XP / Server 2003 not supporting
      online NTFS resize, and the need to use OFFLINE_NTFSRESIZE
      instead.

Regarding the use of /LOGONPASSWORDCHG:NO while using NET USER
to change a user password:
    * This argument is unsupported under Windows XP / Server 2003,
      see http://blog.johnmuellerbooks.com/2011/04/12/working-with-net-user/
    * Its default value is "NO" anyway, so it shouldn't make a
      difference whether it is explicitly specified in the command line
      or not:
      https://answers.microsoft.com/en-us/windows/forum/windows_vista-security/setting-up-passwords-for-new-users/1704349b-31a3-4340-ae9e-1473c5adb919
    * Even if the security policies of a specific Image were set up
      in such way that users *were* required to change their passwords
      immediately upon their first logon, it is not snf-image's job
      to modify this behavior by specifying /LOGONPASSWORDCHG:NO.
      The password policy is Image-specific, and snf-image shouldn't
      mess with it.

Move DisableRemoteDesktopConnections from priority 40
to priority 41, ensuring it runs after 40InstalUnattend.in.

This makes the dependency between the installed answer file and
the DisableRemoteDesktopConnections task explicit:
The task assumes that RDP connections will be re-enabled via
an appropriate <RunSynchronousCommand> entry in the answer file,
which must already exist.

Making the dependency explicit, allows making the process more robust
in the future: The DisableRemoteDesktopConnections task should not
blindly assume that a potentially Image-specific unattend.xml file
contains the specific <RunSynchronousCommand> entry it requires,
but may insert it explicitly, since the answer file is bound to have
been detected or installed via the the previously executed
40InstallUnattend task.

snf-image already supports online resizing of NTFS; it creates
an appopriate DISKPART script inside the target NTFS and assumes
it will be called by SYSPREP via a pre-existing <RunSynchronousCommand>
entry in the unattend.xml answer file. This is generally the safest
option, since it uses native Windows code, but has two drawbacks:
a) It is only supported by Windows Vista and later,
b) It is possible the Image will fail before SYSPREP has a chance
to run the DISKPART script, because it does not have enough free space.

Extend snf-image to also support offline resize of NTFS via ntfsresize,
before the Image is booted. This works with all Windows versions and
ensures the Image is resized to the right size even before booting.

To be on the safe side, offline NTFS resize is not the default:

The user must set the OFFLINE_NTFSRESIZE property explicitly.
Running ntfsresize leaves the filesystem dirty, i.e., a CHKDSK is
performed during the next boot. The user may set the
OFFLINE_NTFSRESIZE_NOCHECK property to skip this.

Use lowntfs-3g with appropriate options when mounting NTFS-based Images:
    * ensure path lookup is case-insensitive,
    * prevent the creation of files with names which are not allowed
      under Windows,
    * complain loudly if the filesystem is dirty or needs recovery.

Mounting the filesystem in a case-insensitive way can simplify task code
significantly; there is no reason to perform case-insensitive lookups
explicitly ("Unattend.xml" vs. "unattend.xml"). It also ensures
attempting to inject "filea" in the image will overwrite "fileA", if it
already exists, as it would under Windows, instead of leading to a
situation where both "fileA" and "filea" exist, causing all sorts of
problems later on.

Finally, complain loudly when attempting to mount an NTFS marked dirty
(requiring a disk check on next boot), or with an unclean journal.
NTFS-3G code seems to wipe the NTFS journal instead of replaying it, and
may lead to data corruption. Images should not contain dirty
filesystems.

See
http://tuxera.com/forum/viewtopic.php?f=2&t=30562:
"Actually the journal is simply wiped out. This is to prevent the
journal to be applied at next mounting on Windows to data which may have
been changed in the meantime.", and
"So far, nobody has been able to understand how the journal is
organized, so there is no real recovery in ntfs-3g, just wiping the
journal."

Also:
http://www.tuxera.com/community/ntfs-3g-manual/
"recover: Recover and try to mount a partition which was not unmounted
properly by Windows. The Windows logfile is cleared, which may cause
inconsistencies. Currently this is the default option."

In the future, it would be best to reject the Image outright,
instead of continuing.

Umounting filesystems mounted via NTFS-3G is not synchronous.
It is possible that NTFS-3g mount processes remain, and continue
writing to the underlying block device, even after umount has completed
successfully. To solve this, wait explicitly for all NTFS-3G mount
processes to terminate, before continuing.

Similarly, there is no guarantee that all of the cached data have
been flushed to disk when shutting down the system abruptly using
the 'o' sysrq key to shut off the system immediately.
To solve this, run "sync" explicitly before shutting system off,
assuming no other process is currently writing to the disk (see above).

The combination of these two issues could lead to data loss.

Implement HELPER_DEBUG mode. When enabled, the helper VM will drop to a
root shell whenever a task fails. This allows the administrator or a
developer to examine its internal state for debugging purposes.

Also add missing CONTRIBUTORS files.

If this property is defined, then only the tasks that are meant to run
before the VM's disk gets mounted (namely FixPartitionTable and
FilesystemResizeUmounted) will be allowed to run during deployment.

This resolves #77

Bump version to 0.18.1next
Conflicts:
	docs/version.py
	version
	version.m4

Most network drivers used to call cidr2mask inside an echo command. This
is dangerous because if the function failed, the execution would not
stop because echo would not propagate the non-zero status of the
function to the shell.

The ConfigureNetwork task should configure a NIC to perform dhcp on boot
if ganeti provides an IP address for this NIC but not a SUBNET.

This resolves #76

Credits to atnaskos@gmail.com for the bug report!

Bump version to 0.18next