Wednesday May 08, 2013

Cloning MacOS X from Linux

I just finished to work out a way to clone a MacOS X installation from Linux (for automatic installation of MacOS X with fai -- fully automatic installation). All tools discussed in the article are available in Debian 7.x (wheezy).


First we save the GUID (Globally Unique Identifier) Partition Table (GPT), so that we can restore the same partition table on the target machine. We use the commandline version of gdisk for this, named sgdisk

sgdisk --backup=/srv/macos/parttable.gpt.sgdisk /dev/sda

The content of the partitions can be cloned using the partclone tool. A default MacOS X installation has three partitions:

  1. EFI system partiton
  2. MacOS X (named "Customer")
  3. Recovery partition

Model: ATA APPLE HDD HTS547 (scsi)
Disk /dev/sda: 976773168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start       End         Size        File system  Name                  Flags
 1      40s         409639s     409600s     fat32        EFI system partition  boot
 2      409640s     975503591s  975093952s  hfs+         Customer
 3      975503592s  976773127s  1269536s    hfs+         Recovery HD

First we create backup images of all partitions:

partclone.vfat -I -c -s /dev/sda1 | gzip > /srv/macos/sda1.partclone.gz
partclone.hfsplus -I -c -s /dev/sda2 | gzip > /srv/macos/sda2.partclone.gz
partclone.hfsplus -I -c -s /dev/sda3 | gzip > /srv/macos/sda3.partclone.gz


To restore the saved MacOS X installation, boot another Mac using Linux (a Knoppix live Linux DVD will work).

Next we restore the partition table (GPT) using sgdisk (/dev/sda is the target disk, all data on that disk will be erased, be warned!):

sgdisk --load-backup=/srv/macos/parttable.gpt.sgdisk /dev/sda

as an alternative, the partitions can be created using parted and the GUID type codes set by sgdisk (important!):

sgdisk --zap /dev/sda
parted -s /dev/sda mklabel gpt
parted -s /dev/sda mkpart primary 40s 409639s
parted -s /dev/sda name 1 "'EFI system partition'"
parted -s /dev/sda set 1 boot on
parted -s /dev/sda mkpart primary 409640s 975503591s
parted -s /dev/sda name 2 "'MacOS X System'"
parted -s /dev/sda mkpart primary 975503592s 976773127s
parted -s /dev/sda name 3 "'Recovery HD'"
sgdisk -t 1:C12A7328-F81F-11D2-BA4B-00A0C93EC93B /dev/sda
sgdisk -t 2:48465300-0000-11AA-AA11-00306543ECAC /dev/sda
sgdisk -t 3:48465300-0000-11AA-AA11-00306543ECAC /dev/sda

Restore the partition content:

zcat /srv/macos/sda1.partclone.gz | partclone.vfat -r -o /dev/sda1
zcat /srv/macos/sda2.partclone.gz | partclone.hfsplus -r -o /dev/sda2
zcat /srv/macos/sda3.partclone.gz | partclone.hfsplus -r -o /dev/sda3

If all went well, the new disk is ready to boot MacOS X.

For reference, here are the GUID details of a default MacOS X 10.8 MacBook Pro install:

root@(none):~# sgdisk -p /dev/sda
Disk /dev/sda: 976773168 sectors, 465.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 497736B0-7EA0-4C45-AB5F-8841CD773D24
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 976773134
Partitions will be aligned on 8-sector boundaries
Total free space is 262157 sectors (128.0 MiB)

root@(none):~# sgdisk -i1 /dev/sda
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: 6749C82F-02C9-4FF5-A889-F31A21726F8E
First sector: 40 (at 20.0 KiB)
Last sector: 409639 (at 200.0 MiB)
Partition size: 409600 sectors (200.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI system partition'

root@(none):~# sgdisk -i2 /dev/sda
Partition GUID code: 48465300-0000-11AA-AA11-00306543ECAC (Apple HFS/HFS+)
Partition unique GUID: 60F7A3AA-69B3-4E59-87A0-3A47BB659255
First sector: 409640 (at 200.0 MiB)
Last sector: 975241447 (at 465.0 GiB)
Partition size: 974831808 sectors (464.8 GiB)
Attribute flags: 0000000000000000
Partition name: 'MacOS X System'

root@(none):~# sgdisk -i3 /dev/sda
Partition GUID code: 48465300-0000-11AA-AA11-00306543ECAC (Apple HFS/HFS+)
Partition unique GUID: FD007AA4-CF3A-42F6-BFC6-B3BC25521FC2
First sector: 975503592 (at 465.2 GiB)
Last sector: 976773127 (at 465.8 GiB)
Partition size: 1269536 sectors (619.9 MiB)
Attribute flags: 0000000000000000
Partition name: 'Recovery HD'

Monday Jul 30, 2012

XCode 4.4 update does not install command line tools (gcc, clang ...)

After upgrading to MacOS X 10.8 (Mountain Lion) and XCode 4.4 through the Apple App store, I found myself unable to compile software on the command line. Apple has split the command line tools in to a separate download, accessible via the MacOS Dev Center download page:

screen shot 2012-07-30 at 3.20.12 pm.png

Dashcode and auxiliary tools like PackageMaker are now also separate downloads and not in the main XCode install anymore.

Monday Mar 05, 2012

Ein durchschnittlicher Font für diese Webseite ...

Gibt es etwas besseres als den echten Durchschnitt? Der Zeichensatz Averia zeigt, das auch der Durchschnitt interessant sein kann.

Monday Apr 18, 2011

Spring cleaning MacPorts

The MacPorts projects offers a fine, easy way to install Unix tools and applications on MacOS X. Over time however, MacPorts can accumulate amounts of dead data, as when applications get updated, the old versions stay until they are removed manually. In addition, after compiling a port from source, both the source and the intermediate object code remains on the harddisk.

Two commands can spring clean the MacPorts installation.

sudo port clean --all installed
will run "make clean" on all installed ports, removing the temporary object code generated during compilation.

sudo port -f uninstall inactive
will remove 'inactive' ports, mainly older versions of applications that have been replaced by a more recent version.

Running these two commands can free up some gigabyte of space on a harddisk (depending on the amount of MacPort applications installed).

Thursday Apr 07, 2011

Managing the MacOS X IPv6 firewall

MacOS X (10.3 and up) contains an IPv6 firewall (ip6fw), which has been inherited from FreeBSD and the KAME project. However there are no configuration or startup scripts, nor any other support available in a stock MacOS X system to manage this firewall.

The script presented here will read a firewall configuration from '/etc/ip6fw.conf' and will apply the IPv6 firewall rules to the MacOS X firewall.

[Read More]

Sunday Apr 03, 2011

Fixing the IPv6 Firewall on MacOS X 10.6

On MacOS X 10.6 (Snow Leopard), the IPv6 firewall command line utility 'ip6fw' is broken. It does not store filter rules for ICMPv6 types above type 127:

# sudo ip6fw add 20020 allow ipv6-icmp from any to any in icmptype 1,2,3,4,128,129
20020 allow ipv6-icmp from any to any in icmptype 1,2,3,4

Here is the fix...[Read More]

Saturday Mar 26, 2011

DNS information in IPv6 Router Advertisement on MacOS X

Pierre Ynard, the developer of 'rdnssd', was able to fix the issue with the 'ppoll' interface on MacOS X (see Debugging a shortlived MacOS application). The fix is now in the SVN code and will be available in the next release of 'rdnssd'.

It is now possible to distribute DNS server information to IPv6 clients via router advertisement messages.[Read More]

Thursday Mar 10, 2011

Debugging a shortlived MacOS application

Yesterday I had to debug a MacOS X commandline program that segfaulted immediatly after starting. This program is 'rdnssd' (Recursive DNS Servers discovery Daemon, 'rdnssd' implements the client part of RFC 5006 - IPv6 Router Advertisement Option for DNS Configuration. This function lets an IPv6 router send out DNS server IP address information as part of the Router Advertisment messages, helping client finding a DNS server without the need of DHCP or local configuration.[Read More]

Tuesday Aug 05, 2008

'Timemachine'ish backup with ZFS and rsync

Apple MacOS X Timemachine is a nice piece of software. However it does not compress the data, and it only works on MacOS X. I was looking to a similar solution that works also on other Unix Systems (as well as MacOS X) and does transparent compression of the data. The idea is to have multiple backups on one disk, each backup showing the state of the source hard disk or directory at the time of the backup, browsable by the normal file system tools, without storing any data duplicated. I found a solution using the ZFS file system and 'rsync' (rsync is pre-installed on most Unixish operating systems).


My tutorial is for MacOS X, but it can be adapted to any of the Systems that support the ZFS file system

Step 1: preparing a ZFS file system

I used the steps from the ZFS on MacOS-Forge site to create a ZFS pool on an external USB drive: Finding the disk:
# diskutil list . . . /dev/disk2 #: type name size identifier 0: Apple_partition_scheme *9.4 GB disk2 1: Apple_partition_map 31.5 KB disk2s1 2: Apple_HFS FW 9.2 GB disk2s3

writing a GPT label on the external disk (be sure to not format your 'main' disk here!):

# diskutil partitiondisk /dev/disk2 GPTFormat ZFS %noformat% 100% Started partitioning on disk disk2 Creating partition map [ + 0%..10%..20%..30%..40%..50%..60%..70%..80%..90%..100% ] Finished partitioning on disk disk2 /dev/disk2 #: type name size identifier 0: GUID_partition_scheme *9.4 GB disk2 1: EFI 200.0 MB disk2s1 2: ZFS 9.0 GB disk2s2

createing a ZFS pool on the disk called 'macbook-backup':

# zpool create macbook-backup /dev/disk2s2

enable compression on the new pool and disable ATIME:

# zfs set compression=on macbook-backup # zfs set atime=off macbook-backup

the hard drive is now prepared.

Step 2: creating the first (the 'base') backup

now I create the first full backup, which I call the 'base' backup. For this I create a new file system called 'base' in the ZFS pool 'macbook-backup':
# zfs create macbook-backup/base

next I copy all files from my backup-source directory (or the whole source disk) to the backup:

# rsync -avh --progress --delete /Users/myuser /Volumes/macbook-backup/base/

depending on the size of the data to backup, this will take a while.

Once the backup is finished, we can access all files under '/Volumes/macbook-backup/base'. With the ZFS command I can check the compression-ratio of our backup:

# zfs get all macbook-backup/base NAME PROPERTY VALUE SOURCE macbook-backup/base type filesystem - macbook-backup/base creation Wed Jan 31 9:08 2007 - macbook-backup/base used 2.21G - macbook-backup/base available 1.76G - macbook-backup/base referenced 2.21G - macbook-backup/base compressratio 1.38x - macbook-backup/base mounted yes - macbook-backup/base quota none default macbook-backup/base reservation none default macbook-backup/base recordsize 128K default macbook-backup/base mountpoint /Volumes/macbook-backup/base default macbook-backup/base sharenfs off default macbook-backup/base shareiscsi off default macbook-backup/base checksum on default macbook-backup/base compression on local macbook-backup/base atime off local macbook-backup/base devices on default macbook-backup/base exec on default macbook-backup/base setuid on default macbook-backup/base readonly off default macbook-backup/base zoned off default macbook-backup/base snapdir hidden default macbook-backup/base aclmode groupmask default macbook-backup/base aclinherit secure default macbook-backup/base canmount on default macbook-backup/base xattr on default

Step 3: creating an incremental backup

Now, a few weeks later, I want to make a new, incremental backup. So I create a new snapshot of the base file system and then clone that snapshot into a new file system:
zfs snapshot macbook-backup/base@20080804 zfs clone macbook-backup/base@20080804 macbook-backup/20080804

The directory for my new, incremental backup will be '/Volumes/macbook-backup/20080804'. So far the new file system does not use any space on the hard drive. Now I do the new backup with 'rsync':

# rsync -avh --progress --delete /Users/myuser /Volumes/macbook-backup/20080804/

The new backup will only take as much new space on the backup hard drive as there were changes compared to the base backup. But still I am able to browse through the file system at '/Volumes/macbook-backup/20080804' and see all files that were available at the date of the 2nd backup.

Any subsequent snapshots for more backups will be done from the 20080804 file system.