Xpra: Ticket #641: OSX .pkg creation scrips

Add OSX .pmdoc (Package Definition) with relative paths

Add make-pkg.sh script

Optionally add to make-all.sh

Tue, 19 Aug 2014 05:03:58 GMT - Scott Bennett:

Formatted the generated xml files so they can be maintained by hand.

Figured out how to make the paths in relative to pwd

Xml files can be regenerated from scratch by:

  1. Use packagemaker.app on OSX 10.5 or 10.6 (possibly 10.7)
  2. running through a code formatting utility (it all comes out on one very long line)
  3. Replace you absolute paths from /your/path/to/trunk/osx/image/Xpra.app to image/Xpra.app

Script (make-pkg.sh) must be run from the osx directory after running make-app.sh

.pmdoc and make-pkg.sh are tested and committed to trunk.

Tue, 19 Aug 2014 05:04:19 GMT - Scott Bennett: owner, status changed

Tue, 19 Aug 2014 05:09:03 GMT - Antoine Martin:

Looks to me like you've added those scripts and files in (was r 7336 - now removed)

We have to do something about the 01xpra-contents.xml: it cannot be kept in svn as it is, it weighs more than the whole xpra source tree combined:

$ du -sk osx/xpra.pmdoc/
4656	osx/xpra.pmdoc/
$ du -sk src/xpra/
3580	src/xpra/

We need some kind of script so we can generate it on the fly instead.

Tue, 19 Aug 2014 05:14:00 GMT - Scott Bennett:

Good point. Let's remove from SVN and host on a web server in a tar.gz and link to ticket for now if anyone wants it. I'll keep the task open and work on a script to generate this file for the next release.

Tue, 19 Aug 2014 05:44:18 GMT - Scott Bennett:

RE: (was in r 7337 - now removed)

  1. Removed large XML files and replaced with a single tar.bz2
  2. Added extract if-not-exists to make-pkg.sh

File size was reduced to 70K is this is still to big, let me know and I'll remove from SVN and post on web server with link.

Tue, 19 Aug 2014 08:24:09 GMT - Antoine Martin:

There were two big (literally) problems which forced me to completely rebuild the svn repository to remove those last two commits:

This file is generated by some tools, we need the commands and scripts used so we can generate it on the fly. Surely, a bit of sed or perl can do this? Otherwise, maybe we can just move to 10.6.x or 10.7.x as build platforms and use this tutorial Making OS X Installer Packages like a Pro - Xcode4 Developer ID Mountain Lion ready pkg. Having #533 finally closed would help.

Important: you will need to re-checkout the project since its history has changed. (safer than selectively pruning)

Thu, 18 Jun 2015 14:17:56 GMT - Antoine Martin:

Once that's done, it should be relatively straightforward to do: Submitting a Python App to the Mac App Store:

We need this to be able to do #407.

Thu, 02 Jul 2015 03:11:44 GMT - Antoine Martin:

As part of #888, I will probably do a large part of what is required for App Sandboxing. See also Migrating an App to a Sandbox (FWIW: this signing + sandboxing doesn't seem very useful, attackers can sign their apps too!) The main difficulty is going to be the unix domain sockets, which require special handling and permissions..

Some pointers:

So I don't see a way of keeping the sockets private without creating our own user directory as /var/tmp/$UID or even /var/tmp/$UID-Xpra..

Fri, 04 Sep 2015 07:48:52 GMT - Antoine Martin: milestone changed

scheduling for 0.16

Mon, 09 Nov 2015 10:17:44 GMT - Antoine Martin: priority changed

@sbennett: please attach the make-pkg.sh script.

Best reference I found:

Fri, 26 Feb 2016 03:44:52 GMT - Antoine Martin:

Excellent link which seems to allow us to sign PKG files without needing any of apple's tools: Signing a Mac OS X package on Linux.

Thu, 31 Mar 2016 12:18:51 GMT - Antoine Martin: owner, status changed

See also: Flat Package Format - The missing documentation

Assuming this gets resolved, we will then want to generate the pmdoc file rather than run a GUI tool every time. But this is Apple, they don't have a clue: their own packages are a complete mess, clearly generated by hand every time, with different spellings half the time, absolutely no consistency anywhere - that's why: you can't expect to automate something that they're not capable of automating themselves. End of rant.

This ticket is a blocker for doing printer forwarding with osx shadow servers (#391)

Sun, 10 Apr 2016 06:05:55 GMT - Antoine Martin: status, milestone changed

Sun, 10 Apr 2016 14:12:13 GMT - Antoine Martin:

Found a new link: bomutils: bomutils are a set of tools to create Mac OS X installer packages on foreign OSes (windows, linux, etc.).. Including a tutorial.

Other relevant links: Porting PackageMaker command line build installer to pkgbuild.

I can build a package using this command line (done on a 10.8 system after building the app on the usual 10.5 system):

/Developer/usr/bin/packagemaker \
    --title "Xpra" \
    --root ./image/ \
    --version 0.17 \
    --filter "\.DS_Store" \
    --root-volume-only \
    --domain system \
    --verbose \
    --no-relocate \
    --target 10.5 \
    --id org.xpra.pkg \
    --out Xpra.pkg

It takes a very very long time, even on a fast i7 with superfast SSD to match - whatever it is doing, it's not IO, just wasting CPU cycles? Interestingly, the "Activity Monitor" shows that "PackageMaker?" is the only non-64bit process running.. (hovering at 8% CPU usage) The resulting pkg does absolutely nothing when you run it, it claims that 360MB of disk space will be used, shows a shiny progress bar - but does absolutely zilch.

Trying again with pkgbuild:

pkgbuild --root ./image \
    --version 0.17.0
    --install-location "/Applications" \
    --identifier "org.xpra.pkg" \

This one works fine, tested on 10.5 and 10.8. Let's merge this, and try to get file association working too #407, and maybe also the cups backend (see ticket:391#comment:9).

Mon, 11 Apr 2016 10:17:16 GMT - Antoine Martin: owner, status changed

r12373 will now spit out a PKG. I have uploaded one made with this script.

bomutils was the way to go: it can be automated easily enough via a simple shell script and it works on both OSX and Linux.

This needs to be added to the moduleset:

Now that we have everything in place: a "flat" pkg, a newer build of openssl (#876), we may be able to do the signing as per comment:11.

@smo: if this package works for you, please merge those new packages into the moduleset and try to do the signing - then I can pay the apple tax and get my packages signed too.

Tue, 12 Apr 2016 12:10:17 GMT - Antoine Martin:

Related changes in r12376 for #391.

We should now be able to deal with #407 (session file association).

Also found some good information on "flat" packages: http://matthew-brett.github.io/docosx/flat_packages.html.

Wed, 04 May 2016 23:20:21 GMT - Smo:

cpio added to moduleset r12530

Wed, 04 May 2016 23:34:40 GMT - Smo:

Having issues adding xar and I think we pull python-rencode from this same source.

I don't think there is much to do on these old systems other than update curl?

curl -k --continue-at - -L https://github.com/downloads/mackyle/xar/xar-1.6.1.tar.gz
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Thu, 05 May 2016 03:09:50 GMT - Antoine Martin:

We already build ssl, so building our own version of curl should be fine.

Thu, 05 May 2016 18:02:58 GMT - Smo:

The problem with building our own version of curl is jhbuild uses curl to download stuff so we have an issue.

Maybe we can just host these 2 source packages ?

Mon, 16 May 2016 19:53:12 GMT - Smo:

Having another issue with curl this time getting openssl which we will need to build new curl. This is becoming increasingly harder on this old outdated machine to maintain.

curl -k --continue-at - -L http://www.openssl.org/source/openssl-1.0.2h.tar.gz - o /Users/user/gtk/source/pkgs/openssl-1.0.2h.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   316  100   316    0     0    888      0 --:--:-- --:--:-- --:--:--     0
curl: (35) Unknown SSL protocol error in connection to www.openssl.org:443
*** Error during phase checkout of openssl: ########## Error running curl -k --continue-at - -L http://www.openssl.org/source/openssl-1.0.2h.tar.gz -o /Users/user/gtk/source/pkgs/openssl-1.0.2h.tar.gz *** [80/115]

Tue, 17 May 2016 03:05:17 GMT - Antoine Martin:

Openssl can be downloaded via ftp:

$ curl --version
curl 7.16.4 (i386-apple-darwin9.0) libcurl/7.16.4 OpenSSL/0.9.7l zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz
osx-2:~ osx$ curl -O ftp://ftp.openssl.org/source/openssl-1.0.2h.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5150k  100 5150k    0     0   614k      0  0:00:08  0:00:08 --:--:--  975k

Thu, 09 Jun 2016 19:00:12 GMT - Smo:

r12770 changes to ftp for openssl

r12771 adds xar

Tue, 26 Jul 2016 08:42:33 GMT - Antoine Martin:

I do have a developer account which should allow me to sign the packages, but Is it possible to have GPL software in the Mac App Store? could be a problem.

Thu, 11 Aug 2016 17:26:19 GMT - Smo:

Another ssl issue downloading Xar

curl -k --continue-at - -L https://github.com/downloads/mackyle/xar/xar-1.6.1.tar.gz -o /Users/build/gtk/source/pkgs/xar-1.6.1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   129    0   129    0     0    178      0 --:--:-- --:--:-- --:--:--   183
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) SSL peer handshake failed, the server most likely requires a client certificate to connect
*** Error during phase checkout of xar: ########## Error running curl -k --continue-at - -L https://github.com/downloads/mackyle/xar/xar-1.6.1.tar.gz -o /Users/build/gtk/source/pkgs/xar-1.6.1.tar.gz *** [1/1]

I can't seem to find any mirror

Fri, 12 Aug 2016 01:30:12 GMT - Antoine Martin:

Once you have openssl, you can build curl. Isn't that enough? If needed, we can download / update the ca cert list to be able to download from github.

Thu, 18 Aug 2016 15:12:49 GMT - Smo:

r13386 adds curl

I can't seem to find the default location or how to set the default for the certificates so this is what i've done inside jhbuild shell to make certificates work

export SSL_CERT_FILE="$HOME/gtk/inst/etc/ssl/cacert.pem"
curl -o ${SSL_CERT_FILE} -k https://curl.haxx.se/ca/cacert.pem

Afterwards I can download/build xar with jhbuild buildone xar from within jhbuild shell.

Thu, 18 Aug 2016 15:42:50 GMT - Smo:

r13387 fixes checkout location for bomutils

I still have an issue with this one when I build with this error

make install DESTDIR=/Users/build/gtk/inst/_jhbuild/root-bomutils
install -d /Users/build/gtk/inst/_jhbuild/root-bomutils/usr/bin
install -d /Users/build/gtk/inst/_jhbuild/root-bomutils/usr/share/man/man1
install -m 0755 build/bin/mkbom build/bin/dumpbom build/bin/lsbom build/bin/ls4mkbom /Users/build/gtk/inst/_jhbuild/root-bomutils/usr/bin
install -m 0644 build/man/mkbom.1.gz build/man/dumpbom.1.gz build/man/lsbom.1.gz build/man/ls4mkbom.1.gz /Users/build/gtk/inst/_jhbuild/root-bomutils/usr/share/man/man1
*** Error during phase install of bomutils: Module failed to install into DESTDIR u'/Users/build/gtk/inst/_jhbuild/root-bomutils-broken' *** [1/1]

It appears to support DESTDIR and i've checked the makefile so i'm not sure why jhbuild doens't like this?

If I copy the binaries to their proper location ~/gtk/inst/bin I can build the pkg no problem

I'll have someone test the pkg to make sure all is well.

Now we seem to have 3 issues to fix with jhbuild openssl, curl and bomutils

Thu, 18 Aug 2016 15:43:45 GMT - Smo: owner changed

Fri, 19 Aug 2016 19:11:38 GMT - J. Max Mena:

Smo asked me to paste the error here, unfortunately it is extremely vague and un-helpful:

Aug 18 12:21:20 Max-Trashcan installd[243]: PackageKit: Install Failed: Error Domain=PKInstallErrorDomain Code=112 "An error occurred while running scripts from the package “Xpra-0.17.5-r13398.pkg”." UserInfo={NSFilePath=./postinstall, NSURL=file://localhost/Users/max/Downloads/Xpra-0.17.5-r13398.pkg#base.pkg, PKInstallPackageIdentifier=org.xpra.pkg, NSLocalizedDescription=An error occurred while running scripts from the package “Xpra-0.17.5-r13398.pkg”.} {
     NSFilePath = "./postinstall";
     NSLocalizedDescription = "An error occurred while running scripts from the package \U201cXpra-0.17.5-r13398.pkg\U201d.";
     NSURL = "file://localhost/Users/max/Downloads/Xpra-0.17.5-r13398.pkg#base.pkg";
     PKInstallPackageIdentifier = "org.xpra.pkg";

Also, the machine on which I tested this is OSX 10.11.5.

Sun, 21 Aug 2016 05:31:32 GMT - Antoine Martin: owner changed

The mistake here is to think that the Apple mothership will let you do what you want, even if you have root on your device. Because "security" of course.

The postinstall error is caused by Apple's How to fix permission issues on Homebrew in OS X El Capitan? for details.

Both should be backported. I can now install the PKG on 10.11.x again.

Thu, 08 Sep 2016 19:08:42 GMT - Smo: owner changed

Got this lovely error updating openssl today

*** Configuring openssl *** [81/118]
./Configure --prefix=/Users/build/gtk/inst --openssldir=/Users/build/gtk/inst/etc/ssl shared  darwin-i386-cc && make install
Perl v5.10.0 required--this is only v5.8.8, stopped at ./Configure line 12.
*** Error during phase configure of openssl: ########## Error running ./Configure --prefix=/Users/build/gtk/inst --openssldir=/Users/build/gtk/inst/etc/ssl shared  darwin-i386-cc && make install *** [81/118]

Trying to close this ticket but seems now I need to build perl on this old system?

Fri, 09 Sep 2016 06:14:16 GMT - Antoine Martin: owner changed

Yes, perl is needed on older build systems: ticket:840#comment:22

Wed, 21 Sep 2016 00:28:34 GMT - Smo:

jhbuild meta-osx-xpra-pkgtools

This will build all that is necessary to make the pkg.

There is still an issue with bomutils so you must copy these.

Thu, 22 Sep 2016 23:13:36 GMT - Smo: status changed; resolution set

problem with bomutils has been fixed in r13828

Everything works as it should now.

Thu, 13 Oct 2016 16:49:39 GMT - Antoine Martin:

Follow up for signing and appstore: #1340.

Sat, 23 Jan 2021 05:01:42 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/641