Auto-update of native application bundles
Igor Nekrestyanov
igor.nekrestyanov at oracle.com
Sun Jun 17 16:12:45 PDT 2012
On 6/17/12 4:00 PM, Daniel Zwolenski wrote:
> Hi Igor,
>
> I think I'm missing some part of the puzzle. From what you are saying
> for an upgrade you would push up the entire new app (as an MSI or some
> custom format?). This new file will contain all the JARs and the JRE
> regardless of whether it's changed or not - i.e. this is similar to my
> approach except in my approach it's all extracted but in yours it's
> all packed into one file. Is this understanding correct?
Update packages might have different formats and which one to use may
depend on specific use case.
(think about it as what image format to use for image file)
It is required to be one file but does not require to have everything in
it. It may have everything.
"Full bundle" in the file should be supported for sure (and it could be
simply zip file or something more advanced).
We can also support some "delta bundle" formats that will be only
applicable for upgrade from specific version.
E.g. i can imaging appcast that says something like:
- if you are running previous version of my app (1.4.5) then you
only need this package with one resource file updated
- if you are running version 1.3 or later then use this package (for
simplicity zip) with application files only
- if you are running older version then you need full bundle as JRE
was updated
> How then does the system work out what needs to be downloaded and
> updated? It has a locally extracted version of the app and a remote
> file (effectively a zip), how does it determine what's changed?
It is defined at the package time.
When developer packages his application he needs to decide if he wants
delta updates.
If not, he simply refer to "full bundle"in the appcast.
If he wants delta updates he need to point to images to be used as
baselines and then packager will figure out what is updated based on
binary comparison.
Let me know if this is still confusing,
-igor
>
> I think I missed something in your description.
>
>
>
> On Mon, Jun 18, 2012 at 8:31 AM, Igor Nekrestyanov
> <igor.nekrestyanov at oracle.com <mailto:igor.nekrestyanov at oracle.com>>
> wrote:
>
> Hi,
>>
>> * In your proposal the "app-cast" system is used, where each
>> update is released as a patch (i.e. the 'diff') to get from
>> one version to the next. If an app was out in the wild
>> running at version 2, and the latest release was actually
>> version 5, then to upgrade we would download the v2->v3
>> patch, install it, then the v3->v4 patch install it, and then
>> the v4->v5 patch and install that (correct me if I got this
>> wrong).
>>
> Sparkle appcast is simply update descriptor (kind of RSS) wth news
> about update availability.
> It is unrelated to the way update is shipped.
>
> Moreover but default update is full image of application and can
> be installed on top of any version.
> Developer may choose to have delta updates if he wants to optimize
> but he is not required too.
> We are using Sparkle to AU JRE on Mac and it works fine.
>>
>> * In my POC I've gone for the "app-profile" approach, which
>> basically defines the latest release of the system in it's
>> entirety. It's up to each app out there to determine what has
>> changed between it's version and the latest version and
>> download what it needs (i.e. the 'diff'). To go from version
>> 2 to version 5, the app would compare the version of the JARs
>> in its own app-profile against the new one, and download the
>> ones that have changed. There is no download of the
>> 'in-between' versions of 3 and 4.
>>
> Sparkle approach does not assume this either.
> Update description must have full image. If there is (optional)
> delta image it is used but otherwise full image is used.
>
> IMHO, the biggest differences between appcast xml and
> app-profile.xml are:
> * appcast is strictly update descriptor and app-profile.xml is
> application descriptor + versioning info
> * app-profile.xml corresponds to subset of "item" element in
> the appcast
> * appcast xml may have multiple items - e.g. define application
> updates for different platforms or several apps
> * item in the appcast describes UPDATE, not application.
> Application is kind of blackbox and this is good.
> I.e. no knowledge of classpath or specific directory
> structure. Only what driver knows how to apply update.
> * item has update specific information, e.g. may include
> release notes that can be used to justify why to upgrade.
>
> It does not mean we can not use efficient compression (e.g.
> pack200 for jars) or update parts of app package (e.g. leave alone
> Java runtime).
> This is up for driver implementation.
>
> And another important difference is simplier model in Sparkle:
> * ONE version to check (whats available vs update version in
> appcast)
> [no need to check every jar, resource file, etc]
> * ONE file with update to sign and stage (less hassle for
> developer)
> [instead of every jar, fewer chance to have staging error
> (not all files are uploaded, etc)]
> * ONE file to download
> [less chance to have mixed state, e.g. developer may
> decide to restage/resign app with same version but some jars are
> already loaded]
>
> JNLP model is ok but it tries to solve "remote launch" problem and
> here we are solving other task.
>
> IMHO, if we separate launch descriptor and update descriptor we
> will have better model for this problem.
>
>>
>> It was with this type of usage in mind that I ended up leaning
>> towards the app-profile approach. I imagined my initial upload to
>> my server would be all the JARs for my app, and then when I
>> release subsequent versions I only upload the couple of JARs that
>> have changed and upload the new app-profile.xml (and the build
>> tools could potentially do the upload as well and they would know
>> which jars had changed).
> Essentially you are implementing specific kind of "delta update".
> Except you can dynamically define what delta is needed for
> arbitrary initial state.
>
> In sparkle model specifics of delta update are up to the update
> driver that knows what to do with update package.
> We can support update packages that only update "important" subset
> of jars, etc.
>
> For background type of updates size is often not an issue at all
> (although there are scenarios where it needs to be optimized).
>
> Anyway, with "update driver" plugins we can extend set of delta
> updates supported as needed.
>
>
>> The flip side of this is that you couldn't choose to go to an
>> in-between version if you wanted to, but I'm not sure that's an
>> overly valid use case?
> Agree, this is not important use case.
> But you may have "different" latest versions - one for XP and
> another for Vista and another for Mac.
>
>>
>> With this approach we also have the option of providing a
>> "repair" function. So a client could run a utility to just
>> completely download the entire app and get fresh copies of all
>> the JARs. This is not critical but it could be quite useful. With
>> the app-cast approach, you have to uninstall and then reinstall
>> with your original MSI and then patch (or install with the new
>> MSI) - the uninstall may delete your 'appdata' though, whereas a
>> "repair" could leave this intact.
> IMHO, repair by reinstalling still needs to be supported and
> existing mechanisms (like MSI or rpm) have means to do this.
> It does not seem as critical requirements for AU module.
>
>>
>> One other niggling thing for me is how to do things like system
>> properties or VM settings (like -Xmx) without an app-profile.
>> Currently you would be hard coding those into your 'exe' I
>> imagine? So if an update wanted to change these things it would
>> have to recompile the launcher exe and that would have to be part
>> of the 'patch'?
> Bundled app has application descriptor/config file. It is called
> package.cfg and very simple now.
> It may and need to be improved to pass in things like JVM options.
>
> In fact launcher is fixed, it is not compiled per app.
> It is embedded into ant task and simply renamed when application
> bundle is generated.
> All variable parameters are coming from package.cfg
>
>> There'd be no easy (possible?) way for the user to change these
>> settings on their machine, not sure if that's a good thing or a
>> bad thing though?
> Well, if we need it (it is questionable) then user may tweak
> package.cfg directly (imho, bad idea. especially if it is system
> wide) or we could have
> some abstraction of "config" API and application may update its
> settings per user. But this is unrelated to AU.
> User settings should be left intact after AU. Default app settings
> are part of application descriptor. So i think we are good with this.
>
>>
>> And finally one other edge use case I thought of today: it's not
>> uncommon to have several 'apps' included in the one install. For
>> example you might have a user UI and an admin UI. In Java-land,
>> these would ideally share the same libraries but have different
>> 'main' classes. I was thinking the app-profile approach could
>> potentially be extended to cover this but it needs more thinking
>> about, and may just be an unsupported use case. Not sure how/if
>> the app-cast approach could handle this if we said it was a
>> requirement.
> It does not seem hard to me.
>
> Key question here -- are they shipped as separate installs or one
> bundle with multiple launchers.
> If they are separate installs they are updated separately and
> there should be application specific means to ensure they are
> consistent
> (because user may simply install incompatible versions to start
> with). Application specific solution may use AU APIs to force
> autoupdate
> if user tries to launch app that is compatible with others.
>
> If they are shipped as one bundle then nothing special to be done
> by AU. AU will update folder with multiple launchers easily as
> they are just files for AU module.
> It is different story how to bundle msi/exe/dmg for this scenario
> but this is separate from AU discussion,
>
> -igor
>
>>
>> Cheers,
>> Dan
>>
>>
>> On Sun, Jun 17, 2012 at 5:26 PM, Igor Nekrestyanov
>> <igor.nekrestyanov at oracle.com
>> <mailto:igor.nekrestyanov at oracle.com>> wrote:
>>
>> Hi Dan,
>>
>> thanks for the great writeup (and pushing this ahead).
>> I finally started to look into this more actively. Some
>> questions/suggestions below.
>>
>> On 6/15/12 5:02 PM, Daniel Zwolenski wrote:
>>
>> *The General Approach*
>>
>> Richard made reference to
>> Sparkle<http://sparkle.andymatuschak.org/> and
>>
>> like him, I feel this is a nice reference point for the
>> style of solution
>> we want. The neat thing that Sparkle does is it provides
>> an API that
>> developers can use inside their code to completely
>> control and customise
>> the upgrade of their application. This is the approach
>> I've gone for as
>> well. My POC includes a Java API that you can call to
>> check for updates and
>> trigger downloads.
>>
>> IMHO, main great thing about Sparkle is that it is easy to
>> add basic AU functionality to the application
>> (see https://github.com/andymatuschak/Sparkle/wiki)
>> You do not need to use API unless you want custom solution.
>>
>> Therefore i am not focusing here on the public API part
>> (keeping this for later).
>>
>>
>> I think this approach is neat and powerful, but it's
>> worth mentioning that
>> there is still some debate amongst the JFX team about
>> whether to go down
>> this road or avoid the Java API and use off-the-shelf
>> native updaters. Your
>> input will very likely influence this decision so if you
>> have a preference
>> one way or the other (or have other suggestions), be vocal.
>>
>>
>> I think there is no debate on that having some "all in java
>> with minimal native code solution" is good idea.
>> Debate is about whether it should be the only option or it
>> also make sense to support other native update mechanisms
>> (e.g. use Sparkle directly on Mac).
>>
>> As Dan said, thoughts on this are very welcome.
>>
>>
>>
>> An XML deployment descriptor, called 'app-profile.xml' is
>> used to describe
>> the JARs, JRE versions and other launch details - it is
>> semantically
>> comparable to a traditional JNLP file. When an update is
>> triggered via the
>> API, the code checks the remote URL for an updated
>> app-profile.xml. If it
>> finds one it downloads this and checks this with its
>> local app-profile.xml.
>> Any jars that have different version numbers are
>> downloaded, and if a new
>> JRE version is defined in the app-profile then the zip
>> for this is
>> downloaded and extracted as well.
>>
>> Where these files are placed after download?
>>
>> Once all the downloading is complete, the
>> local app-profile is then overwritten with the newer one
>> and this newer one
>> will be used by the native launcher to set the classpath
>> next time the app
>> is started.
>>
>>
>> IMHO, we can simplify this model following Sparkle ideas.
>> Sparkle does not know anything about application structure,
>> etc. and focus only on update process.
>> This makes it easy to use, simple and robust as it focuses
>> on solves one problem.
>>
>> Important points are:
>> A. Sparkle uses "appcast" to describe what updates are
>> available
>>
>> (https://github.com/andymatuschak/Sparkle/wiki/publishing-an-update)
>> Appcast includes
>> - version of updates available
>> - meta info about each update (type of update (e.g.
>> regular or mandatory security), release notes, system
>> requirements (e.g. Windows version 7+), etc.)
>> - set of available update packages (aka enclosures)
>> (e.g. full app image, delta patches for several versions, etc.)
>>
>> B. Sparkle knows where to look for update meta info in the
>> application package (on Mac it is part of Info.plist that is
>> some kind of application manifest).
>> However, it only needs to know:
>> - URL of appcast
>> - location of key to verify update signatures
>> - application update settings/preferences
>> The expectation is that once app is updated this info
>> will be updated (as it is part of "update package")
>> [Note: this is not so simple on Windows if we want to
>> make sure we are good citizens and need to update registry]
>>
>> C. Update package is black box for Sparkle core.
>> It is modeled as
>> - file hosted at given URL
>> - signed with given key (signature is part of appcast)
>> - package type defines what "update driver" will be
>> handling this update
>> (this allows to add support for various update
>> packages as plugins/modules)
>> - for most type for packages (except running
>> installers) Sparkle goal is to update set of application
>> files only
>> [Windows or linux require more work as we may
>> need to update registry or package info]
>>
>> D. If admin permissions are needed they only asked before
>> update is finally installed.
>> Sparkle does not keep any internal state in the app
>> folder as it may be only writable for admins.
>>
>> E. Application relaunch after update
>>
>> This is close to your approach but there are some important
>> differences.
>> Following Sparkle more closely may simplify things and at the
>> same time add some desired features
>> (e.g. ability to validate origin of autoupdate)
>>
>> My current thoughts on "ideal" basic experience are as follows.
>>
>> ========
>> Developer experience (no java coding needed):
>>
>> 1. At the package time developer requests autoupdate support
>> in the native cobundle by providing:
>> * http URL to appcast + public key/cert to validate
>> update signatures
>> * or https url
>> and define application update mode (mandatory update, ask
>> user and allow him to skip version, etc.)
>>
>> 2. Once update is needed developer creates "update package"
>> (single file).
>> For this he will need to run same packager utility/ant
>> task but specify base image, certificate to sign package and
>> and desired type of update (full or incremental)
>>
>> 3. Developer need to create an appcast xml file (default can
>> be generated in step 2 but he may want to customize it)
>>
>> 4. Developer stages both appcast and package bundle
>> =========
>>
>> End user (background updates):
>> * installs application with background update policy
>> * every time application is launched it will check appcast
>> (if online)
>> * if update is detected it will be loaded silently while
>> app is running
>> * Once update is loaded and verified user will be prompted
>> to install it (it may be notification if update is mandatory)
>> * If he agrees then he may see UAC dialog (unless
>> installation is on per user basis)
>> * application relaunch after update
>> * attempt to launch app being updated should fail with
>> reasonable dialog and should not fail update process (it may
>> need to wait will .exe is unlocked)
>> =========
>>
>> Implementation wise i was thinking along the lines of what
>> you have (only glimpsed though your code it though) with
>> following
>> notable differences:
>>
>> * update package is single file and we only worry about it
>> version.
>> No need to worry about version of individual components
>> as we do not want arbitrary mix&match.
>>
>> * updates are loaded to ~/.javafx/app-key (where app-key
>> could be derivation from app install location)
>> We also keep user answers for this app in that folder
>> (e.g. had we offered him this version already? when was last
>> time we checked for update, etc)
>>
>> * Update process is:
>> - build temp full app image in the
>> ~/.javafx/app-key/version
>> - if update accepted then exit application and run
>> helper app co copy image over
>> - possibly run "post upgrade script" as part of helper app
>> (this script is specific to "bundler" technology
>> used. E.g. for EXE/MSI bundlers we may want to update few
>> keys in registry
>> - relaunch app
>> - AU module running in the relaunched app could detect
>> temp "image" that is the same as current version and erase it
>>
>> Note that there could be several alternative helper
>> apps. E.g. one that pops UAC and one that does not for per
>> user installs.
>> Or helper up to install update packaged as MSI (by
>> running msiexec silently and showing progress)
>>
>> I was even thinking that "helper" can be implemented in
>> java with minimal native code (to elevate permissions and
>> update registry if needed).
>>
>> * Integration with application bundle
>> - details on whether AU is needed and what type is it
>> may be embedded application manifest for main app jar
>> - embedded jauncher will read them and use internal AU
>> APIs accordingly
>> (this is why no user code changes are needed)
>> - current app version, update url and signature are
>> part of package.cfg or manifest in the main app jar
>> - AU support itself is part of jfxrt.jar, bundle also
>> includes helper app to "move image into final location" and
>> script to do "post update config"
>>
>> What do you think?
>>
>> -igor
>>
>>
>
>
More information about the openjfx-dev
mailing list