Auto-update of native application bundles

[Daniel Zwolenski]

Thanks Jose for you feedback - much appreciated.

One topic I forgot to mention in the previous email - currently there is no
signing and validation of the download. It assumes the URL you reference
for AU is trustworthy and if you download something from that you are
getting what you asked for and therefore it's ok to run that code. I tend
to feel this is ok but I am generally on the relaxed end in this area. My
logic is there's no guarantee the original install was not a hack/nasty, so
why complicate the later downloads by trying to guarantee they are from the
same source.

What do people think about this, what do we need to make things "safe" and
if we do add security and jar signing should this be mandatory or optional?
I, for one, would prefer not to complicate my build for an app that is only
going to be installed/used on an intranet and VPN and therefore is probably
safe enough without signing.

Right from the main title screen they would see that an upgrade is
> available and given the option to upgrade now.  This mimics some of my
> favorite apps.
>

Can you elaborate on what you mean by the 'main title screen'? I'm guessing
you are saying that a popup should be shown on startup if an upgrade is
available but please let me know if I misunderstood.


> Release notes.  I am used to how IOS does it, a user can see the list of
> changes before they upgrade an app.
>

Good to know. I left out release notes as I was not sure exactly how people
would want to use them. Adding it to the pre-install step makes sense.


> this should cover all releases between current and latest
>

A simple solution would be just to link to a webpage and the developer is
responsible for maintaining this page. This would allow you to completely
style and manage this page as you want (e.g. something like
http://www.jetbrains.com/idea/whatsnew/index.html).

Trying to record and derive individual release details and merge them would
complicate the build process quite a lot I imagine and in some cases may
not make sense. I'd be interested to hear more on how people would use this
feature and what would work best for them.

I'd also be interested to know about Terms & Conditions pages - do people
need this, should it be shown before the upgrade (seems logical), and does
there need to be some record of the T&C acceptance legally, or is it
generally enough that they can;t get past that page without clicking
'accept'?

Extra credit: option to upgrade to an intermediary version.... i do not
> know how popular this would be, I know for some apps in the past I had to
> hunt down version in between currently-installed and latest.
>

Good use case to know about but I'm thinking this would not be part of the
default work-flow. It adds complexity to a lot of levels that I'm not sure
would be totally worth it. We definitely should make the underlying
framework customisable enough however that this could be implemented by the
developer. Happy to hear more thoughts on this one, and if the base
framework allows for it (which it will) then it could always be added in a
later version.


>   2. what ways would you likely want to customise this experience for more
>   specialised scenarios?
>
> When a user purchases a license for my app they will be allowed to
> download future updates up to a certain point  So if they have a very old
> license then they can't upgrade to latest.  Do not know if this is
> something you want to build into the framework, but it at least must be
> implementable.  If the user chooses to buy a new license to upgrade,
> would be cool if they just enter the new license and then they get to
> upgrade.  So I guess I'm saying license based upgrade would be cool or at
> least not blocked by the framework.
>

This is a good point. I definitely think the base framework should/will
allow for this to be implemented on top of what comes by default. Thinking
about it there is really an enhanced process that is common for most
paid-for applications as opposed to your standard freebie or back-office
application. Not sure this should be part of the core library, but perhaps
- Igor and Richard might have some ideas on this as we get further along. A
way for developers to easily licence/charge for their apps may well
encourage more developers (as seen by the iOS app store).

Perhaps though, this payment based AU system would be a nice open source
project that extends the base functionality. It could then make use of
third party licence, ssl, hashing and payment-gateway libraries to
implement some powerful stuff on this. For example, I don't see any
technical reason why someone keen couldn't build their own 'pay now' screen
as part of the upgrade wizard and hook this into a payment provider (such
as PayPal). That would be pretty sweet.


> On a separate topic... how did you implement your details popup?  I
> notice how it displayed outside of your main window and how it would remain
> within my desktop's borders.
>

I just borrowed the old lightbox concept from web land (see
http://gettopup.com/)

Basically, instead of adding the root of the scene directly to the Scene, I
create a StackPane to wrapper it. When the "popup" is shown it is added to
the top layer of this stackpane and so covers everything underneath. I use
a semi-transparent pane to contain the popup and it is this
semi-transparent pane that greys out the underlying scene and blocks mouse
input, etc. All pretty basic stuff.

I've used these a lot instead of dialogs, they have the advantage that they
can be animated in/out and can be any shape, etc. They have the
disadvantage that they can't really be application modal if you have
multiple top-level windows (i.e. scenes) and I don't know how well they
would block keyboard input such as shortcut keys.

I've thought about open sourcing the code for this (it's very simple) but
never gotten round to it. If you are interested let me know.



> *From:* Daniel Zwolenski <zonski at gmail.com>
> *To:* openjfx-dev at openjdk.java.net
> *Sent:* Saturday, July 21, 2012 8:10 PM
>
> *Subject:* Re: Auto-update of native application bundles
>
> *Apologies for the previous, half-written email - I have a new keyboard and
> bumped some magic button. Here is the full version:*
>
>
> For those that are interested, I have put together an alternate version of
> the Auto Updating proposal.
>
> The installer can be found at:
>
>   - http://zenjava.com/demo/update/ensemble/EnsembleWithZenUpdate-1.0.exe
>
>
> To make things nicer, this time I have used the "JavaFX Ensemble" app as
> the base application. I've added a 'version bar' to the bottom of this and
> you can click the check for updates link on this. The newer version that
> gets downloaded has a red (ugly) bar, just to highlight the difference in
> version.
>
> This implementation differs from the last in two main ways:
>
>   1. It works with the existing native installers that Igor has created
>   2. It assumes compile-time deltas will be used instead of runtime deltas
>
>
>
> Source code for everything can be found at:
>
>   - http://code.google.com/p/zen-update/source/browse/trunk/
>
>
> *User Experience*
>
>
> As before, the user experience shown is not the one we want as the final
> version. I have improved on the last experience, but I would really like to
> get a conversation going on what people see as the ideal experience.
>
> I believe the framework should provide a 'default' experience that
> developers can very easily plug in to their application (e.g. there should
> be something like a CheckForUpdatesAction you can hook into your menus or
> button). This should cover the 80% use case. The framework should then
> allow the developer to hook into it and fully customise the experience
> however they like.
>
> So my two main questions are:
>
>   1. what would you consider the perfect auto-updating user experience?
>   2. what ways would you likely want to customise this experience for more
>
>   specialised scenarios?
>
> Things to consider:
>
>   - How should the user be notified of new updates and then start the
>
>   check for updates process (e.g. on start up, from a menu, from a status
>   bar, from a status bar tooltip bubble)?
>   - Would you like to see the auto-update be done in a dialog or some
>
>   other mechanism (such as a panel you then embed somewhere)?
>   - What information would you like to see about a current release and
>
>   available upgrade? Visually do you have any thoughts on how this should
> be
>   displayed?
>   - What pages/steps should the auto-update wizard take the user through,
>
>   what should be on each step, and how should this look visually (screen
>   mockups would be awesome, but general comments are good too)?
>
> And of course anything else you want to comment on would be welcome.
>
> *Deltas*
>
>
> Currently only a 'full' upgrade is used, there are no 'deltas' - with one
> variation on this theme: if the JRE version has not changed then the JRE
> does not need to be included in the upgrade. So your first download (the
> 'exe') will include all the JARs and the JRE (36MB) but the subsequent
> upgrade is a zip containing just the new JARs (8MB for Ensemble - your app
> may be more or less).
>
> Igor's thinking (please correct me if I misquote you Igor!) is that this
> approach will be the recommended norm and that deltas will be much less
> frequently used. If your app is small or released infrequently then this is
> probably acceptable. I'd be very interested to hear people's opinion on
> this one - once we head down one road (and we are already heading down it)
> then it will get very hard to change direction.
>
> *Implementation notes *
> *
> *
>
> To add AU to the Ensemble app, I cheated and just used Ensemble as a third
> party jar, instantiated the UI and added it to my own Scene, with the Auto
> Updating status bar added to the bottom. Ideally this should be better
> integrated in the long run.
>
> You can see the code for this here:
>
>   -
>
> http://code.google.com/p/zen-update/source/browse/trunk/ensemble-with-zen-update/
>
>   -
>
> http://code.google.com/p/zen-update/source/browse/trunk/ensemble-with-zen-update/src/com/zenjava/update/EnsembleWithZenUpdate.java
>
> The actual AU framework code can be found here:
>
>   - http://code.google.com/p/zen-update/source/browse/trunk/zen-update/
>
>   -
>
> http://code.google.com/p/zen-update/source/browse/trunk/zen-update-patcher/
>
> This time I have used ANT as the build tool instead of Maven, since most
> people seem to be using this, although I have used normal/plain ANT
> scripts, not the weird Netbeans specific ones that Netbeans seems to
> create.
>
> Additional comments:
>
>   - This is still a windows only implementation (but technically should be
>
>   easy to port to the others since Igor's done the hard work with his
>   packagers - if any Mac or Linux developers want to volunteer I would be
>   more than happy to talk you through what's needed - probably not a lot of
>   work involved just some simple Objective C or C++ coding).
>
>   - In general, the implementation could be used in your own app if you
>
>   really wanted to (though I would highly recommend fixing up the User
>   Experience) - with one major limitation: I currently hard-code the name
> of
>   the 'exe' for the Ensemble launcher in the C++ code. This needs to be
>   dynamic and I have been talking to Igor about ways that I could work out
>   what this 'exe' is called from within my C++ code.
>
>   - Since Igor's native installer does not yet install to 'program files',
>
>   I've not bothered dealing  nicely with Windows' UAC in this one. There is
>   no code for it in this repository but it could be easily added.
>
>   - As before the code is intended as a rapid, throw-away POC only, the
>
>   final version will likely look considerably different at a code level
>   (e.g. better structured, more robust, more complete). Don't get too hung
> up
>   on the structure of the code at this stage, just focus on the general
>   high-level approach of the solution.
>
> Cheers,
> Dan
>
>
>
> >
> > On Wed, Jun 20, 2012 at 11:59 AM, Daniel Zwolenski <
> zonski at googlemail.com>wrote:
> >
> >> I've uploaded a new MSI to: http://zenjava.com/demo/update/testapp.msi
> >>
> >> This is version 3 and it is blue. If you install and check for updates,
> >> you will see that version 4 is available for download. If you upgrade
> you
> >> will see that it's red. Note the upgrade is now quite quick as they both
> >> use the same Java version.
> >>
> >> This is now all in JavaFX instead of Swing. It is a nicer UI and user
> >> experience but it is still very raw and would need a lot of work. It
> should
> >> give you more of an idea though, and, of course, if anyone wants to
> look at
> >> making it nicer please let me know.
> >>
> >> *Note: If you have the older version you can auto update to this newer
>
> >> version but because the whole point is to demonstrate the new
> auto-update
> >> code, you would be better served by uninstalling, downloading the new
> one,
> >> installing and then upgrading. *
>
> >>
> >> I'm hoping (perhaps optimistically) for feedback on the two proposed
> >> approaches from the community regarding "deltas assembled at compile
> time"
> >> vs "deltas determined at runtime" (where a 'delta' is the same thing as
> a
> >> 'patch'). The floor is open, but if there isn't any feedback in the next
> >> few days I will focus on just one approach - in the absence of any other
> >> input it will likely be Igor's "deltas assembled at compile time"
> approach.
> >> Speak now or forever hold your peace (or at least apologise later when
> you
> >> make me change to another way).
> >>
> >> Just to re-cap, here are the main pros/cons of each solution:
> >>
> >>    - with *compile-time deltas* we can assemble all the changes into
>
> >>    one, easy to deploy file (per delta/OS).
> >>    - with *runtime deltas* the app has to be uploaded to the server
>
> >>    unzipped (i.e. as it would appear on your local system) so the apps
> can
> >>    work out diffs on the fly
> >>
> >>    - with *compile-time deltas* you have to create and maintain all
>
> >>    deltas you want to be available (e.g. v1-to-v2, v2-to-v3 and
> v1-to-v3). If
> >>    you don't provide a delta, the system reverts to 'full' update (i.e.
> >>    downloads everything). Currently this could mean a 50MB download
> (instead
> >>    of say a 4MB delta), but Igor has indicated this might be reduced
> >>    considerably. Deltas also require some extra steps at compile time to
> >>    define/detect what has changed.
> >>    - with *runtime deltas* you only maintain/upload the latest code -
>
> >>    the apps all work out the delta's on the fly and only download what
> they
> >>    need (e.g. the 4MB of jars that have changed). You do not need to
> create or
> >>    maintain any deltas. You do need to maintain an 'app-profile' but
> this
> >>    could be mostly generated (easily from Maven, I imagine ant could
> automate
> >>    it if you used versioned jars).
> >>
> >>    - the single file of the *compile-time deltas* make life
> >>    easier/quicker for signing.
> >>    - with *runtime deltas* the tool has to sign every jar individually
>
> >>    (although tooling would hide this - and signing requirements are not
> yet
> >>    fully explored)
> >>
> >> There are likely other smaller benefits as well (Igor, did I miss any
> big
> >> ones?) but these major ones are what I would be weighing up.
> >>
> >> There is another possible usage of *runtime deltas* that I've not
>
> >> brought up before because I still can't decide if it is a positive or a
> >> negative. With this approach the JARs don't have to come from the same
> URL
> >> source, so for example you could download the third party JARs (like
> >> hibernate, spring, etc) directly from centralised servers - and the
> Maven
> >> repos provide exactly this already. This would be useful for corporate,
> >> intranet style apps where they could maintain their own Maven repo, but
> for
> >> public apps it might be a little risky? It also opens up lots of
> questions
> >> regarding JAR signing, but I personally feel signing is not as
> necessary in
> >> all cases as might be the current thinking (but that's probably another
> >> debate).
> >>
> >> Any and all feedback on all the above will be useful for determining a
> >> solution.
> >>
> >
> >
>
>
>