RFC: Netx - implement -Xclearcache command line option
Dr Andrew John Hughes
ahughes at redhat.com
Wed Oct 6 11:40:22 PDT 2010
On 12:04 Wed 06 Oct , Omair Majid wrote:
> On 10/05/2010 07:19 PM, Dr Andrew John Hughes wrote:
> > On 14:45 Tue 05 Oct , Omair Majid wrote:
> >> On 10/04/2010 03:17 PM, Omair Majid wrote:
> >>> On 09/30/2010 05:27 PM, Deepak Bhole wrote:
> >>>> * Omair Majid<omajid at redhat.com> [2010-09-30 16:50]:
> >>>>> On 09/30/2010 04:32 PM, Deepak Bhole wrote:
> >>>>>> * Omair Majid<omajid at redhat.com> [2010-09-30 16:10]:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I posted this patch way back in 2009, but it looks like it got
> >>>>>>> lost/ignored.
> >>>>>>>
> >>>>>
> >>>>> Ok, turns out it was my fault: just noticed
> >>>>> http://mail.openjdk.java.net/pipermail/distro-pkg-dev/2009-August/006666.html.
> >>>>>
> >>>>> Somehow it slipped off my radar.
> >>>>>
> >>>>
> >>>> Doh. Glad it was re-visited. I doubt the issue will be encountered, but
> >>>> we might as well fix it instead of waiting for a bug report.
> >>>>
> >>>>>>> On 07/30/2009 05:18 PM, Omair Majid wrote:
> >>>>>>>> The attached patch adds the option -Xclearcache to Netx.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Which (and I neglected to explain this last time) makes javaws clean
> >>>>>>> out its cache by deleting the contents of ~/.netx/cache/
> >>>>>>>
> >>>>>>
> >>>>>> Hmm, what happens if I have another netx app running and javaws is
> >>>>>> called
> >>>>>> with -Xclearcache? AFAIK the code has no provisioning to detect if a
> >>>>>> cache file has been deleted, and to re-download it... does it?
> >>>>>>
> >>>>>
> >>>>> As far as I can tell, very bad things (not eating babies kind of
> >>>>> bad, but still...). Depending on the exact timing either the VM will
> >>>>> crash or a java Exception might be thrown. If the JVM has opened the
> >>>>> (missing) jar earlier and and wants to open it again to load
> >>>>> additional classes the VM will most probably crash. Otherwise javaws
> >>>>> simply wont be able to find the needed jar and will fail to start.
> >>>>>
> >>>>> I suppose I could make some sort of global application lock to stop
> >>>>> any other javaws instances from starting while -Xclearcache is
> >>>>> running and to make sure -Xclearcache does not start if any other
> >>>>> javaws applications are running.
> >>>>>
> >>>>
> >>>> It would be hard to set up a locking system that can handle odd cases
> >>>> like crashed application, multiple instances of same app., etc.
> >>>>
> >>>
> >>> Yeah, you are right. I thought about it for a bit, and I dont see a
> >>> simple way to make it work using locks.
> >>>
> >>>> Easiest way I think is to execute jps -l and see if there is more than
> >>>> one instance of net.sourceforge.jnlp.runtime.Boot. If there is, refuse
> >>>> to run with -Xclearcache. I am not as worried about the 'don't start
> >>>> while -Xclearcache is running' case, as there is a far smaller window
> >>>> where things can go wrong. Plus if they do, it will be right away (as
> >>>> opposed to an already started app that has modified data, crashing).
> >>>>
> >>>
> >>> That's a great idea! Unfortunately I dont think we can rely on jps. jps
> >>> is part of the JDK as opposed to the JRE - we can not be sure that users
> >>> have it on their machine. I tried looking into the sources for jps to
> >>> find out how it works. The JVM creates /tmp/hsperfdata_$USER/$PID files
> >>> which contain the information that jps can parse and display. The files
> >>> are in a binary format so parsing it is not trivial (and the file format
> >>> is not guaranteed to be final). I am going to dig around a bit to see if
> >>> there is any other simple solution to this
> >>>
> >>
> >> I am attaching the updated patch. This one uses an alternate
> >> implementation based on file locks (exclusive and shared). A normal
> >> javaws process acquires a shared lock on /tmp/$USER/netx/instance during
> >> startup (when an instance of Launcher is created), and releases the lock
> >> when the JVM shutsdown. Multiple javaws processes can share the lock and
> >> run simultaneously.
> >>
> >> The javaws process that is trying to clear the cache checks for the file
> >> and tries to acquire an exclusive lock. If it succeeds, it goes ahead
> >> and clears the cache; otherwise it prints out an error to the user and
> >> exits.
> >>
> >> Thoughts?
> >>
> >
> > Sounds good to me. I was actually going to suggest this before Deepak weighed
> > in with the jps option. Not knowing what jps was, I didn't no any pros or cons
> > as to using it.
> >
>
> Updated patch attached. Also fixes another problem - the previous patch
> was creating lock files for applets too :/.
>
> > Could you not just reuse LOCKS_DIR for this rather than introducing something
> > new?
>
> Ah, good point. Switched to $LOCKS_DIR/netx_running.
>
> > Also your mail says 'instance' but the code says 'instances' :-)
>
> I cant come up with a name I am happy with... something that indicates
> the purpose of those files clearly. NETX_RUNNING_FILE and
> $/LOCKS/netx_running is what I am using now.
>
I like netx_running best. It's clear and obvious :-)
> >
> >> + FileChannel channel = fileLock.channel();
> >> + fileLock.release();
> >> + channel.close();
> >
> > could be just:
> >
> > fileLock.release();
> > fileLock.channel().close();
> >
>
> Nice catch. Fixed.
>
> > Why do we have a method called just 'R' in CacheUtil?
> >
>
> R is the translation function. It is supposed to localize strings. A
> number of classes in netx declare R, which is a shortcut to
> JNLPRuntime.getMessage().
>
Does it have to be called R and not 'translate' or something more readable?
> > Should we really be catching IOExceptions rather than allowing them to propogate?
> >
>
> Catching IOExceptions is just ignoring the errors. So the question is do
> we want to ignore the errors and continue on or should we stop if we hit
> any errors. Since the lock file is meant to stop a race condition
> (between deleting jars and using them) which is not likely to happen, I
> think an error in lock file creation should be ignored. If for some
> reason a lock file can not be created, Netx should not abort running the
> application. If you dont think this makes sense, please let me know and
> I will change it.
>
That sounds sensible. I'm just always wary of catch blocks. I've seen them
used all too often to mask exceptions instead of handling them (including in
example code used by lecturers!)
> Thanks for the feedback!
>
> Cheers,
> Omair
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/Launcher.java
> --- a/netx/net/sourceforge/jnlp/Launcher.java Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/Launcher.java Wed Oct 06 11:58:46 2010 -0400
> @@ -20,12 +20,17 @@
> import java.applet.Applet;
> import java.awt.Container;
> import java.io.File;
> +import java.io.FileInputStream;
> +import java.io.FileOutputStream;
> +import java.io.IOException;
> import java.lang.management.ManagementFactory;
> import java.lang.management.ThreadMXBean;
> import java.lang.reflect.Method;
> import java.net.InetAddress;
> import java.net.URL;
> import java.net.UnknownHostException;
> +import java.nio.channels.FileChannel;
> +import java.nio.channels.FileLock;
> import java.util.LinkedList;
> import java.util.List;
> import java.util.jar.JarFile;
> @@ -78,6 +83,9 @@
> /** If the application should call System.exit on fatal errors */
> private boolean exitOnFailure = true;
>
> + /** a lock which is held to indicate that an isntance of netx is running */
> + private FileLock fileLock;
> +
> /**
> * Create a launcher with the runtime's default update policy
> * and launch handler.
> @@ -127,6 +135,7 @@
>
> this.handler = handler;
> this.updatePolicy = policy;
> +
> }
>
> /**
> @@ -385,6 +394,11 @@
> if (!file.isApplication())
> throw launchError(new LaunchException(file, null, R("LSFatal"), R("LCClient"), R("LNotApplication"), R("LNotApplicationInfo")));
>
> + markJavawsRunning();
> + Runtime.getRuntime().addShutdownHook(new Thread() {
> + public void run() { markJavawsStopped(); }
> + });
> +
> try {
>
> try {
> @@ -686,6 +700,69 @@
> return null; // chose to continue, or no handler
> }
>
> + /**
> + * Indicate that netx is running by creating the {@link JNLPRuntime#INSTANCE_FILE} and
> + * acquiring a shared lock on it
> + */
> + private void markJavawsRunning() {
> + try {
> + String message = "This file is used to check if netx is running";
> +
> + File javawsRunningFile = new File(JNLPRuntime.NETX_RUNNING_FILE);
> + javawsRunningFile.getParentFile().mkdirs();
> + if (javawsRunningFile.createNewFile()) {
> + FileOutputStream fos = new FileOutputStream(javawsRunningFile);
> + try {
> + fos.write(message.getBytes());
> + } finally {
> + fos.close();
> + }
> + }
> +
> + if (!javawsRunningFile.isFile()) {
> + if (JNLPRuntime.isDebug()) {
> + System.err.println("Unable to create instance file");
> + }
> + fileLock = null;
> + return;
> + }
> +
> + FileInputStream is = new FileInputStream(javawsRunningFile);
> + FileChannel channel = is.getChannel();
> + fileLock = channel.tryLock(0, Long.MAX_VALUE, true);
> + if (fileLock != null && fileLock.isShared()) {
> + if (JNLPRuntime.isDebug()) {
> + System.out.println("Acquired shared lock on " +
> + JNLPRuntime.NETX_RUNNING_FILE + " to indicate javaws is running");
> + }
> + } else {
> + fileLock = null;
> + }
> + } catch (IOException e) {
> + e.printStackTrace();
> + }
> +
> + }
> +
> + /**
> + * Indicate that javaws is stopped by releasing the shared lock on
> + * {@link JNLPRuntime#INSTANCE_FILE}.
> + */
> + private void markJavawsStopped() {
> + if (fileLock == null) {
> + return;
> + }
> + try {
> + fileLock.release();
> + fileLock.channel().close();
> + fileLock = null;
> + if (JNLPRuntime.isDebug()) {
> + System.out.println("Release shared lock on " + JNLPRuntime.NETX_RUNNING_FILE);
> + }
> + } catch (IOException e) {
> + e.printStackTrace();
> + }
> + }
>
>
> /**
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/cache/CacheUtil.java
> --- a/netx/net/sourceforge/jnlp/cache/CacheUtil.java Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/cache/CacheUtil.java Wed Oct 06 11:58:46 2010 -0400
> @@ -19,6 +19,7 @@
>
> import java.io.*;
> import java.net.*;
> +import java.nio.channels.FileChannel;
> import java.util.*;
> import java.lang.reflect.*;
> import java.security.*;
> @@ -37,6 +38,10 @@
> */
> public class CacheUtil {
>
> + private static String R(String key) {
> + return JNLPRuntime.getMessage(key);
> + }
> +
> private static String R(String key, Object param) {
> return JNLPRuntime.getMessage(key, new Object[] {param});
> }
> @@ -129,6 +134,72 @@
> }
>
> /**
> + * Clears the cache by deleting all the Netx cache files
> + *
> + * Note: Because of how our caching system works, deleting jars of another javaws
> + * process is using them can be quite disasterous. Hence why Launcher creates lock files
> + * and we check for those by calling {@link #okToClearCache()}
> + */
> + public static void clearCache() {
> +
> + if (!okToClearCache()) {
> + System.err.println(R("CCannotClearCache"));
> + return;
> + }
> +
> + File cacheDir = new File(JNLPRuntime.getBaseDir() + File.separator + "cache");
> + if (!(cacheDir.isDirectory())) {
> + return;
> + }
> +
> + if (JNLPRuntime.isDebug()) {
> + System.err.println("Clearing cache directory: " + cacheDir);
> + }
> + try {
> + FileUtils.recursiveDelete(cacheDir, JNLPRuntime.getBaseDir());
> + } catch (IOException e) {
> + throw new RuntimeException(e);
> + }
> + }
> +
> + /**
> + * Returns a boolean indicating if it ok to clear the netx application cache at this point
> + * @return true if the cache can be cleared at this time without problems
> + */
> + private static boolean okToClearCache() {
> + File otherJavawsRunning = new File(JNLPRuntime.NETX_RUNNING_FILE);
> + try {
> + if (otherJavawsRunning.isFile()) {
> + FileOutputStream fis = new FileOutputStream(otherJavawsRunning);
> + try {
> + FileChannel channel = fis.getChannel();
> + if (channel.tryLock() == null) {
> + if (JNLPRuntime.isDebug()) {
> + System.out.println("Other instances of netx are running");
> + }
> + return false;
> + }
> +
> + if (JNLPRuntime.isDebug()) {
> + System.out.println("No other instances of netx are running");
> + }
> + return true;
> +
> + } finally {
> + fis.close();
> + }
> + } else {
> + if (JNLPRuntime.isDebug()) {
> + System.out.println("No instance file found");
> + }
> + return true;
> + }
> + } catch (IOException e) {
> + return false;
> + }
> + }
> +
> + /**
> * Returns whether there is a version of the URL contents in the
> * cache and it is up to date. This method may not return
> * immediately.
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/resources/Messages.properties
> --- a/netx/net/sourceforge/jnlp/resources/Messages.properties Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/resources/Messages.properties Wed Oct 06 11:58:46 2010 -0400
> @@ -139,6 +139,7 @@
> BOViewer = Shows the trusted certificate viewer.
> BOUmask = Sets the umask for files created by an application.
> BXnofork = Do not create another JVM.
> +BXclearcache= Clean the JNLP application cache.
> BOHelp = Print this message and exit.
>
> # Cache
> @@ -149,6 +150,8 @@
> CChooseCache=Choose a cache directory...
> CChooseCacheInfo=Netx needs a location for storing cache files.
> CChooseCacheDir=Cache directory
> +CConfirmClearCache=Are you sure you want to clear the cache? This may running javaws applications to misbehave.
> +CCannotClearCache=Can not clear cache at this time
>
> # Security
> SFileReadAccess=The application has requested read access to {0}. Do you want to allow this action?
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/runtime/Boot.java
> --- a/netx/net/sourceforge/jnlp/runtime/Boot.java Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/runtime/Boot.java Wed Oct 06 11:58:46 2010 -0400
> @@ -40,6 +40,7 @@
> import net.sourceforge.jnlp.ParseException;
> import net.sourceforge.jnlp.PropertyDesc;
> import net.sourceforge.jnlp.ResourcesDesc;
> +import net.sourceforge.jnlp.cache.CacheUtil;
> import net.sourceforge.jnlp.cache.UpdatePolicy;
> import net.sourceforge.jnlp.security.VariableX509TrustManager;
> import net.sourceforge.jnlp.security.viewer.CertificateViewer;
> @@ -114,6 +115,7 @@
> + " -strict "+R("BOStrict")+"\n"
> + " -umask=value "+R("BOUmask")+"\n"
> + " -Xnofork "+R("BXnofork")+"\n"
> + + " -Xclearcache "+R("BXclearcache")+"\n"
> + " -help "+R("BOHelp")+"\n";
>
> private static final String doubleArgs = "-basedir -jnlp -arg -param -property -update";
> @@ -202,6 +204,17 @@
> JNLPRuntime.setSecurityEnabled(null == getOption("-nosecurity"));
> JNLPRuntime.initialize(true);
>
> + /*
> + * FIXME
> + * This should have been done with the rest of the argument parsing
> + * code. But we need to know what the cache and base directories are,
> + * and baseDir is initialized here
> + */
> + if (null != getOption("-Xclearcache")) {
> + CacheUtil.clearCache();
> + return null;
> + }
> +
> try {
> new Launcher().launch(getFile());
> }
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/runtime/JNLPRuntime.java
> --- a/netx/net/sourceforge/jnlp/runtime/JNLPRuntime.java Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/runtime/JNLPRuntime.java Wed Oct 06 11:58:46 2010 -0400
> @@ -18,6 +18,7 @@
> package net.sourceforge.jnlp.runtime;
>
> import java.io.*;
> +import java.nio.channels.FileLock;
> import java.awt.*;
> import java.text.*;
> import java.util.*;
> @@ -132,6 +133,15 @@
> public static final String LOCKS_DIR = TMP_DIR + File.separator + USER + File.separator
> + "netx" + File.separator + "locks";
>
> + /**
> + * The /tmp/$USER/netx/locks/netx_running file is used to indicate if any
> + * instances of netx are running (this file may exist even if no instances
> + * are running). All netx instances acquire a shared lock on this file. If
> + * this file can be locked (using a {@link FileLock}) in exclusive mode, then
> + * other netx instances are not running
> + */
> + public static final String NETX_RUNNING_FILE = LOCKS_DIR + "netx_running";
> +
> /** the java.home directory */
> public static final String JAVA_HOME_DIR = System.getProperty("java.home");
>
> diff -r f41b0a7fbd70 netx/net/sourceforge/jnlp/util/FileUtils.java
> --- a/netx/net/sourceforge/jnlp/util/FileUtils.java Mon Oct 04 18:13:02 2010 -0400
> +++ b/netx/net/sourceforge/jnlp/util/FileUtils.java Wed Oct 06 11:58:46 2010 -0400
> @@ -17,6 +17,9 @@
> package net.sourceforge.jnlp.util;
>
> import java.io.File;
> +import java.io.IOException;
> +
> +import net.sourceforge.jnlp.runtime.JNLPRuntime;
>
> /**
> * This class contains a few file-related utility functions.
> @@ -121,4 +124,36 @@
> return prefix + OMITTED + suffix;
> }
>
> + /**
> + * Recursively delete everything under a directory. Works on either files or
> + * directories
> + *
> + * @param file the file object representing what to delete. Can be either a
> + * file or a directory.
> + * @param base the directory under which the file and its subdirectories must be located
> + * @throws IOException on an io exception or if trying to delete something
> + * outside the base
> + */
> + public static void recursiveDelete(File file, File base) throws IOException {
> + if (JNLPRuntime.isDebug()) {
> + System.err.println("Deleting: " + file);
> + }
> +
> + if (!(file.getCanonicalPath().startsWith(base.getCanonicalPath()))) {
> + throw new IOException("Trying to delete a file outside Netx's basedir: "
> + + file.getCanonicalPath());
> + }
> +
> + if (file.isDirectory()) {
> + File[] children = file.listFiles();
> + for (int i = 0; i < children.length; i++) {
> + recursiveDelete(children[i], base);
> + }
> + }
> + if (!file.delete()) {
> + throw new IOException("Unable to delete file: " + file);
> + }
> +
> + }
> +
> }
--
Andrew :)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint = F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
More information about the distro-pkg-dev
mailing list