From John.Coomes at sun.com Tue Aug 11 11:33:33 2009 From: John.Coomes at sun.com (John Coomes) Date: Tue, 11 Aug 2009 11:33:33 -0700 Subject: hg push getting worse? In-Reply-To: <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> Message-ID: <19073.47485.333748.991898@sun.com> Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: > > ... > > From my experience with the OpenJDK servers, I think it may be jcheck. > > ??Unfortunately, jcheck itself is not Free Software, so it is > > impossible for me to say for definite. ??Instead, we seem to be > > discovering what checks it performs only when failures occur. > > > > I do know it does the following: > > > > * Checks the use of whitespace in changesets, rejecting the use of > > tabs and trailing whitespace. > > * Checks the format of the commit message. ??It must even follow the > > bugid/summary/reviewer format documented in the developer guide or > > simply be 'Merge'. > > * Checks that the bugid used is not used by any other changeset. > > > > I've run into all of these with different changesets. ??I can > > definitely see how the latter could take a while on something like the > > jdk repository (which dwarfs the others in size). I think jcheck is relatively light-weight compared to the zfs snapshot and auto-push that are also done. But it would be nice to know for sure. > > jcheck appears to perform its check on the changeset once committed to > > the remote repository and then performs a rollback if it fails, > > although you don't see any of this feedback in realtime on the client. > > ??I presume this also means it has to get Mercurial to generate the > > changeset (and others for the bugid check) which would take time. That's the standard way that mercurial hooks work, and the reason we have the extra *-gate forests. In more recent versions of mercurial, the pending changesets aren't visible until the hooks have completed successfully, so the *-gate forests become unnecessary. We haven't been able to update because the server-side of the forest extension doesn't work w/recent mercurial releases. > > It would be much better if we could perform these sanity checks > > locally, though we'd still need some way of checking this had been > > done on the server side (or who knows, we could trust the developers > > to have done it...). > > I should also note that duplicate bugids can appear completely > legitimately in certain merge cases. This is the current issue with > updating OpenJDK6's HotSpot.. Fixes for bugs occur both in the > original (rebased) OpenJDK6 HotSpot and the copy of HotSpot being > merged from the hs14 repository, resulting in duplicate bugids which > jcheck rejects. I presume this is to protect against the case that > someone mistypes the bugid as another legitimate bugid, but it is > certainly an expensive check for what seems a fairly unlikely > occurrence. The goal is to prevent people from checking in fixes piece-meal, scattering a bug fix over 2, 3 or more changesets. Having a bug fix isolated to a single changeset makes backporting it to a different release much, much easier. We lived with the practice of partial fixes for quite some time, and I'd rather not go back. -John From John.Coomes at sun.com Tue Aug 11 12:34:01 2009 From: John.Coomes at sun.com (John Coomes) Date: Tue, 11 Aug 2009 12:34:01 -0700 Subject: hg push getting worse? In-Reply-To: <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> Message-ID: <19073.51113.585193.108720@sun.com> Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: > 2009/8/11 John Coomes : > > I think jcheck is relatively light-weight compared to the zfs snapshot > > and auto-push that are also done. ??But it would be nice to know for > > sure. > > True. It's hard to tell from the client side as hg appears to hang > 'searching for changes' and then suddenly a burst of information > appears, either reporting success or failure. I presume this is part > of the design of hg unfortunately, but it would be helpful if things > were more verbose. I presume hooks were never meant to utilise enough > time that this would become an issue. > > I can see how zfs snapshotting would take longer over time. Although > the snapshots are, I presume, relative to the last so that the size > remains fairly sane, the sheer number of them is likely to cause a > slowdown. Is there a significant benefit to creating these? I don't know the openjdk infrastructure, so am just guessing, but I suspect there are only a finite number of snapshots maintained. The benefit is being able to recover :-). AFAIK, we've only used it once or twice, but having the snapshots made it possible. > ... And are > they also created in Maurizio's FX project? I have no idea. > >> > jcheck appears to perform its check on the changeset once committed to > >> > the remote repository and then performs a rollback if it fails, > >> > although you don't see any of this feedback in realtime on the client. > >> > ????I presume this also means it has to get Mercurial to generate the > >> > changeset (and others for the bugid check) which would take time. > > > > That's the standard way that mercurial hooks work, and the reason we > > have the extra *-gate forests. ??In more recent versions of mercurial, > > the pending changesets aren't visible until the hooks have completed > > successfully, so the *-gate forests become unnecessary. ??We haven't > > been able to update because the server-side of the forest extension > > doesn't work w/recent mercurial releases. > > > > Is there any further news on whether the forest extension will become > a standard part of Mercurial? When I went searching for a client-side > version to support newer versions, I had to resort to using a snapshot > of their repository for forest. I doubt forest will ever become part of mercurial. Mercurial now has an experimental sub-repo feature for dealing with nested repositories. It doesn't have the flexibility of the forest extension, so wouldn't work for openjdk, at least as it stands now. > ... > All changes have to be approved first, so could not this issue be > flagged at that point? The history could also be scanned for > duplicates at that point if someone really wanted to be sure. The > case is even worse with the whitespace checks. Given jcheck clearly > can find the issues, can it not also fix them? Or could some of these > checks be applied on the client-side? Worst case, could the whole > repo. not just be sanitised on a regular basis? As is, you push a > commit, wait several minutes and then get a result from jcheck saying > there is an extra space at the end of one line. So, you have to > rollback the commit, remove the space, reapply the commit with the > same comments and then push again. In all, a lot of time is spent for > very little gain. I think the problem is access to jcheck. I'm not sure why it isn't available externally. We run jcheck on our local repos before pushing, and it's reasonably convenient. Or maybe I should say not too inconvenient :-). -John From gnu_andrew at member.fsf.org Tue Aug 11 11:58:20 2009 From: gnu_andrew at member.fsf.org (Andrew John Hughes) Date: Tue, 11 Aug 2009 19:58:20 +0100 Subject: hg push getting worse? In-Reply-To: <19073.47485.333748.991898@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> Message-ID: <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> 2009/8/11 John Coomes : > Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: >> > ... >> > From my experience with the OpenJDK servers, I think it may be jcheck. >> > ??Unfortunately, jcheck itself is not Free Software, so it is >> > impossible for me to say for definite. ??Instead, we seem to be >> > discovering what checks it performs only when failures occur. >> > >> > I do know it does the following: >> > >> > * Checks the use of whitespace in changesets, rejecting the use of >> > tabs and trailing whitespace. >> > * Checks the format of the commit message. ??It must even follow the >> > bugid/summary/reviewer format documented in the developer guide or >> > simply be 'Merge'. >> > * Checks that the bugid used is not used by any other changeset. >> > >> > I've run into all of these with different changesets. ??I can >> > definitely see how the latter could take a while on something like the >> > jdk repository (which dwarfs the others in size). > > I think jcheck is relatively light-weight compared to the zfs snapshot > and auto-push that are also done. ?But it would be nice to know for > sure. > True. It's hard to tell from the client side as hg appears to hang 'searching for changes' and then suddenly a burst of information appears, either reporting success or failure. I presume this is part of the design of hg unfortunately, but it would be helpful if things were more verbose. I presume hooks were never meant to utilise enough time that this would become an issue. I can see how zfs snapshotting would take longer over time. Although the snapshots are, I presume, relative to the last so that the size remains fairly sane, the sheer number of them is likely to cause a slowdown. Is there a significant benefit to creating these? And are they also created in Maurizio's FX project? >> > jcheck appears to perform its check on the changeset once committed to >> > the remote repository and then performs a rollback if it fails, >> > although you don't see any of this feedback in realtime on the client. >> > ??I presume this also means it has to get Mercurial to generate the >> > changeset (and others for the bugid check) which would take time. > > That's the standard way that mercurial hooks work, and the reason we > have the extra *-gate forests. ?In more recent versions of mercurial, > the pending changesets aren't visible until the hooks have completed > successfully, so the *-gate forests become unnecessary. ?We haven't > been able to update because the server-side of the forest extension > doesn't work w/recent mercurial releases. > Is there any further news on whether the forest extension will become a standard part of Mercurial? When I went searching for a client-side version to support newer versions, I had to resort to using a snapshot of their repository for forest. >> > It would be much better if we could perform these sanity checks >> > locally, though we'd still need some way of checking this had been >> > done on the server side (or who knows, we could trust the developers >> > to have done it...). >> >> I should also note that duplicate bugids can appear completely >> legitimately in certain merge cases. ?This is the current issue with >> updating OpenJDK6's HotSpot.. ?Fixes for bugs occur both in the >> original (rebased) OpenJDK6 HotSpot and the copy of HotSpot being >> merged from the hs14 repository, resulting in duplicate bugids which >> jcheck rejects. ?I presume this is to protect against the case that >> someone mistypes the bugid as another legitimate bugid, but it is >> certainly an expensive check for what seems a fairly unlikely >> occurrence. > > The goal is to prevent people from checking in fixes piece-meal, > scattering a bug fix over 2, 3 or more changesets. ?Having a bug fix > isolated to a single changeset makes backporting it to a different > release much, much easier. ?We lived with the practice of partial > fixes for quite some time, and I'd rather not go back. > I agree. I'm just not sure that the best way of enforcing this is a technical one. The OpenJDK6 example is fairly convoluted, but I imagine our IcedTea forest would be prone to this too, were it to be jchecked. We often commit a fix there early, and then also receive the same fix in a later pull from the JDK7 forest. All changes have to be approved first, so could not this issue be flagged at that point? The history could also be scanned for duplicates at that point if someone really wanted to be sure. The case is even worse with the whitespace checks. Given jcheck clearly can find the issues, can it not also fix them? Or could some of these checks be applied on the client-side? Worst case, could the whole repo. not just be sanitised on a regular basis? As is, you push a commit, wait several minutes and then get a result from jcheck saying there is an extra space at the end of one line. So, you have to rollback the commit, remove the space, reapply the commit with the same comments and then push again. In all, a lot of time is spent for very little gain. > -John > > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 From Jonathan.Gibbons at Sun.COM Tue Aug 11 13:27:57 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 11 Aug 2009 13:27:57 -0700 Subject: hg push getting worse? In-Reply-To: <19073.51113.585193.108720@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> Message-ID: <4A81D44D.2040305@sun.com> John Coomes wrote: > Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: > >> 2009/8/11 John Coomes : >> >> >> Is there any further news on whether the forest extension will become >> a standard part of Mercurial? When I went searching for a client-side >> version to support newer versions, I had to resort to using a snapshot >> of their repository for forest. >> > > I doubt forest will ever become part of mercurial. Mercurial now has > an experimental sub-repo feature for dealing with nested repositories. > It doesn't have the flexibility of the forest extension, so wouldn't > work for openjdk, at least as it stands now. > > What are the issues with the Mercurial sub-repos. It seems close to what we want. What is the lack of flexibility that you have observed that concerns you? -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/web-discuss/attachments/20090811/860fa5a9/attachment.html From gnu_andrew at member.fsf.org Tue Aug 11 13:34:06 2009 From: gnu_andrew at member.fsf.org (Andrew John Hughes) Date: Tue, 11 Aug 2009 21:34:06 +0100 Subject: hg push getting worse? In-Reply-To: <19073.51113.585193.108720@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> Message-ID: <17c6771e0908111334y26e701fagabb826335afbc825@mail.gmail.com> 2009/8/11 John Coomes : > Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: >> 2009/8/11 John Coomes : >> > I think jcheck is relatively light-weight compared to the zfs snapshot >> > and auto-push that are also done. ??But it would be nice to know for >> > sure. >> >> True. ?It's hard to tell from the client side as hg appears to hang >> 'searching for changes' and then suddenly a burst of information >> appears, either reporting success or failure. ?I presume this is part >> of the design of hg unfortunately, but it would be helpful if things >> were more verbose. ?I presume hooks were never meant to utilise enough >> time that this would become an issue. >> >> I can see how zfs snapshotting would take longer over time. ?Although >> the snapshots are, I presume, relative to the last so that the size >> remains fairly sane, the sheer number of them is likely to cause a >> slowdown. Is there a significant benefit to creating these? > > I don't know the openjdk infrastructure, so am just guessing, Same here. but I > suspect there are only a finite number of snapshots maintained. ?The > benefit is being able to recover :-). ?AFAIK, we've only used it once > or twice, but having the snapshots made it possible. > Ah, sounds worth it then ;) >> ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?And are >> they also created in Maurizio's FX project? > > I have no idea. > >> >> > jcheck appears to perform its check on the changeset once committed to >> >> > the remote repository and then performs a rollback if it fails, >> >> > although you don't see any of this feedback in realtime on the client. >> >> > ????I presume this also means it has to get Mercurial to generate the >> >> > changeset (and others for the bugid check) which would take time. >> > >> > That's the standard way that mercurial hooks work, and the reason we >> > have the extra *-gate forests. ??In more recent versions of mercurial, >> > the pending changesets aren't visible until the hooks have completed >> > successfully, so the *-gate forests become unnecessary. ??We haven't >> > been able to update because the server-side of the forest extension >> > doesn't work w/recent mercurial releases. >> > >> >> Is there any further news on whether the forest extension will become >> a standard part of Mercurial? When I went searching for a client-side >> version to support newer versions, I had to resort to using a snapshot >> of their repository for forest. > > I doubt forest will ever become part of mercurial. ?Mercurial now has > an experimental sub-repo feature for dealing with nested repositories. > It doesn't have the flexibility of the forest extension, so wouldn't > work for openjdk, at least as it stands now. > >> ... >> All changes have to be approved first, so could not this issue be >> flagged at that point? ?The history could also be scanned for >> duplicates at that point if someone really wanted to be sure. ?The >> case is even worse with the whitespace checks. ?Given jcheck clearly >> can find the issues, can it not also fix them? ?Or could some of these >> checks be applied on the client-side? Worst case, could the whole >> repo. not just be sanitised on a regular basis? ?As is, you push a >> commit, wait several minutes and then get a result from jcheck saying >> there is an extra space at the end of one line. ?So, you have to >> rollback the commit, remove the space, reapply the commit with the >> same comments and then push again. ?In all, a lot of time is spent for >> very little gain. > > I think the problem is access to jcheck. ?I'm not sure why it isn't > available externally. ?We run jcheck on our local repos before > pushing, and it's reasonably convenient. ?Or maybe I should say not > too inconvenient :-). > Exactly. Now I know about them (through failed changesets), I could probably come up with my own checks for the whitespace issues I mentioned, but I have no idea what other checks may be being performed by jcheck. It's a black box, and our only evidence of its behaviour is by testing with our changesets as input. > -John > > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 From John.Coomes at sun.com Tue Aug 11 14:10:51 2009 From: John.Coomes at sun.com (John Coomes) Date: Tue, 11 Aug 2009 14:10:51 -0700 Subject: hg push getting worse? In-Reply-To: <4A81D44D.2040305@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> <4A81D44D.2040305@sun.com> Message-ID: <19073.56923.879858.229096@sun.com> Jonathan Gibbons (Jonathan.Gibbons at Sun.COM) wrote: > John Coomes wrote: > > Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: > > > >> 2009/8/11 John Coomes : > >> > >> Is there any further news on whether the forest extension will become > >> a standard part of Mercurial? When I went searching for a client-side > >> version to support newer versions, I had to resort to using a snapshot > >> of their repository for forest. > > > > I doubt forest will ever become part of mercurial. Mercurial now has > > an experimental sub-repo feature for dealing with nested repositories. > > It doesn't have the flexibility of the forest extension, so wouldn't > > work for openjdk, at least as it stands now. > > > What are the issues with the Mercurial sub-repos. It seems close to what > we want. > What is the lack of flexibility that you have observed that concerns you? The main one is the inability to get a partial tree. But there's also the inability to avoid recursing into sub-repos when doing an update. There were a couple of requests for the latter on the mercurial-devel list, but I don't think anything has happened. And it seems to be just an implementation problem, but the location of the sub-repos is kept in a versioned file which can't be overridden from the command line. -John From Jonathan.Gibbons at Sun.COM Tue Aug 11 14:28:55 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Tue, 11 Aug 2009 14:28:55 -0700 Subject: hg push getting worse? In-Reply-To: <19073.56923.879858.229096@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> <4A81D44D.2040305@sun.com> <19073.56923.879858.229096@sun.com> Message-ID: <4A81E297.2060806@sun.com> John Coomes wrote: > Jonathan Gibbons (Jonathan.Gibbons at Sun.COM) wrote: > >> John Coomes wrote: >> >>> Andrew John Hughes (gnu_andrew at member.fsf.org) wrote: >>> >>> >>>> 2009/8/11 John Coomes : >>>> >>>> Is there any further news on whether the forest extension will become >>>> a standard part of Mercurial? When I went searching for a client-side >>>> version to support newer versions, I had to resort to using a snapshot >>>> of their repository for forest. >>>> >>> I doubt forest will ever become part of mercurial. Mercurial now has >>> an experimental sub-repo feature for dealing with nested repositories. >>> It doesn't have the flexibility of the forest extension, so wouldn't >>> work for openjdk, at least as it stands now. >>> >> What are the issues with the Mercurial sub-repos. It seems close to what >> we want. >> What is the lack of flexibility that you have observed that concerns you? >> > > The main one is the inability to get a partial tree. But there's also > the inability to avoid recursing into sub-repos when doing an update. > There were a couple of requests for the latter on the mercurial-devel > list, but I don't think anything has happened. And it seems to be > just an implementation problem, but the location of the sub-repos is > kept in a versioned file which can't be overridden from the command > line. > > -John > > Ouch. Thanks for the update. -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/web-discuss/attachments/20090811/ec781c3d/attachment.html From Tim.Bell at Sun.COM Tue Aug 11 15:04:07 2009 From: Tim.Bell at Sun.COM (Tim Bell) Date: Tue, 11 Aug 2009 15:04:07 -0700 Subject: hg push getting worse? In-Reply-To: <19073.51113.585193.108720@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> Message-ID: <4A81EAD7.1030809@sun.com> > I don't know the openjdk infrastructure, so am just guessing, but I > suspect there are only a finite number of snapshots maintained. The > benefit is being able to recover :-). AFAIK, we've only used it once > or twice, but having the snapshots made it possible. I was just on the hg server to shake loose a changeset in the jdk6-gate/langtools repo. The most obvious thing is that the push script is spending a lot of time running '/sbin/zfs list -Ho mountpoint,name' Running only that command by hand takes 3 mins 17 secs and produces 11485 lines of output. During that time, truss reports hundreds of these: brk(0x0CC6F000) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ZFS snapshots are supposed to be lightweight and inexpensive. We still have them going back to the beginning of time on hg.ojn. Perhaps we have accumulated enough to expose quadratic (or worse) performance problems. I will do some searching over on OpenSolaris. Tim From Tim.Bell at Sun.COM Thu Aug 13 13:33:23 2009 From: Tim.Bell at Sun.COM (Tim Bell) Date: Thu, 13 Aug 2009 13:33:23 -0700 Subject: hg push getting worse? In-Reply-To: <4A81EAD7.1030809@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> <4A81EAD7.1030809@sun.com> Message-ID: <4A847893.30007@sun.com> I wrote: > ZFS snapshots are supposed to be lightweight and inexpensive. We > still have them going back to the beginning of time on hg.ojn. > > Perhaps we have accumulated enough to expose quadratic (or worse) > performance problems. I will do some searching over on OpenSolaris. I spent ten or fifteen minutes searching on this issue. The short version is that there has been a significant amount of ZFS performance work done since "Solaris Express Developer Edition 9/07 snv_70a X86" which is the release hg.ojn is running. That was the best choice back when we set the server up, but now it is pretty old. Here are a few examples - all of these are marked FIXed in builds more recent than 70a. (FYI: OpenSolaris is currently up around b121): 6596190 ""zfs list" is slow due to version property" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6596190 6386929 "initial "zfs list" is slow" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6386929 6755389 "Initial run of any zfs command that iterates over datasets can be slow" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6755389 Here is a message thread related to 'zfs list' being slow: http://mail.opensolaris.org/pipermail/zfs-discuss/2008-July/019956.html hg.ojn has been extremely stable since we set it up - it currently has 652 days of uptime. All the same, it it probably time to start planning an upgrade. Tim From Jonathan.Gibbons at Sun.COM Thu Aug 27 11:16:09 2009 From: Jonathan.Gibbons at Sun.COM (Jonathan Gibbons) Date: Thu, 27 Aug 2009 11:16:09 -0700 Subject: hg push getting worse? In-Reply-To: <4A847893.30007@sun.com> References: <4A81AAB8.5060203@sun.com> <17c6771e0908111102h6a5b1d9ale21e5c744874f05c@mail.gmail.com> <17c6771e0908111109r75cdf242u4e45f348ec95d15a@mail.gmail.com> <19073.47485.333748.991898@sun.com> <17c6771e0908111158o4d2f4d6dm64e08f23fff5aa97@mail.gmail.com> <19073.51113.585193.108720@sun.com> <4A81EAD7.1030809@sun.com> <4A847893.30007@sun.com> Message-ID: <4A96CD69.9070607@sun.com> Any chance of that upgrade happening? What would it take to make it happen? -- Jon Tim Bell wrote: > I wrote: > > >> ZFS snapshots are supposed to be lightweight and inexpensive. We >> still have them going back to the beginning of time on hg.ojn. >> >> Perhaps we have accumulated enough to expose quadratic (or worse) >> performance problems. I will do some searching over on OpenSolaris. >> > > I spent ten or fifteen minutes searching on this issue. The short > version is that there has been a significant amount of ZFS > performance work done since > "Solaris Express Developer Edition 9/07 snv_70a X86" > which is the release hg.ojn is running. That was the best choice > back when we set the server up, but now it is pretty old. > > Here are a few examples - all of these are marked FIXed in builds > more recent than 70a. (FYI: OpenSolaris is currently up around b121): > > 6596190 ""zfs list" is slow due to version property" > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6596190 > > 6386929 "initial "zfs list" is slow" > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6386929 > > 6755389 "Initial run of any zfs command that iterates over datasets can be slow" > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6755389 > > Here is a message thread related to 'zfs list' being slow: > http://mail.opensolaris.org/pipermail/zfs-discuss/2008-July/019956.html > > > hg.ojn has been extremely stable since we set it up - it currently > has 652 days of uptime. All the same, it it probably time to start > planning an upgrade. > > Tim > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/web-discuss/attachments/20090827/b59c0b3a/attachment.html