RFR [15] 8237909: Remove zipped index files feature

[Jonathan Gibbons]

I concur with all the preceding discussion, about removing the 
zipped-files feature, and about the desirability of asynchronous loading 
if that is practical.

Separate RFE:  the UI should indicate if/while the results are incomplete.

There's a TODO in AbstractIndexWriter.java

472 if (!searchIndex.isEmpty()) { // TODO: write to disk straight

I'm pleased to see this line go:

318 tree.add(new RawHtml("<!--[if IE]>"));

We need a (separate?) plan of action for documenting this change and 
recommending that webservers delivering big API index files should be 
configured to use compression.

> I intend to let this RFR sit here for at least 2 weeks.
> The perceived severity of this change should not be underestimated.
I think it is equally important to get this pushed early in the release, 
so that it can
be tested with real-world docs, such as the EA docs for JDK 15.

-- Jon


On 01/28/2020 07:55 AM, Pavel Rappo wrote:
> Hello,
>
> Please review the change for https://bugs.openjdk.java.net/browse/JDK-8237909:
>
>     http://cr.openjdk.java.net/~prappo/8237909/webrev.00/
>
> This change removes the "zipped index files" feature, which was introduced as
> part of 8141492: Implement search feature in javadoc.
>
> The "zipped index files" feature consists of generating the zipped index files
> on the back end, and fetching & unzipping mechanics on the front end.
>
> When documenting source files, the standard doclet accumulates index which is
> later used by the JavaScript code serving the interactive search. The index
> is written in two formats, .js (JavaScript) and .json (JSON). The latter is
> then zipped.
>
> When a browser accesses the pages using "http://" urls, the .zip index files are
> transferred using XHR. Those files are then unzipped by the browser, using the
> JSZip library, and parsed as JSON. If the transfer of the .zip index files fails
> for whatever reason, the browser falls back on the alternative mechanism. This
> mechanism transfers the .js index files by referring to them from dynamically
> inserted <script src="... .js"> elements. Those files then are not additionally
> parsed, as they are already data hardcoded in JavaScript code.
>
> One of the reasons the .zip index files transfer may fail is using javadoc pages
> in the "standalone" mode. When a browser accesses "file://" urls, there's no
> HTTP server to send the XHR requests to. So the fallback mechanism kicks in and
> the browser loads the .js index files instead.
>
> Analysis
> ========
>
>  From what I understand, the original intent was to reduce the transfer size of
> the index files. The observations made during the recent upgrade of JSZip
> (JDK-8236700) suggest that the feature is not working as intended. It is not
> clear if it ever did. The proposal is to remove it for the following reasons:
>
> 1. The feature in its current state does more harm than good (see JDK-8236922)
> 2. Fixing, debugging, testing, and evolving require expertise beyond that of
>     typical for the javadoc area
> 3. The feature significantly complicates the front end and less so the back end
>     code
> 4. The feature relies on the 3rd party libraries, which require tracking &
>     periodical upgrades
> 5. The difference in size between the .zip and .js files is not that big (see below)
> 6. The index files are transferred once and then used from cache
> 7. Modern HTTP servers provide compression. This makes the net result
>     virtually the same, compare:
>
>                        | (current) js + zip, MB | (proposal) js files, MB
>      ------------------+------------------------+------------------------
>      no compression                        7.4                      5.8
>      HTTP compression                      2.7                      1.4
>
> Had this feature worked as intended, we would always transfer only the zipped
> index files and the transfer size would not depend on whether the server uses
> HTTP compression. But does this really outweigh the reasons stated above?
>
> Summing all up. Removing the zipped index files feature will make the overall
> interactive search feature (JDK-8141492) more robust. It will be less
> complicated, have fewer dependencies (JSZip, JSZip Utils), and will push the
> optimization down to HTTP.
>
> Testing
> =======
>
> Here is how I tested this change.
>
> 1. make clean && make docs
> 2. Standalone test
>      2.1. Opened the browser at file://...images/docs/index.html
> 3. HTTP test
>      3.1. Started an HTTP server at build/...images/docs
>      3.2. Opened the browser at http://localhost...images/docs/index.html
>
> Browser cache was cleared each time immediately before accessing the index.html page.
> In both cases I checked that no zipped index files or the related JavaScript
> libraries were accessed, and that the search worked as intended.
>
> I also tried to access the resulting javadoc pages, served by an HTTP server on
> my laptop, from a couple of mobile devices, all of which were on the same WiFi
> network. Everything worked as intended.
>
> Thanks,
> -Pavel
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/javadoc-dev/attachments/20200203/77bbeb95/attachment.htm>