RFR [15] 8237909: Remove zipped index files feature

[Pavel Rappo]

This concludes 2 weeks period for this review. I haven't heard any concerns on
the proposed change. The tweaks suggested in this thread have been made.
The functionality has then been retested, both manually and automatically, using
our CI system.

This change doesn't seem to require a CSR. Still, a release note, JDK-8238760,
has been created. I've also added further details to the parent task's "Comments"
section, JDK-8237909.

The change will be publicly available in the JDK 15 Early-Access Builds:

    https://jdk.java.net/15/

Please try it and report any concerns.

-Pavel

> On 28 Jan 2020, at 15:55, Pavel Rappo <pavel.rappo at oracle.com> wrote:
> 
> Hello,
> 
> Please review the change for https://bugs.openjdk.java.net/browse/JDK-8237909:
> 
>   http://cr.openjdk.java.net/~prappo/8237909/webrev.00/
> 
> This change removes the "zipped index files" feature, which was introduced as
> part of 8141492: Implement search feature in javadoc.
> 
> The "zipped index files" feature consists of generating the zipped index files
> on the back end, and fetching & unzipping mechanics on the front end.
> 
> When documenting source files, the standard doclet accumulates index which is
> later used by the JavaScript code serving the interactive search. The index
> is written in two formats, .js (JavaScript) and .json (JSON). The latter is
> then zipped.
> 
> When a browser accesses the pages using "http://" urls, the .zip index files are
> transferred using XHR. Those files are then unzipped by the browser, using the
> JSZip library, and parsed as JSON. If the transfer of the .zip index files fails
> for whatever reason, the browser falls back on the alternative mechanism. This
> mechanism transfers the .js index files by referring to them from dynamically
> inserted <script src="... .js"> elements. Those files then are not additionally
> parsed, as they are already data hardcoded in JavaScript code.
> 
> One of the reasons the .zip index files transfer may fail is using javadoc pages
> in the "standalone" mode. When a browser accesses "file://" urls, there's no
> HTTP server to send the XHR requests to. So the fallback mechanism kicks in and
> the browser loads the .js index files instead.
> 
> Analysis
> ========
> 
> From what I understand, the original intent was to reduce the transfer size of
> the index files. The observations made during the recent upgrade of JSZip
> (JDK-8236700) suggest that the feature is not working as intended. It is not
> clear if it ever did. The proposal is to remove it for the following reasons:
> 
> 1. The feature in its current state does more harm than good (see JDK-8236922)
> 2. Fixing, debugging, testing, and evolving require expertise beyond that of
>   typical for the javadoc area
> 3. The feature significantly complicates the front end and less so the back end
>   code
> 4. The feature relies on the 3rd party libraries, which require tracking &
>   periodical upgrades
> 5. The difference in size between the .zip and .js files is not that big (see below)
> 6. The index files are transferred once and then used from cache
> 7. Modern HTTP servers provide compression. This makes the net result
>   virtually the same, compare:
> 
>                      | (current) js + zip, MB | (proposal) js files, MB
>    ------------------+------------------------+------------------------
>    no compression                        7.4                      5.8
>    HTTP compression                      2.7                      1.4
> 
> Had this feature worked as intended, we would always transfer only the zipped
> index files and the transfer size would not depend on whether the server uses
> HTTP compression. But does this really outweigh the reasons stated above?
> 
> Summing all up. Removing the zipped index files feature will make the overall
> interactive search feature (JDK-8141492) more robust. It will be less
> complicated, have fewer dependencies (JSZip, JSZip Utils), and will push the
> optimization down to HTTP.
> 
> Testing
> =======
> 
> Here is how I tested this change.
> 
> 1. make clean && make docs
> 2. Standalone test
>    2.1. Opened the browser at file://...images/docs/index.html
> 3. HTTP test
>    3.1. Started an HTTP server at build/...images/docs
>    3.2. Opened the browser at http://localhost...images/docs/index.html
> 
> Browser cache was cleared each time immediately before accessing the index.html page.
> In both cases I checked that no zipped index files or the related JavaScript
> libraries were accessed, and that the search worked as intended.
> 
> I also tried to access the resulting javadoc pages, served by an HTTP server on
> my laptop, from a couple of mobile devices, all of which were on the same WiFi
> network. Everything worked as intended.
> 
> Thanks,
> -Pavel
>