Nashorn comment nodes in the syntax tree

[Sundararajan Athijegannathan]

Hi,

Sorry for the delayed response. I'm yet to check your test. I'll file a
bug once I reproduce/check this at my end and let you know the bug id.

Thanks,

-Sundar


On 7/12/2016 2:51 PM, Emilian Bold wrote:
> Yes, it must be a bug: comments are certainly swallowed in the nodes
> on my JRE. I see this for FunctionNode but also for IdentNode and
> perhaps more.
>
> It makes getting actual offsets a pain and I would strip-out comments
> from the text before sending it to the parser, but I need those too.
>
> $ uname -a
> Darwin mac.local 15.5.0 Darwin Kernel Version 15.5.0: Tue Apr 19
> 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 x86_64
>
> $ java -version
> java version "1.8.0_66"
> Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
>
> Java 1.8.0_66
>
> Script text:
> --begin--
> function hello(){ 
>  return 'world';
> }
>
> //some comment
> hello();
> --end--
>
> [<unknown>] function {U%}hello()
> Function text:
> --begin--
> { 
>  return 'world';
> }
>
> //some comment
> --end-
>
>
> junit.framework.AssertionFailedError: The function should not swallow
> the comment
>
>
> Test:
> package com; import jdk.nashorn.internal.ir.FunctionNode; import
> jdk.nashorn.internal.ir.LexicalContext; import
> jdk.nashorn.internal.ir.visitor.NodeVisitor; import
> jdk.nashorn.internal.runtime.ErrorManager; import
> jdk.nashorn.internal.runtime.ScriptEnvironment; import
> jdk.nashorn.internal.runtime.Source; import
> jdk.nashorn.internal.runtime.options.Options; import
> junit.framework.TestCase; import java.io.PrintWriter; public class
> NashornCommentBugTest extends TestCase { public void testComment(){
> System.out.println("Java " + System.getProperty("java.version"));
> System.out.println(); String text = "function hello(){ \n" + " return
> 'world';\n" + "}\n\n" + "" + "//some comment\n" + "hello();"; Source
> source = Source.sourceFor("test.js", text); //NOI18N Options options =
> new Options("nashorn"); // NOI18N options.process(new String[]{
> "--parse-only=true", // NOI18N "--empty-statements=true", // NOI18N
> "--debug-lines=false"}); // NOI18N ScriptEnvironment env = new
> ScriptEnvironment(options, new PrintWriter(System.out), new
> PrintWriter(System.err)); ErrorManager errorManager = new
> ErrorManager(); jdk.nashorn.internal.parser.Parser parser = new
> jdk.nashorn.internal.parser.Parser(env, source, errorManager);
> FunctionNode root = parser.parse(); System.out.println("Script
> text:\n--begin--\n" + text+"\n--end--\n"); root.accept(new
> NodeVisitor<LexicalContext>(new LexicalContext()) { @Override public
> boolean enterFunctionNode(FunctionNode functionNode) { if
> (!functionNode.getKind().equals(FunctionNode.Kind.SCRIPT)) {
> System.out.println(functionNode.toString()); String functionText =
> text.substring(functionNode.getStart(), functionNode.getFinish());
> System.out.println("Function text:\n--begin--\n" +
> functionText+"\n--end-\n"); assertFalse("The function should not
> swallow the comment", functionText.contains("some comment")); } return
> super.enterFunctionNode(functionNode); } }); } }
>
>
>
>
>
> --emi
>
> On Mon, Jun 13, 2016 at 6:31 AM, Sundararajan Athijegannathan
> <sundararajan.athijegannathan at oracle.com
> <mailto:sundararajan.athijegannathan at oracle.com>> wrote:
>
>     start and finish should not include comments around. If it does,
>     it is a bug. Please do file a bug and send us test case here and
>     we'll file a bug
>
>     Thanks,
>
>     -Sundar
>
>
>     On 6/10/2016 9:05 PM, Emilian Bold wrote:
>>     BTW, jdk.nashorn.internal.ir.IdentNode (and other classes) seem
>>     to have the habit of "absorbing" the nearby comments and have
>>     incorrect start/finish offsets.
>>
>>     Is this intentional or a bug?
>>
>>     I haven't tested yet but I wonder if it propagates to the TreeAPI
>>     (ie. does IdentifierTree also swallow adjacent comments?)
>>
>>
>>
>>     --emi
>>
>>     On Wed, Apr 27, 2016 at 5:18 PM, Sundararajan Athijegannathan
>>     <sundararajan.athijegannathan at oracle.com
>>     <mailto:sundararajan.athijegannathan at oracle.com>> wrote:
>>
>>         Hi,
>>
>>         I filed an enhancement request:
>>         https://bugs.openjdk.java.net/browse/JDK-8155242
>>
>>         Thanks,
>>
>>         -Sundar
>>
>>
>>         On 4/27/2016 12:40 AM, Emilian Bold wrote:
>>         > Hello,
>>         >
>>         > NetBeans used a modified Rhino that had useful features for
>>         an IDE like
>>         > comment nodes and parser-level sanitization.
>>         >
>>         > As far as I know there is no way to get the comments in the
>>         Nashorn Tree
>>         > API (created as part of JEP 236) or in the
>>         jdk.nashorn.internal.ir <http://jdk.nashorn.internal.ir> Node
>>         > subclasses.
>>         >
>>         > Obviously having access to comment contents and comment
>>         offsets is very
>>         > useful. An IDE, for example, would provide code folding for
>>         the multiline
>>         > comment.
>>         >
>>         > Am I missing some proper way to get this information?
>>         >
>>         > I believe an extra option for Parser.create() would be nice
>>         (plus a
>>         > CommentTree and a visitComment method).
>>         >
>>         > --emi
>>
>>
>
>