RFR: 8275405: Linking error for classes with lambda template parameters and virtual functions

[Stefan Karlsson]

We encountered the following linking error when trying to build Generational ZGC on Windows:

jvm.exp : error LNK2001: unresolved external symbol "const ZBasicOopIterateClosure<class <lambda_9767402e468f9fc654281195f9700e48> >::`vftable'" (??_7?$ZBasicOopIterateClosure at V<lambda_9767402e468f9fc654281195f9700e48>@@@@6B@)


I narrowed this down to a simple reproducer, which doesn't link when built through the HotSpot build system:

#include <functional>
std::function<void()> = [](){};


I found that we have a line in our make files that filters out symbols that contain the string vftable (though it checks the mangled name, so a bit hard to find):

else ifeq ($(call isTargetOs, windows), true)
  DUMP_SYMBOLS_CMD := $(DUMPBIN) -symbols *.obj
  FILTER_SYMBOLS_AWK_SCRIPT := \
      '{ \
        if ($$7 ~ /??_7.*@@6B@/ && $$7 !~ /type_info/) print $$7; \
      }'


The following line prints the vftable symbol if it doesn't contain the string 'type_info':
 if ($$7 ~ /??_7.*@@6B@/ && $$7 !~ /type_info/) print $$7;

The printed values are added to a list of symbols that get filtered out of the mapfile, which is then passed to the linker.

I can get the code to link if I add a second exception for vftable symbols containing the string 'lambda':
 if ($$7 ~ /??_7.*@@6B@/ && $$7 !~ /type_info/ && $$7 !~ /lambda/) print $$7;

I did an additional experiment where I completely removed this filtering of vftable symbols. When I did that the linker complained that we used more than 64K symbols.

-------------

Commit messages:
 - 8275405: Linking error for classes with lambda template parameters and virtual functions

Changes: https://git.openjdk.java.net/jdk/pull/6030/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6030&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8275405
  Stats: 18 lines in 1 file changed: 17 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/6030.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/6030/head:pull/6030

PR: https://git.openjdk.java.net/jdk/pull/6030