"os" - make this a real namespace?

[Thomas Stüfe]

Hi David,

my intent was not to attack the existing code, but to ask about the
original design intentions and possibly come up with ideas to improve it.
See my further answers inline.

On Wed, Oct 19, 2016 at 2:19 PM, David Holmes <david.holmes at oracle.com>
wrote:

> Hi Thomas,
>
> On 19/10/2016 7:07 PM, Thomas Stüfe wrote:
>
>> Hi David!
>>
>> On Wed, Oct 19, 2016 at 9:02 AM, David Holmes <david.holmes at oracle.com
>> <mailto:david.holmes at oracle.com>> wrote:
>>
>>     Hi Thomas,
>>
>>     On 19/10/2016 4:10 PM, Thomas Stüfe wrote:
>>
>>         Hi all,
>>
>>         a small question.
>>
>>         I sometimes stumble over the fact that "os" is a class, not a
>>         namespace.
>>
>>
>>     ?? AFAIK everything in hotspot is a class not a namespace - we don't
>>     use "namespaces".
>>
>>
>> I meant that it is used as one would use a C++ namespace, not a class.
>> As far as I see any class derived from AllStatic is actually a namespace
>> in the sense that it serves as bracket for a number of related static
>> functions.
>>
>
> Okay ... if that is what you mean by a namespace ... I always thought
> namespaces were like packages, a level up from classes.
>
>
I always used namespace as a common scope for declarations which belong
together, be that classes, global functions, variables. I always thought
this is how class "os" is used in the hotspot.

I also use C++ namespace to isolate coding in large projects I do not own
but where my symbols are, for technical reasons, visible anywhere but I
want to avoid name clashes. A typical porter headache. E.g. we have a
namespace "sap" in our coding, just to keep our stuff separate from global
symbols.


>
>>         And that we include the platform dependent additions into the
>>         middle of this class.
>>
>>
>>     Build-time specialization. It allows for the os API to actually be
>>     different on different platforms, as opposed to just being
>>     implemented differently.
>>
>>
>> The "os" API is a shared, platform independent API, so there should be
>> no difference between platforms. There should be no need for it to
>> export platform specifics - its whole intent is to hide those specifics.
>>
>
> But that is your current design perspective of what the os API should be.
> What it is is something with a very long history and which has had to
> accommodate different things over time. At one point a lot of the JDK
> native code would call into the VM for functionality that is now directly
> implemented in JDK native code. There's a lot of historical baggage here.
>
>
Looking into the various os_<os>.hpp files, I see:
>> 1) things where the declaration does differ between oses and therefore
>> they cannot be called in shared code without #ifdefs (e.g. all <os>
>> subclasses).
>> 2)Things where the declaration is shared but the implementation differs.
>> Again, two cases:
>>     a) Either implementation is not time critical. In that case the
>> declaration should live in os.hpp and the implementation should live in
>> some platform dependent C++ file.
>>     b) Or implementation is time critical and must be inline, in which
>> case a separate platform dependent header would be needed.
>>
>> For (1) and (2b), C++ namespaces would be more convenient. Now, you are
>> forced to include the platform specific file into the class os{}
>> declaration, because there can just be one:
>>
>> os.hpp
>> class os {
>> ...
>> #include <os_xxx.hpp>
>>
>> };
>>
>> . With a namespace, you can add functions to the namespace in various
>> disjunct places and hence could write:
>>
>> os.hpp
>> namespace os { ... functions ... }
>>
>> os_xxx.hpp
>> namespace os { ... functions ... }
>>
>> which would be more natural and
>>
>
> Yes I can see that as an alternative way to expand the os API. Though I
> still prefer to group functionality in a class.
>
>
I would argue that the advantage of namespaces here is that the special
handling of platform specific headers is not needed anymore. Now, when
reading any os_<os>/<cpu> header, I need to keep in mind that the content
of this header gets inserted into the middle of a class definition. That is
just rather exotic and unexpected and tripped me over a few times already.



>         This has a number of repercussions, like not being able to
>>         include the
>>         platform dependent files (os_<os>_<cpu>) directly, not being able
>> to
>>
>>
>>     I'd call that a feature - they are not intended to be standalone APIs.
>>
>
There are a number of useful "os" APIs which I would sometimes like to use
without the bagage of including the whole os.hpp header. For instance,
os::malloc(). Normally, I would forward declare them, but this is not
possible for class functions.


>
>>
>> Right now os_<os>.hpp exports the os::<os> api, which one may want to
>> use separately from "os" because they expose platform dependent APIs
>> which are conceptionally lower than the os namespace. At least that is
>> how I always did interpret the intention behind os::<os>.
>>
>
> I wouldn't say lower - they extend the os API with platform specific
> functionality and concepts. The idea is that <os> specific code that wants
> to use OS facilities that are specific to that <os> access them through the
> os::<os> class.


>
> But actually, because the "Aix" class is part of "os", cannot be used
>> separately and is exposed to the whole of the VM, I always avoided
>> putting anything os::Aix if it could be helped. Hence, for AIX, we added
>> porting_aix.hpp for AIX specific functions which are not to be used
>> outside os/aix/vm. Or mostly just plain left functions to be file scope
>> static inside os_aix.cpp. So, os::Aix was pretty useless for me as a
>> porter.
>>
>
> Seems you made a decision that the os::AIX class didn't meet your ideas as
> to how platform specifics should be handled and so went with an alternative
> design. Wouldn't that make it "useless" because you chose not to use it?


When the AIX port started, the os interface was not well documented. Nor
was there anyone I could ask because OpenJDK did not yet exist. So I had to
deduce the intent of the original authors from the code and try to fill it
with life as best as possible.


>
>>         forward declare functions from the "os" namespace (e.g.
>>         os::malloc) etc. I
>>         also cannot split implementations from "os" functions to different
>>         implementation files without problems.
>>
>>         It seems to me all compiler nowadays support namespaces, would
>>         it not make sense to convert "os" to a real namespace?
>>
>>
>>     Not being a C++ aficionado I'm not sure exactly what that would
>>     entail - as far as I know we don't use C++ namespaces anywhere in
>>     hotspot.
>>
>>
>> We start using all kinds of modern C++ features. Templates pop up all
>> over the place. Namespaces in contrast are an old and easily understood
>> feature. We already use it inside our own port.
>>
>>
>>         While we are at it, what is the reason for the "<os>" sub
>>         classes? e.g.
>>         os::Bsd, os::Aix etc? It makes integrating patches between
>> platforms
>>         difficult and, to me, does not seem to serve any clear purpose.
>>
>>
>>     Must admit this arrangement has also had me confused at times. I
>>     think it is way to add a per-OS helper class for the main os API
>>     implementation.
>>
>>         If the purpose is to be a very low wrapper around OS
>>         particularities, it
>>         makes no sense to have them in the "os" namespace and to make
>>         them visible
>>         to the shared sections of the VM. E.g. there should be no reason
>>         to access
>>         "os::Bsd" functions from outside os/bsd/vm, or to access
>> "os::Posix"
>>         functions outside implementations specific for Posix platforms.
>>
>>
>>     Not sure how you make, for example os::BSD accessible from all
>>     classes in os/bsd/vm yet not be visible anywhere else ??
>>
>>
>> I think there is no real reason for os::Bsd to exist at all. Either we
>> have shared functions with platform dependent implementation, then they
>> should be declared in "os". Or they are completely platform specific,
>> then they can be moved to a platform specific header outside of "os"
>> like we did with porting_aix.hpp.
>>
>
> Sure you could do that. But 20 years ago that wasn't how things were
> designed and we have what we have today. As I said a lot of baggage.
>
> Personally I find the nesting of the concrete os API quite natural:
> os::win32 to me is better than unrelated os and win32 classes or namespaces.
>
>
I think there is code structure, and then there is exposure. Both are
separate things but currently interwoven in the hotspot.

Putting Win32 specifics into os::win32 makes sense. But there is no reason
to expose os::win32 to shared parts of the VM, considering how much the
hotspot programmers at Oracle dislike platform specific #ifdefs in shared
code.

I understand that this is historical? Before namespaces, you had to define
a class as common bracket, and a class definition must be complete to be
valid, so os::<OS> is automatically visible everywhere. But with namespaces
this could be disentangled. We could have a globally visible "os" namespace
with platform independent shared functions, as well as an "os::win32"
namespace which is only visible in windows specific implementation files.

Kind Regards, Thomas


> Cheers,
> David
> -----
>
>
>  Plus it also needs to potentially be visible from os_cpu/bsd_XXX/vm.
>>
>>
>>     There is a lot of cleanup in this area slated for the future -
>>     hopefully Java 10. POSIX refactoring etc.
>>
>>
>> Sure!
>>
>> Kind Regards, Thomas
>>
>>
>>     Cheers,
>>     David
>>
>>
>>         Thanks, and Kind Regards, Thomas
>>
>>
>>