Extension methods

[ІП-24 Олександр Ротань]

Yeah, I've heard that I have addressed most of the points in the following
letter in compiler dev. I will copy my mail below, but if you don't really
want to consider this, it's really upsetting. Community craves for it and
asks all the time, and I'm sure If it was proposed as JEP it will be gladly
accepted by Java devs. As for me, that's the most important thing, not
decisions that hand been made a long time ago in the past.

I am pretty sure the point I will provide as advantages has already been
brought up here numerous times, but I will take some time and would
appreciate it if someone from API designers, who once rejected this
proposal, would spare some time to discuss this topic with me.

1. "Poor reflective discoverability" essentially means extension methods
are not accessible when inspecting class members. That is not some
inherent issue of this feature, this is just the way it should be. I'm
not really sure if there is someone who has ever been hurt by this, besides
maybe some parser-based solutions, but let's be honest, this is a:
solvable, b: ridiculously exotic to consider.

2. Documentation accessibility is a strange point for me to be fair. Every
IDE nowadays is capable of fetching the right documentation, as well as
explicitly mentioning where the method comes from, as it is done in C#,
kotlin, swift and many other languages. I don't think anyone has ever heard
complaints about poor documentation of LinQ. Unless someone is writing in
notepad, this is poorly applicable.

3. Not overridable. Should they be? I don't think there is a way to achieve
some kind of "polymorphic" extensions, and I don't think there should be:
extension methods should provide polymorphic target handling, floow LSP
etc., not the other way around.

4. C# extension methods indeed have some very serious drawbacks compared to
Java's, but doesn't this also go the other way around? Canonical utility
functions breach object-oriented code-style, making users write procedural
code like Utils.doSome(obj, params) instead of obj.doSome(params). Its
common issue users just aren't aware of the existence of certain utility
classes and end up with nonoptimal code.
Code without extension methods is always much more verbose, if the API is
supposed to be fluent developer could end up with deep nested invocations.
This brings numerous problems, such as majorly reduced readability, as the
utility methods wrap each other and the developer reads the processing
pipeline "from the end". Also reduced readability always means increased
change of errors.
Writing code using utility methods instead of extensions is just slower,
developers have to waste more time writing a statement than it would be
with extension methods.
Some other advantages I will list below, after this list is ended.

5. One of the answers from the first thread you provided (
https://stackoverflow.com/a/29494337) states that omitting extension
methods is a "philosophical choice", as API developers should define
the API. I have to strongly disagree with that. Extension methods are NOT
part of the API, they are EXTENSION to it. It does not breach
encapsulation as it can't access any internal members of API classes.
Extensions have to be imported explicitly (not the containing class), so
they are explicitly mentioned in the imports list. Also, are utility
methods also breaching this rule then? The only real difference I see is
differences in notation, and extension methods are clearly much more
concise.

Now moving on to some of my personal points. I am also developing some core
libs APIs, and sometimes extension methods are just craving to be used.
Modifying widely-used interfaces is always painful, but that's the only way
to provide a concise way to communicate with existing APIs. Moreover, Java
is an old language and some internal implementations of APIs, like Stream
implementations, are so juncted that adding something new to these classes
directly becomes a spec-ops task. Some APIs, like Gatherer API or Collector
API, arise just from this simple necessity to introduce new behaviour
without modifying extended class itself.

Extension methods are de-facto standard for modern languages: C#, Kotlin,
Swift, Rust and any other modern language provide this option. JS also has
its own unique way of extending APIs. The only modern widely-used language
that does not have extension methods is Python, but that only applies to
built-ins, custom classes could be extended as well. When Java refuses to
introduce this feature, I suffer major damage in the eyes of potential
switchers from ANY other language available right now, which is retrograde
and, as for me and any other supporter of Java, really upsetting.

Introduction of extension will not invalidate previously written code: if
one wants, they can still use utilities as earlier and pretend that nothing
happened. However, for other, significant, if not to say major, part of the
community, this would be a valuable addition..Virtually every Java utility
library: lombok, manifold, xtend - provide extension method functionality,
which clearly shows there is a demand for a feature.

The extension methods are just syntax sugar, nothing more. They provide
better developer experience, make code less verbose and more readable,
which reduces chance of errors and helps to develop apps faster, while not
affecting performance in any way, which is crucial in today's world and may
be a major concurrent advantage.

Also, last but not least, noone from Java developers teams will have to put
efforts into implementation. I am willing to take one full feature
development lifecycle, from drafts to testing and integration. Of course,
final changes will need to be reviewed, but I don't think this will be an
unbearable burden. Regarding implementation, these changes are really
non-invasive, and are really unlikely to introduce any issues, it already
passes all existing tests.

I sincerely hope this letter will bring up this discussion once again, as I
am open for dialogue, but I am standing my ground about numerous advantages
this feature has. If that's possible, I may file a draft JEP and let the
community vote for or against it, to see what Java users think about this
proposal.

Best regards,
Hoping to receive some feedback

Author words: then there were another two mails:

1.
One thing I really don't like about extensions is when people say "you
never know where it comes from". Firstly, if extension is really an
extension and not just a utility method that has been made extension for
fun (which is just bad code design and language shouldn't really assume
that when designing features), its name should describe what it does just
like regular instance method describes. Secondly,  If you require more
elaboration, any IDE is capable of fetching docs for extensions correctly
nowadays, as well as pointing out scope where it comes from, which I guess
should be taken into account.

Regarding ambiguity, the common practice (which also is applied in my
design) is to prioritize members over extensions, there is no ambiguity if
behaviour is well defined.This "potentially" could sometimes result in
source incompatibility with some third-party libraries, but as soon as this
is not conflicting with anything from stdlib, that's not really our
concern. Also, I think it's kind of exotic scenario, and may crash only
with utilities from stdlib classes, which cuts off virtually all production
code from the risk group.And as you mentioned earlier, this problem has
already been present long enough, just not that widely.

Syntax that you have provided, to be fair, smells a bit like PHP to me for
some reason. I don't see why not to use dot notation along with static
imports instead of lists. For me the statement you provided was not that
easy to understand at a first glance, however that's still much better
then traditional utilities.

Also I guess worth mentioning that other languages like C# and kotlin and
many other leave with the same dot notation invocation and I never really
heard that updating language version was really a trouble, especially in
kotlin, even though extension methods have been part of the language for a
very long time already.

2.
And one more remark: when updating Java version, if there are potential
ambiguities is source files of some dependencies for new version, but it
compiled fine for older, code will still work as it was, because my
implementation relies on compile-time code transformation and therefore,
already compiled code won't suddenly start invoking some other methods

I really hope I could at least bring this discussion again as it is indeed
really wanted feature by community!

вт, 23 апр. 2024 г. в 17:56, ІП-24 Олександр Ротань <
rotan.olexandr at gmail.com>:

> Yeah, I've heard that I have addressed most of the points in the following
> letter in compiler dev. I will copy my mail below, but if you don't really
> want to consider this, it's really upsetting. Community craves for it and
> asks all the time, and I'm sure If it was proposed as JEP it will be gladly
> accepted by Java devs. As for me, that's the most important thing, not
> decisions that hand been made a long time ago in the past.
>
> I am pretty sure the point I will provide as advantages has already been
> brought up here numerous times, but I will take some time and would
> appreciate it if someone from API designers, who once rejected this
> proposal, would spare some time to discuss this topic with me.
>
> 1. "Poor reflective discoverability" essentially means extension methods
> are not accessible when inspecting class members. That is not some
> inherent issue of this feature, this is just the way it should be. I'm
> not really sure if there is someone who has ever been hurt by this, besides
> maybe some parser-based solutions, but let's be honest, this is a:
> solvable, b: ridiculously exotic to consider.
>
> 2. Documentation accessibility is a strange point for me to be fair. Every
> IDE nowadays is capable of fetching the right documentation, as well as
> explicitly mentioning where the method comes from, as it is done in C#,
> kotlin, swift and many other languages. I don't think anyone has ever heard
> complaints about poor documentation of LinQ. Unless someone is writing in
> notepad, this is poorly applicable.
>
> 3. Not overridable. Should they be? I don't think there is a way to
> achieve some kind of "polymorphic" extensions, and I don't think there
> should be: extension methods should provide polymorphic target handling,
> floow LSP etc., not the other way around.
>
> 4. C# extension methods indeed have some very serious drawbacks compared
> to Java's, but doesn't this also go the other way around? Canonical utility
> functions breach object-oriented code-style, making users write procedural
> code like Utils.doSome(obj, params) instead of obj.doSome(params). Its
> common issue users just aren't aware of the existence of certain utility
> classes and end up with nonoptimal code.
> Code without extension methods is always much more verbose, if the API is
> supposed to be fluent developer could end up with deep nested invocations.
> This brings numerous problems, such as majorly reduced readability, as the
> utility methods wrap each other and the developer reads the processing
> pipeline "from the end". Also reduced readability always means increased
> change of errors.
> Writing code using utility methods instead of extensions is just slower,
> developers have to waste more time writing a statement than it would be
> with extension methods.
> Some other advantages I will list below, after this list is ended.
>
> 5. One of the answers from the first thread you provided (
> https://stackoverflow.com/a/29494337) states that omitting extension
> methods is a "philosophical choice", as API developers should define
> the API. I have to strongly disagree with that. Extension methods are NOT
> part of the API, they are EXTENSION to it. It does not breach
> encapsulation as it can't access any internal members of API classes.
> Extensions have to be imported explicitly (not the containing class), so
> they are explicitly mentioned in the imports list. Also, are utility
> methods also breaching this rule then? The only real difference I see is
> differences in notation, and extension methods are clearly much more
> concise.
>
> Now moving on to some of my personal points. I am also developing some
> core libs APIs, and sometimes extension methods are just craving to be
> used. Modifying widely-used interfaces is always painful, but that's the
> only way to provide a concise way to communicate with existing APIs.
> Moreover, Java is an old language and some internal implementations of
> APIs, like Stream implementations, are so juncted that adding something new
> to these classes directly becomes a spec-ops task. Some APIs, like Gatherer
> API or Collector API, arise just from this simple necessity to introduce
> new behaviour without modifying extended class itself.
>
> Extension methods are de-facto standard for modern languages: C#, Kotlin,
> Swift, Rust and any other modern language provide this option. JS also has
> its own unique way of extending APIs. The only modern widely-used language
> that does not have extension methods is Python, but that only applies to
> built-ins, custom classes could be extended as well. When Java refuses to
> introduce this feature, I suffer major damage in the eyes of potential
> switchers from ANY other language available right now, which is retrograde
> and, as for me and any other supporter of Java, really upsetting.
>
> Introduction of extension will not invalidate previously written code: if
> one wants, they can still use utilities as earlier and pretend that nothing
> happened. However, for other, significant, if not to say major, part of the
> community, this would be a valuable addition..Virtually every Java utility
> library: lombok, manifold, xtend - provide extension method functionality,
> which clearly shows there is a demand for a feature.
>
> The extension methods are just syntax sugar, nothing more. They provide
> better developer experience, make code less verbose and more readable,
> which reduces chance of errors and helps to develop apps faster, while not
> affecting performance in any way, which is crucial in today's world and may
> be a major concurrent advantage.
>
> Also, last but not least, noone from Java developers teams will have to
> put efforts into implementation. I am willing to take one full feature
> development lifecycle, from drafts to testing and integration. Of course,
> final changes will need to be reviewed, but I don't think this will be an
> unbearable burden. Regarding implementation, these changes are really
> non-invasive, and are really unlikely to introduce any issues, it already
> passes all existing tests.
>
> I sincerely hope this letter will bring up this discussion once again, as
> I am open for dialogue, but I am standing my ground about numerous
> advantages this feature has. If that's possible, I may file a draft JEP and
> let the community vote for or against it, to see what Java users think
> about this proposal.
>
> Best regards,
> Hoping to receive some feedback
>
> Author words: then there were another two mails:
>
> 1.
> One thing I really don't like about extensions is when people say "you
> never know where it comes from". Firstly, if extension is really an
> extension and not just a utility method that has been made extension for
> fun (which is just bad code design and language shouldn't really assume
> that when designing features), its name should describe what it does just
> like regular instance method describes. Secondly,  If you require more
> elaboration, any IDE is capable of fetching docs for extensions correctly
> nowadays, as well as pointing out scope where it comes from, which I guess
> should be taken into account.
>
> Regarding ambiguity, the common practice (which also is applied in my
> design) is to prioritize members over extensions, there is no ambiguity if
> behaviour is well defined.This "potentially" could sometimes result in
> source incompatibility with some third-party libraries, but as soon as this
> is not conflicting with anything from stdlib, that's not really our
> concern. Also, I think it's kind of exotic scenario, and may crash only
> with utilities from stdlib classes, which cuts off virtually all production
> code from the risk group.And as you mentioned earlier, this problem has
> already been present long enough, just not that widely.
>
> Syntax that you have provided, to be fair, smells a bit like PHP to me for
> some reason. I don't see why not to use dot notation along with static
> imports instead of lists. For me the statement you provided was not that
> easy to understand at a first glance, however that's still much better
> then traditional utilities.
>
> Also I guess worth mentioning that other languages like C# and kotlin and
> many other leave with the same dot notation invocation and I never really
> heard that updating language version was really a trouble, especially in
> kotlin, even though extension methods have been part of the language for a
> very long time already.
>
> 2.
> And one more remark: when updating Java version, if there are potential
> ambiguities is source files of some dependencies for new version, but it
> compiled fine for older, code will still work as it was, because my
> implementation relies on compile-time code transformation and therefore,
> already compiled code won't suddenly start invoking some other methods
>
> I really hope I could at least bring this discussion again as it is indeed
> really wanted feature by community!
>
> вт, 23 апр. 2024 г. в 17:49, Kasper Nielsen <kasperni at gmail.com>:
>
>> Hi Rotan,
>>
>> If you search the net, you should be able to find a number of
>> resources for why extension methods will not be added to Java. Here
>> are some
>>
>>
>> https://stackoverflow.com/questions/29466427/what-was-the-design-consideration-of-not-allowing-use-site-injection-of-extensio/29494337#29494337
>> https://www.youtube.com/watch?v=qKeMB7OoGJk&t=1855s
>>
>>
>> /Kasper
>>
>> On Tue, 23 Apr 2024 at 15:35, ІП-24 Олександр Ротань
>> <rotan.olexandr at gmail.com> wrote:
>> >
>> > Recently I wrote an email to compiler-dev about my proposal (and
>> implementation) of extension methods. One of the users forwarded me here
>> saying that my proposal might be considered here. Below I will duplicate
>> the mail I have sent to compiler dev. Hope for your feedback.
>> >
>> > Dear Java Development Team,
>> >
>> > I hope this email finds you all in good spirits. I am writing to
>> propose the integration of extension methods into the Java programming
>> language, a feature that I believe holds considerable promise in enhancing
>> code readability and maintainability.
>> >
>> > Extension methods offer a means to extend the functionality of existing
>> classes in a manner that aligns with Java's principles of static typing and
>> object-oriented design. The proposed syntax, exemplified as follows:
>> >
>> > public static void extensionMethod(extends String s) { ... }
>> >
>> > adheres to established conventions while providing a concise and
>> intuitive means of extending class behavior. Notably, the use of the
>> `extends` keyword preceding the type parameter clearly denotes the class to
>> be extended, while the method itself is declared as a static member of a
>> class.
>> >
>> > I wish to emphasize several advantages of extension methods over
>> traditional utility functions. Firstly, extension methods offer a more
>> cohesive approach to code organization by associating functionality
>> directly with the class it extends. This promotes code clarity and reduces
>> cognitive overhead for developers, particularly when working with complex
>> codebases.
>> >
>> > Secondly, extension methods enhance code discoverability and usability
>> by integrating seamlessly into the class they extend. This integration
>> allows developers to leverage IDE features such as auto-completion and
>> documentation tooltips, thereby facilitating more efficient code
>> exploration and utilization.
>> >
>> > Lastly, extension methods promote code reusability without the need for
>> subclassing or inheritance, thereby mitigating the risks associated with
>> tight coupling and inheritance hierarchies. This modularity encourages a
>> more flexible and adaptable codebase, conducive to long-term
>> maintainability and scalability.
>> >
>> > In light of these benefits, I believe that the integration of extension
>> methods into Java would represent a significant step forward for the
>> language, aligning it more closely with modern programming paradigms while
>> retaining its core strengths.
>> >
>> > I am eager to discuss this proposal further and collaborate with you
>> all on its implementation. Your insights and feedback would be invaluable
>> in shaping the future direction of Java development.
>> >
>> > Thank you for considering this proposal. I look forward to our
>> discussion.
>> >
>> > The draft implementation can be found in the following branch of the
>> repository: https://github.com/Evemose/jdk/tree/extension-methods. I am
>> new to Java compiler development, so any tips or remarks about what I have
>> done in the wrong way or in the wrong place. I will add complete test
>> coverage a bit later, but for now, there is a link to the archive on my
>> google drive, which contains built in jdk for windows x86-64. If someone is
>> willing to participate in testing as a user, I would appreciate any help.
>> >
>> > Best regards
>> >
>> > PS: Note about internal implementation: it introduces a new flag -
>> EXTENSION, that is equal to 1L<<32. It seems like it takes the last vacant
>> bit in a long value type that has not been taken by flags. Not sure what
>> the compiler development community should do about this, but it feels like
>> it could be an obstacle to new features that might be introduced later.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/amber-dev/attachments/20240423/b460a49e/attachment-0001.htm>