Consider using LinkML-Scala?

[Ostrzyciel]

Hi! I've been following your hard work here for a while :) really appreciate it. (Also, thank you for maintaining Protégé! I taught it to students when I was in academia.)

I've just noticed your issue here: linkml/linkml#3912

I tried it with LinkML-Scala (our implementation of the LinkML spec from Scala), and it is indeed reported as an error:

In general, we took the approach of starting from a blank canvas and sticking to the spec as closely as possible. We were also frustrated by linkml/linkml's spec non-compliance.

This made me think – would it be possible to reuse some of our work for your purposes? The "secondary aims" you list in the README are already possible in LinkML-Scala, on the JVM. We have a fully-featured SchemaView interface, schema derivation, and infrastructure for building generators. These generators can be simply Java classes, you only need a Maven dependency on LinkML-Scala: https://central.sonatype.com/artifact/eu.neverblink.linkml/schemaview_3

Example generator (RDFS): https://github.com/NeverBlink-OSS/linkml-scala/blob/main/generator/src/eu/neverblink/linkml/generator/rdfs/RdfsGenerator.scala

With this, it's totally possible to write a Java generator in pure Java. It could use either direct string interpolation or something like javapoet.

Does this sound interesting? To be honest, we will also need a good Java generator at some point, so I would be very happy to support your work here in some way. If for example the Scala interfaces turn out to be annoying to use from Java, we could add Java façades.

[gouttegd]

Hi!

I've been following your hard work here for a while :)

I’ve been looking at your LinkML-Scala as well after you announced it. :)

Also, thank you for maintaining Protégé! I taught it to students when I was in academia.

Thanks, much appreciated. :)

In general, we took the approach of starting from a blank canvas and sticking to the spec as closely as possible.

Yes, I’ve seen your position in this discussion (especially this reply).

I cannot, however, follow that position. I don’t think that taking the LinkML “spec” at its words and ignoring LinkML-Py is a good idea, sorry. Whether we like it or not (and for the record, I do not like it ;) ), right now, LinkML-Py is LinkML. Or rather, LinkML-Py decides what LinkML is.

The “spec” is, at best, an aspirational document; woefully incomplete, and at times, blatantly wrong (in that it is contradicted by LinkML-Py). But LinkML users don’t use the spec – they use LinkML-Py.

We were also frustrated by linkml/linkml's spec non-compliance.

So am I, but I don’t think the solution is to stick to a flawed spec and to ignore the reference implementation. Instead, the solution is to patiently point out all the flaws in the spec (e.g. all the underspecified behaviours, and all the specified behaviours that are contradicted by the actual behaviours implemented by LinkML-Py), until the spec is brought to a state where it can actually be called a “spec”. :)

would it be possible to reuse some of our work for your purposes? […] With this, it's totally possible to write a Java generator in pure Java.

Code generation directly from Java is still very far on my radar for now. I will most certainly do it at some point, but my main focus is still about making it possible to use LinkML-defined data from within a Java program. Having to rely on LinkML-Py for the code generation step is slightly annoying but not a big deal.

I consider supporting stuff such as “dynamic enums” much more important than code generation, because many LinkML schemas out there use them. Several of the projects I am involved with will also need such a support.

When the time will come to enable code generation, I will look again at LinkML-Scala. :) But do keep in mind what I said above: I cannot follow your “stick to the spec and ignore LinkML-Py” approach. LinkML-Java needs to work with schemas designed by and for people who use LinkML-Py.

[Ostrzyciel]

Thanks for the clear answers :)

I cannot, however, follow that position. I don’t think that taking the LinkML “spec” at its words and ignoring LinkML-Py is a good idea, sorry. Whether we like it or not (and for the record, I do not like it ;) ), right now, LinkML-Py is LinkML. Or rather, LinkML-Py decides what LinkML is.

I think that "ignoring LinkML-Py" is an oversimplification, and my unclear wording is most likely to blame here.

• There are things that are not in the spec, but are implemented in LinkML-Py that we think make sense – so we just implement them!
• There are also things that are not in the spec, that we don't agree with (e.g., schemas being non-hermetic). Still, we follow LinkML-Py's behaviors, because otherwise few existing LinkML schemas would work with our implementation.
• Finally, there are some things that we consider actively harmful that we choose to implement differently, such as the issue I mentioned in my original post here. These we choose to implement differently.

You can test LinkML-Scala with your schemas and see if it works. In our testing, it actually worked with more large schemas out of the box than LinkML-Py 🤷

Instead, the solution is to patiently point out all the flaws in the spec (e.g. all the underspecified behaviours, and all the specified behaviours that are contradicted by the actual behaviours implemented by LinkML-Py), until the spec is brought to a state where it can actually be called a “spec”. :)

100% agreed! This is actually what we are doing! :D

You can look for mine & @niegrzybkowski 's PRs and issues in linkml and linkml-model. We also want to separate spec issues from LinkML-Py issues to make this process easier to manage, see here: https://github.com/orgs/linkml/discussions/3833

Some time ago, we were also one of the people advocating for the CODEOWNERS system, which is a step in the right direction.

We have also started making conformance tests, which we think should make it at some point into the spec: https://github.com/NeverBlink-OSS/linkml-scala/tree/main/tests/resources/models

Currently our main bottlenecks are:

• LinkML spec is needlessly tied to LinkML-Py, e.g., in the issue trackers, which creates confusion and makes spec-specific issues hard to discuss.
• There is no clear spec contribution process.
• There is no clear spec governance.
• There is no understanding whether alternative implementations can exist or not. The LinkML porting guide says "yes", the community seems to be confused at best.
• There is no notion of spec compliance in LinkML-Py.

If you could support us on these fronts we would be very grateful... because I think we have the exact same goals. Apologies if I wasn't clear about that from the start, but this is really the direction of our work. It's just that we are finding a lot of friction when doing it.