Skip to content

Add hints mechanism for scyjava converters#54

Merged
ctrueden merged 10 commits intomasterfrom
to-xarray
Nov 3, 2022
Merged

Add hints mechanism for scyjava converters#54
ctrueden merged 10 commits intomasterfrom
to-xarray

Conversation

@elevans
Copy link
Copy Markdown
Member

@elevans elevans commented Oct 26, 2022

Intro

This PR adds **kwargs to scyjava's converter mechanism. This change is necessary to support the to_xarray() changes in the pipe for pyimagej (see imagej/pyimagej#231) for more details.

Features

  • Adds **kwarg** support -- enables ij.py.to_java(data, dim_order=["x", "y", "z"])
  • Inspects signature of the identified converter. inspect.signature doesn't work on Java objects. Here we just drop the kwargs. I think this is better than trying to add signature detection for Java methods.

@elevans elevans requested review from ctrueden and gselzer October 27, 2022 14:40
@ctrueden ctrueden changed the title Add **kwargs to scyjava Add hints mechanism for scyjava converters Nov 1, 2022
Copy link
Copy Markdown
Member

@ctrueden ctrueden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @elevans. I reviewed this PR thoroughly, and force-pushed an improved version of the logic. There were a few things that needed to be addressed:

  1. I changed the kwargs name from kwargs to hints, since the intent is for these key/value pairs to provide hints on specifics of the conversion operation.
  2. I bumped the minor version (1.7.1 → 1.8.0) because this is new API: it modifies existing functions to now have kwargs support, which is a backwards-compatible API change.
  3. There were no tests, so I remedied that. I added new Python-to-Java converters for int-to-Byte and int-to-Short in case the hint type='byte' or type='short' respectively is given. While at it, I also made type='long' work to convert to long when values are within int boundaries (although I didn't test this latter feature yet 😐).
  4. In the course of writing tests, I discovered a bug: the hints mechanism was not propagating hints to the predicate functions, only to the converter functions. So I fixed that as well.
  5. In the course of doing all this, it became convenient to introduce a new jinstance function for testing whether an object is an instance of a particular Java class. This was useful for testing immediately, but will also be more important later as the JEP work continues, since isinstance unfortunately does not behave properly with JEP as of this writing.
  6. Finally, I did a couple of other miscellaneous cleanups.

Now that all of that is addressed, I think this PR is looking good to merge!

@ctrueden
Copy link
Copy Markdown
Member

ctrueden commented Nov 1, 2022

There is one problem though: the CI is currently broken on the master branch. We should fix that first, and then rebase this PR one more time on top, to be sure all checks continue to pass.

@ctrueden ctrueden force-pushed the to-xarray branch 2 times, most recently from f2f6291 to 7f54af5 Compare November 3, 2022 01:46
ctrueden and others added 10 commits November 2, 2022 22:34
Certainly we require at least OpenJDK 8, although on conda-forge,
8.0.112 is the oldest anyway, so this rule has no effect in practice.

For the developer environment, for now we restrict to OpenJDK 11
maximum, not OpenJDK 17, because we want CI to use OpenJDK 11
(the scijava-tables library has a bug with OpenJDK 17 as of this
writing). In the long term, restricting OpenJDK in this way is the
wrong thing to do, but it's tolerable for now to get tests passing.
There were two conflicting assumptions in the codebase:

1. If the library is not found, a falsy value is returned; and
2. If the library is not found, an exception is raised.

When initializing converters, it's looking for the falsy value,
i.e. just a simple test of whether the dependency is available.
But when importing the dependency in a function that actually uses it,
an exception is preferred, to ensure the import never ends up as None.

This commit fulfills both scenarios by adding a boolean required flag
to the respective import functions. If a gentle check for availability
is preferred, one can pass required=False to obtain that behavior now.
This is more consistent with the other imports from typing.

We didn't do it before because previously, List was a property
for the jimported java.util.List class. But now all imported
Java classes are inside the JavaClasses struct, so we're OK.
* Fix float and double converter predicates.
* Add tests for floating point infinity and NaN.
* Use clearer and more consistent variable names.
* Use jinstance function to test converted types.
* Assert converted types and values more consistently.
* Remove the convoluted stringified asserts.
Hints are freeform key/value pairs that converters may use if desired to
affect whether and how particular objects are converted between types.

Co-authored-by: Curtis Rueden <ctrueden@wisc.edu>
And while we're at it, stabilize the Converter API. Rather than calling
the predicate and converter functions directly, we now introduce stable
supports and convert methods that always accept hints kwargs, regardless
of whether the linked predicate and/or converter functions do.
And test them, to improve confidence that conversion hints work.
@ctrueden ctrueden merged commit aeb5c3d into master Nov 3, 2022
@ctrueden ctrueden deleted the to-xarray branch November 3, 2022 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants