Use ImagingCore.ptr instead of ImagingCore.id

[homm]

Partially suppress #8340

The non-primary extensions in Pillow don't have direct access to Python's _imaging types, such as Imaging_Type and ImagingObject. However, they often still need to interact with objects passed from Python code. Currently, the raw memory pointer to an Imaging object (ImageCore.id) is passed as an integer and then cast to a memory pointer:

static PyObject *
cms_transform_apply(CmsTransformObject *self, PyObject *args) {
    Py_ssize_t idIn, idOut;
    Imaging im, imOut;

    if (!PyArg_ParseTuple(args, "nn:apply", &idIn, &idOut)) {
        return NULL;
    }

    im = (Imaging)idIn;
    imOut = (Imaging)idOut;

    return Py_BuildValue("i", pyCMSdoTransform(im, imOut, self->transform));
}

It's not safe and can lead to segmentation faults. Python has Capsule objects, which provide a safer way to pass pointers to C code. These are exposed through ImageCore.ptr.

This PR deprecates the raw pointer properties ImageCore.id and ImageCore.unsafe_ptrs, and migrates non-primary extensions to interact with Capsule objects.

[homm]

@python-pillow/pillow-team I need help here)

Could anyone debug pypy3.10 on windows? I can't get what's wrong, since as I can see, the capsule object is correctly restored from the pointer and the capsule object itself is alive.

https://github.com/python-pillow/Pillow/actions/runs/10664609198/job/29556218383?pr=8341#step:27:3342

[homm]

Get it! In pypy repr(capsule) holds the capsuled pointer itself, not the pointer to the PyObject. And even hex(id(capsule)) is not the pointer to PyObject. Lucky, we can reliably distinguish pypy capsule from cpython capsule by absence of trailing "<>".

https://github.com/pypy/pypy/blob/branches/py3.10/pypy/objspace/std/capsuleobject.py#L39

[radarhere]

So is the motivation here just to simplify the code?

Also, we don't necessarily need the deprecations - #4532 (comment)

Pillow makes a commitment to stable public interfaces, which are defined at the Python layer. The C interfaces are explicitly internal, and no effort is made to keep them stable even between minor releases, and no support or warning is given when they change. (IIRC, there are/were people using the internal interfaces, but they've always been explicitly on their own with that)

[homm]

So is the motivation here just to simplify the code?

Added description to PR.

Also, we don't necessarily need the deprecations

While ImageCore is internal Pillow object, it's still Python API. I believe ImageCore.id could be used by third-party extensions (Github search) and can't be removed in a one day.

[radarhere]

I imagine this change was just because you didn't like image.new_block(self.__mode, im.size)?

Rather than moving new_block in C, wouldn't it have been better for the C method to access the size on the image object instead of being passed it? Then this call could become image.new_block(self.__mode).

[homm]

I’d like to clarify that the reason for this change is not a matter of personal preference. The new_block method doesn't pertain to an existing image but instead creates a new block based on the passed parameters. As such, it makes more sense for it to reside in the Image.core module rather than being a method of the image object itself.

There is no intent to modify or enhance the behavior; I was simply correcting the method's improper placement.

[homm]

What's the advantage of using repr(capsule) in the first place? Why not just pass the capsule itself all the time?

I'm not aware of TK API, but it seems it converts any arguments to strings, at least PyImagingPhotoPut receives arguments as const char **argv. So looks like there is no way to pass the capsule object itself, instead repr(capsule) will be passed any way. I just made this conversion more explicit.

[radarhere]

it seems it converts any arguments to strings, at least PyImagingPhotoPut receives arguments as const char **argv. So looks like there is no way to pass the capsule object itself, instead repr(capsule) will be passed any way. I just made this conversion more explicit.

In CPython, sure, but from your comments, not in PyPy.

It seems simpler to note that CPython converts the capsule to a string,
rather than converting to a string and noting that PyPy doesn't convert it?

Depends on whether your thinking was already aligned with how Tk works, I guess. I've created uploadcare#148, but feel free to say you prefer the current version.

[homm]

but from your comments, not in PyPy.

That isn't in my comments. In both implementations, the Capsule is converted to a string. In the case of CPython, it's a string like <capsule object "Pillow Imaging" at 0xffff>, while in PyPy, it's a string like capsule object "Pillow Imaging" at 0xeeee. Where 0xffff is PyCapsule object address, and 0xeeee is the pointer stored in the capsule itself.

[radarhere]

// Special case for PyPy, where the string representation of a Capsule
// refers directly to the pointer itself, not to the PyCapsule object.

Oh, I see, I very much misunderstood this comment.

[radarhere]

Get it! In pypy repr(capsule) holds the capsuled pointer itself, not the pointer to the PyObject.

Interestingly, I guess this is what is preventing you from checking the capsule name against the magic with PyCapsule_IsValid() in ImagingFind().

if (strncmp(name, expected, strlen(expected))) { is has already performed the essence of that check by that point though, right?

[homm]

This check is the best we can do here. It’s not as secure as PyCapsule_IsValid(), since it only verifies that the passed string is a correctly formatted string representation of the Capsule object. It’s still possible to craft a well-formatted string with the wrong pointer, which could result in a segmentation fault when calling a Tk operation. However, it’s still a significant improvement over the previous implementation, which simply obtained some number and assumed it was a pointer.

[radarhere]

For the sake of interest, unsafe_ptrs was originally added in #476 for PyAccess (which was removed in #8182). id has been there since PIL.

I'm not aware of TK API, but it seems it converts any arguments to strings, at least PyImagingPhotoPut receives arguments as const char **argv.

I looked, and this would be mirroring https://www.tcl.tk/man/tcl8.6/TclLib/CrtCommand.htm