[C API] Add PyLong_GetNumBits() function

[vstinner]

Feature or enhancement

In Python 3.13 alpha1, I removed the private _PyLong_NumBits() function. It seems like this function is used by pywin32 and MariaDB projects: see issue gh-119336.

I propose to add a public function to replace it:

Py_ssize_t _PyLong_GetNumBits(PyObject *obj);

• Return the number of bits on success: greater than or equal to zero.
• Set an exception and return -1 on error.
• Set an OverflowError exception, and return -1 if the number of bits doesn't fit into Py_ssize_t.

The C function is similar to the Python int.bit_length() method.

See also the proposed PyLong_GetSign() function.

Linked PRs

• gh-119714: Add PyLong_GetNumBits() function #119715

[serhiy-storchaka]

Why not return (size_t)-1 as indicator of error? It would be more consisting with the existing C API.

[encukou]

We have a size type that can be negative to express errors. IMO, this should be Py_ssize_t PyLong_GetNumBits(PyObject *obj);.

Can we first add the underscored version back? IMO, we should design the API first, and only then start removing a function that's been there since 2003 and is used in popular software like mariadb or panda3d.
I see no reason to break people's working code. We have a better deprecation mechanism coming up in PEP-743.

[vstinner]

Can we first add the underscored version back?

The revert is discussed in a separated issue: #119336 I just enabled automerge on its PR: #119418.

First, I wasn't sure that it's worth it to restore it if a single project uses it, since you can easily call PyObject_CallMethod(obj, "bit_length", NULL) in C to get the same value. Since I discovered more projects using it, I proposed to add a public function.

[vstinner]

@serhiy-storchaka:

Why not return (size_t)-1 as indicator of error? It would be more consisting with the existing C API.

@encukou:

We have a size type that can be negative to express errors. IMO, this should be Py_ssize_t PyLong_GetNumBits(PyObject *obj);.

I prefer to return an int to indicate success or error. But since you both prefer to reuse the return value to also indicate an error, I changed the return type to Py_ssize_t (which was my first idea in fact).

[vstinner]

I wrote the PR #119715 to implement the function.

[serhiy-storchaka]

Py_ssize_t has a range twice smaller than size_t. Usually this is not a problem if used for size or index, because no object can be larger than PY_SSIZE_T_MAX bytes. But you can create an integer having more than 2**31 bits on 32-bit platform, and even more than 2**32 bits. It is not accident that _PyLong_NumBits uses size_t instead of Py_ssize_t.

It is still limited. This is why int.bit_length does not use it. And this is the reason of not adding the public C API earlier -- it does not support all int objects.

Perhaps returning a 64-bit integer, even on 32-bit platform, can solve the issue. In this case it does not matter whether the return type is signed or unsigned.

[vstinner]

IMO 2**31 limit on a 32-bit system is a reasonable limitation, since Python is slow to manage such very large number. The OverflowError is already documented in the API documentation, we can also suggest calling bit_length() for very large integer.

[vstinner]

we can also suggest calling bit_length() for very large integer

Done in the PR.

[vstinner]

I created a ticket in the C API Working Group: capi-workgroup/decisions#28

[erlend-aasland]

See also previous slightly related discussion:

• create a numbits() method for int and long types #47689

[vstinner]

The C API Working Group rejected this API: PyLong_AsNativeBytes() should be used instead.