Incompatible with Google Colab (Python 3.7)

[sgbaird]

!pip install ml-matrics
from ml_matrics.parity import density_hexbins

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-1-48d4b62e713f>](https://localhost:8080/#) in <module>()
----> 1 from ml_matrics.parity import density_hexbins

1 frames
[/usr/local/lib/python3.7/dist-packages/ml_matrics/__init__.py](https://localhost:8080/#) in <module>()
      1 from .correlation import marchenko_pastur, marchenko_pastur_pdf
      2 from .cumulative import add_dropdown, cum_err, cum_res
----> 3 from .elements import (
      4     count_elements,
      5     hist_elemental_prevalence,

[/usr/local/lib/python3.7/dist-packages/ml_matrics/elements.py](https://localhost:8080/#) in <module>()
      1 from __future__ import annotations
      2 
----> 3 from typing import TYPE_CHECKING, Any, Literal, Sequence
      4 
      5 import matplotlib.pyplot as plt

ImportError: cannot import name 'Literal' from 'typing' (/usr/lib/python3.7/typing.py)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

[sgbaird]

Simple fix suggested here.

Using Literal in all Python versions (1)

Literal was added to typing.py in 3.8, but you can use Literal in older versions anyway.

First install typing_extensions (pip install typing_extensions) and then

from typing_extensions import Literal

This approach is supposed to work also in Python 3.8 and later.

Using Literal in all Python versions (2)

For completeness, I'm also adding the try-except approach to import Literal:

try:
    from typing import Literal
except ImportError:
    from typing_extensions import Literal

This should also work for all Python versions, given that typing_extensions is installed if you're using Python 3.7 or older.

[CompRhys]

There are a few other 3.8+ features janosh likes and uses all the time that make it more than just the type hints to work on 3.7 we tried for a while to support it for me but then gave up

[sgbaird]

@CompRhys ah, good to know. Thanks!

[sgbaird]

I'm also now noticing the Python 3.8+ badge prominently displayed on the README 😅

[janosh]

There are a few other 3.8+ features janosh likes and uses all the time

@CompRhys Just 1 other feature actually which are self-documenting f-strings f"{mystr=}" for which unfortunately there is no Python 3.7 backport.

@sgbaird We had this

try:
    from typing import Literal
except ImportError:
    from typing_extensions import Literal

for a while but since it only solves some of the problem, I got rid of it again.

Supposedly it's possible to set up python 3.8 on Colab. I've never tried it myself, but perhaps you wanna give it a shot. I don't see much point in 3.7 support at this point since numpy, pymatgen, sk-learn, matplotlib, etc. have all dropped it as well.

[sgbaird]

@janosh Thanks for linking the PR. I actually didn't know about self-documenting f-strings until you mentioned it. Very convenient!

Since the packages you mentioned all used to support 3.7, I'd imagine that for most of these Google Colab is installing older compatible versions rather than the latest ones (due to e.g. a python-requires constraint); though this has the downside of no new features. Either way, I use all the packages you mentioned on Google Colab without much issue right now. My use of Google Colab has mostly been in the context of lessening the barrier to materials informatics/coding newcomers, faster/cleaner tests of other people's code, and extending CUDA functionality to those without NVIDIA GPUs. ml-matrics is one I was hoping to use for quick plotting demos for others to run.

The workaround to install python 3.8 via Anaconda might lessen some of that newcomer friendliness, but I'll keep it in mind if there's a more pressing need.

[janosh]

I actually didn't know about self-documenting f-strings until you mentioned it. Very convenient!

Yes, big fan here. 😄

There should be a version of ml_matrics from a while back that runs fine on Google Colab. Though unfortunately since I messed up the python_requires which you just fixed in #18, pip doesn't know which version that is and tries to install the latest.

There are 2 long-standing issues on upgrading the OS on which Colab runs which would also update the Python version it offers:

• Upgrading Colab host environment to Ubuntu LTS 20.04 googlecolab/colabtools#1880
• Update Python version to latest googlecolab/colabtools#2165

Unfortunately, that endeavor seems to have stalled quite badly. Not sure why it's so difficult. You may want to subscribe there for updates.

[sgbaird]

Thanks for the info! I knew it wasn't supported but didn't know why. I might poke around with the older versions and see which one I can install.

[sgbaird]

A workaround for this might be to perform a local installation via cloning and modifying the self-documenting f-strings:

Clone

!git clone https://github.com/janosh/pymatviz.git
%cd pymatviz

Modify f-strings

import os, fnmatch
def findReplace(directory, find, replace, filePattern):
    """https://stackoverflow.com/a/6257321/13697228"""
    for path, dirs, files in os.walk(os.path.abspath(directory)):
        for filename in fnmatch.filter(files, filePattern):
            filepath = os.path.join(path, filename)
            with open(filepath) as f:
                s = f.read()
            s = s.replace(find, replace)
            with open(filepath, "w") as f:
                f.write(s)

# to make compatible with Python 3.7
findReplace("pymatviz", "=}", "}", "*.py")

This seemed to work OK for another use-case (CDVAE on Colab). Will probably try it out next time I'm surfacing an example of some package on Colab and want to use pymatviz.

[janosh]

@sgbaird That's cool. Thanks for reporting this trick. I added a link to your comment to the readme in 65105e9.

[sgbaird]

I'm getting a Pylance warning stating that use of | is only supported for Python 3.10+ at:

pymatviz/pymatviz/ptable.py

Lines 21 to 24 in c4381d3

	if TYPE_CHECKING:
	from typing import TypeAlias
	ElemValues: TypeAlias = dict[str | int, int | float] | pd.Series | Sequence[str]

Alternative syntax for unions requires Python 3.10 or newer

Though the tests, which are on Python 3.8, seem to be working fine.

I guess it's probably because TYPE_CHECKING=False.

[janosh]

If it's just a warning, I think it's safe to ignore. But I'm still surprised. Pylance should know better.

from __future__ import annotations

backports the pipe operator for unions and other modern type-hint syntax all the way to py37.

[sgbaird]

Ah, got it! Thanks @janosh

[sgbaird]

Just in the process of splicing out your nice count_elements function for matbench-genmetrics (Python 3.6+). Of course + a link and the LICENSE

[janosh]

Just in the process of splicing out your nice count_elements function

Glad if it can be useful. 😄

[sgbaird]

Would it be too painful to create a release and version compatible with Python 3.7 via the above script (and with python_requires >= 3.7), revert the changes, and then create an updated release and version with python_requires >=3.8?

So many times I've wanted to use pymatviz on Google Colab 😅 For visualization, I'm using ase instead (for now)

[janosh]

Not much pain i.t.o. workload. Should just take a few minutes. The pain is more psychological. 😄
I'm annoyed with Google for building Colab which is a great product but then completely abandoning it. My usual policy is not to put effort into supporting unmaintained platforms. But heck what are a few minutes...

[sgbaird]

Hmm.. yeah the really old Python version is disappointing. I still find it very convenient for sharing teaching examples, but I think I can understand the psychological pain 😅😭. Thanks for fulfilling my selfish request 🙏