Upgrade to JupyterLab 4

[dlqqq]

Description

Updates the extension to JupyterLab 4. Closes #173.

We are currently debating which release this PR would be included under. This would be released either as 0.11.0 or as 1.0.0.

[dlqqq]

Had a discussion with the team. We have settled on the following versioning strategy:

• Version 1.x will track Jupyter AI on JupyterLab 3 -- we will backport pertinent fixes and features as we deem fit.

• Version 2.x will track Jupyter AI on JupyterLab 4. This will be the main branch and where all development takes place.

I will update the documentation in this PR accordingly.

[dlqqq]

I noticed an error in our user documentation:

You can use conda or pip to install Jupyter AI. If you're using macOS on an
Apple Silicon-based Mac (M1, M1 Pro, M2, etc.), we strongly recommend using
conda.

This is not true. We previously had users install certain dependencies via Conda due to upstream issues with Ray, but Jupyter AI does not have a Conda Forge distribution. You cannot install Jupyter AI with Conda, though you can install it within Conda. This issue is tracked here: #181

I'll update the documentation as I see fit here. @JasonWeill This should require your review.

[dlqqq]

I also took the liberty of rearranging the sections in the user documentation. For some reason, the "Requirements" and "Installation" section were interrupted by the "Model providers" section in between. I found this to be really confusing. I've moved "Model providers" after "Installation" to remedy this.

[dlqqq]

@andrii-i

Did clean install following steps of the potential user and found this part confusing. Following the page from top to bottom, I/user have already done Prerequisites step via pip or conda.

Yes, but you did those prerequisites in a separate Conda environment. If you create and activate a new Conda environment, you need to reinstall JupyterLab.

Following structure would be more clear:

I disagree. That would involve duplicating sections of the documentation, which can easily cause users to be lost in a sea of mostly similar yet slightly different information.

I'd mention the need to make sure appropriate conda environment is active in the same spot conda is first mentioned on the page, in Prerequisites (or conda installation - Prerequisites if structure suggested in my comment above would be used)

We're assuming that they're following the documentation from top-down, i.e. it's guaranteed JupyterLab is already installed in the same Python environment before they try to install Jupyter AI.

[dlqqq]

@JasonWeill @andrii-i Thank you both for your feedback! I've implemented most of @JasonWeill's suggestions and formatted them in the documentation.

[JasonWeill]

Great work on this! One minor additional suggestion, otherwise it looks good to go.

[andrii-i]

Did some manual testing and everything works well. Good job.

Following structure would be more clear:

I disagree. That would involve duplicating sections of the documentation, which can easily cause users to be lost in a sea of mostly similar yet slightly different information.

It's not clear to me which sections of the documentation would be duplicated that are not currently duplicated. At the same time I don't want to bikeshed / block PR based on this so please feel free to merge if you believe it's more clear this way.