Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] Default Modern Theme + complete / linkable built-in TS lib docs. #2335

Open
typhonrt opened this issue Jul 13, 2023 · 21 comments
Open
Labels
discussion Discussion on large effort changes to TypeDoc

Comments

@typhonrt
Copy link

typhonrt commented Jul 13, 2023

I'd like to discuss aspects that are worthy of committing back to the default TypeDoc theme from the recent work that I have released that addresses some general issues. I will include a general announcement below about the resources I have released followed up by aspects that could make it back to the default theme.

The big performance gains are ~90% less disk space utilization and ~80% faster doc generation over the default theme.

I also posted a quick overview video of the resources discussed:
https://www.youtube.com/watch?v=P-TUSPbtLQ0


I have just released a few projects that might help your TypeDoc related usage; particularly for large project documentation generation w/ TypeDoc. I have created the "Default Modern Theme" / "DMT" and its what I call a "theme augmentation" package that functions by augmenting the output of the default TypeDoc theme instead of entirely replacing it. It saves up to ~90% of disk space utilized and is ~80% faster than generating docs w/ the default theme. I also have a few stylistic additions that make things just a bit nicer and increased a11y support with more to come.

Here are the resources to review:
Typedoc Theme (Default Modern Theme)
https://www.npmjs.com/package/@typhonjs-typedoc/typedoc-theme-dmt

Complete and linkable API docs for the TS built-in library declarations + WebGPU / WebCodecs / WebXR APIs. When you generate docs you can add an additional set of TypeDoc plugins to link to every symbol in the declarations to your project docs.
https://www.npmjs.com/package/@typhonjs-typedoc/ts-lib-docs

Jump right to the hosted TS lib docs for an example of large projects generated w/ the DMT:
DOM + WebGPU / WebCodecs / WebXR,
ES2023,
Web Worker APIs.

The TS lib API docs also cross link every symbol against the MDN browser compatibility package and provide links to MDN and relevant specifications integrated in the API docs.

Check out the README and the configuration descriptions available in the links above.

Background:
The main bottleneck with the TypeDoc default theme especially for large projects is the verbose HTML for the left-hand navigation that linearly grows for each page based on the project size and consumes a massive amount of disk space; see this TypeDoc issue. The DMT caches the left-hand navigation HTML and dynamically creates a shared web component that is utilized across all pages only making a single copy of the navigation HTML. This reduces disk space utilization by up to 90% and also makes doc generation ~80% faster. I also include some style additions and replace the main search index generation using compressed MesssagePack instead of JSON which reduces the search index size by more than 90%.

The TS lib DOM API for instance is over 1.4GB with the standard default TypeDoc theme, but w/ the DMT it is 137MB. Well within the size to host on Github Pages.

Being that these resources are new I did "soft-launch" the TS lib API docs using Github Pages for hosting. If things take off and it becomes a popular global resource for documentation generation / linking I will likely have to find a more permanent host. So just a heads up that possibly sometime in the future the hosting location may need to change.

Feedback:
I'd be glad to hear feedback on how I can improve theses resources! My general plan is to try and upstream as much as possible back to the Typedoc default theme over the coming months and continue to add usability features including better search capabilities for large projects to the DMT.

@typhonrt typhonrt added the question Question about functionality label Jul 13, 2023
@typhonrt
Copy link
Author

Not everything is suitable for upstreaming to the default TypeDoc theme, but I think there are areas that can be accepted to improve the default theme. Some of the obvious aspects are style changes to clean up the default theme as that may make sense.

Another procedural one that I'd like to see that is very specific is a better way to handle the inline SVG for the various reflection kinds: namespace, function, etc. The SVG elements that display in the default theme per page. Right now the first usage of the SVG for a particular reflection kind has the full element and all successive usages refer to the original embedding. It would be great to either have all SVG / reflection kinds cached on the page or have them as external resources that can be loaded.

A major reason for such a change is that you can not refer to a reflection kind that is not already in the page due to the way that only the SVG for reflection kinds in the current page is available. For instance I plan on continuing to improve the main search index and offer a new "quick search" UI and it would be nice to show the SVG for any reflection kind found in the respective search UI and that may include reflection kinds not presently loaded on the current page, so externally hosting this SVG is useful.


Anyway.. There is a lot to potentially discuss. The technique or approach I took w/ the "DMT" I think is relatively unique. I'm going more for augmenting the output of the default theme than purely replacing it. The general hope is that I'll be able to keep things synched w/ the default theme as it continues to change and use the DMT as an advanced prototyping stage to then upstream whatever makes sense.

@Gerrit0 Gerrit0 added discussion Discussion on large effort changes to TypeDoc and removed question Question about functionality labels Sep 4, 2023
@Gerrit0
Copy link
Collaborator

Gerrit0 commented Sep 4, 2023

Very delayed response here on my part, apologies for that! You have built something really cool here. I agree with your assessment that at least some of these changes ought to make it into TypeDoc's default theme, and just finished reworking navigation to be built dynamically. With this change, the DOM library is 185mb with TypeDoc's default theme, and I haven't compressed the search index yet!

Incidentally, this required that I require all icons be embedded in each page. I looked at having them in an external page, but unfortunately this seems to make it impossible to style them properly for theme changes, so with 0.25.1, they'll all be there!

@typhonrt
Copy link
Author

No worries on the delay. I'm just ramping up for my next OSS dev cycle Oct-Dec. Definitely appreciate the nod on what I created. The ts-lib-docs effort certainly was a soft launch as I don't have a whole lot of reach in getting the word out about the OSS tooling I release. I posted on the TS Reddit and that contributed to the initial push of ~50 installs, but overall its not visible to the TS / JS community using TypeDoc. I also need to adjust the keywords in the NPM package to get picked up by the TypeDoc website.

I'll definitely spend some time w/ the latest TypeDoc release soon to see any changes and start to evaluate what can be upstreamed from the DMT / theme effort as well.

Incidentally, this required that I require all icons be embedded in each page.

For some of the features I have in the DMT theme I had to do some shuffling of the icon definitions. If all of the icons are present on the page in a separate reference section that can work too.

I think I'll have more input in the second half of October, so definitely would like to circle around and see what is possible to upstream in that general timeframe.

@typhonrt
Copy link
Author

typhonrt commented Dec 13, 2023

@Gerrit0 I spent a lot of time on this one ~160 hours and really honed a lot of things I'd like to upstream into the default theme as it makes sense. Beside the detailed navigation index work one area that I spent a lot of time on is accessibility / keyboard navigation as well as honing styles / CSS to make keyboard navigation more intelligible. Let's continue the discussion as I'm open to actually contribute / upstream what makes sense to do so.


I have just released the next beta of my "Default Modern Theme" (DMT) augmentation for the default TypeDoc theme. This takes full advantage of the new navigation index capability of TypeDoc 0.25.1+. The original DMT release in July spurred the creation of the navigation index for TypeDoc 0.25.x. The latest DMT release is fully ready to go, but I just want to get a little feedback before releasing 0.2.0, so if anyone has the gumption to try it out I'd be glad to get feedback (open an issue on the DMT repo).

package.json:

{
   "devDependencies": {
      "@typhonjs-typedoc/typedoc-theme-dmt": "^0.2.0-next.1",
      "typedoc": "^0.25.0"
   }
}

typedoc.json:

{
  "plugin": [
    "@typhonjs-typedoc/typedoc-theme-dmt"
  ],

  "theme": "default-modern",
}

My main OSS project is putting out a large framework for Svelte. The API docs covers three NPM packages. My framework uses many sub-path exports so without the full navigation tree of the DMT there is like ~65+ modules represented in the default TypeDoc navigation index. The latest DMT vastly improves navigation across the API docs and understandability as well.

You can check out the generated docs for this framework here w/ the latest DMT for an example:
https://typhonjs-fvtt-lib.github.io/api-docs/

The DMT by default creates a full tree for "modules" that are otherwise concatenated in the default navigation index. You have control over the navigation index though and can decide at which depth concatenation starts.

The theme option dmtNavModuleDepth controls at which depth module path concatenation occurs. If you set that to 0 then there is no path concatenation and the output is just like the default theme. Set it to 1 then there is one level before concatenation occurs. You don't have to set this option though as a full navigation tree is generated by default.

There are many other neat features of the DMT and I'll be expanding on documentation and put out a video tutorial in the coming weeks.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Dec 17, 2023

Accessibility and keyboard navigation improvements would be very much appreciated! I am rarely working in the frontend with the exception of this project, so know that I tend to miss things.

The nested navigation by splitting on / in module names is something I've been considering doing for a while too -- just haven't gotten around to implementing it yet. I also want TypeDoc's implementation to turn color/colord into one dropdown in yoru example page, since there's only one item under that entry. I'm not entirely sure it's useful to have as an option -- all of the feedback I've seen regarding that nesting behavior has indicated people want the full structure.

@typhonrt
Copy link
Author

typhonrt commented Dec 18, 2023

Yeah.. Let's try and find a way to actually upstream various aspects of the DMT that are applicable. I'm definitely willing to become a direct contributor. Styles / CSS is relatively low hanging fruit for sure along with potential template modifications. There aren't too many in this category, but a few places that extend flexibility / use of flex box wrapping elements, etc.

Re: navigation index changes. I certainly have the code worked out for modifying the current navigation index generated to provide nesting. There is an option dmtNavModuleDepth that controls at what depth concatenation starts. You can test things out presently by setting dmtNavModuleDepth to 1, 2, 3, etc. This will concatenate paths at that given depth; set it to 0 and the output is the same as the default theme / full paths.

There presently isn't an "auto-concatenate" option to concatenate singular paths like color/colord IE the example you gave from my framework docs presented; that should be an extra option though as it's rather specific. It's definitely possible without too much extra work though; I might take a look at it now.

The only "major" config related HTML structural change w/ the latest DMT is that sidebarLinks and navigationLinks are placed in the navigation sidebar. Full text links in the top header bar doesn't work as well for different screen sizes. I have introduced "icon links" for the header bar. dmtLinksService provides ~5 well used services that have icons ready to go and is easier to configure. Another option dmtLinksIcon allow for user supplied icon links.


The main angle of upstream consideration is potentially switching over to a compile time library / framework. I have a preference for Svelte / use it currently in the DMT. It really does give a bit of extra polish over the homegrown "light component" wrappers IE: src/lib/output/themes/default/assets/typedoc/Component.ts and friends. You may have some strong opinions in this regard. I am also not entirely clear how many other 3rd party themes may use this lightweight API for anything front end.

Regarding the navigation index. In the DMT I make it into a data structure that is stateful and queryable. There are a lot of things that the current default theme implementation misses in this regard. Try loading any URL with a hash fragment for an internal symbol to a given page. The default theme doesn't correctly select the navigation element in this case, etc. So lots of little things are a bit disconnected in the current default theme. The DMT will run a couple of queries with and without the hash fragment to find the correct page / NavigationElement entry to select, etc.

With the forthcoming 0.2.0 DMT release (pre-release now) I have things really optimized in terms of theme / Svelte component bundle file sizes. With the new navigation index I was able to really slim things down as the big workaround of the initial release 0.1.0 / TypeDoc 0.24.x is no longer necessary.

I think it is good to have a comparison w/ the DMT regarding polish / final fit and finish as a way to show where the default theme can go. Since it is an augmentation of the default theme output vs a replacement the DMT provides a bit of a clearer picture on where things can potentially be improved.


Ooo.. I also have something spicy incoming shortly for an automatic CLI for doc generation from package.json exports / configuration. I'll make a post on that issue when it's out.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Dec 27, 2023

You may have some strong opinions in this regard

I am not a frontend enjoyer, I have really only one hard rule for the default theme.

The site shall work without JavaScript. It might not be perfect (search, filter, full navigation tree), but it will at least be readable. Frontend frameworks seem to all be of the "render everything" opinion rather than being built for progressive enhancement.

A second goal is that adjusting something in the theme should not require understanding a massive framework. Everything I've used (Svelte isn't on that list) has always had annoying edge cases to fall into that end up compounding the DOM's weirdness...

@typhonrt
Copy link
Author

typhonrt commented Dec 27, 2023

I am not a frontend enjoyer, I have really only one hard rule for the default theme. The site shall work without JavaScript. It might not be perfect (search, filter, full navigation tree), but it will at least be readable. Frontend frameworks seem to all be of the "render everything" opinion rather than being built for progressive enhancement.

So, I got everything working again w/ no script loading for the DMT.

I'm sure you have noticed with the default theme the visual flash and inconsistent state of the UI as the additional default theme JS is loaded. This is very noticeable w/ the "settings" / "on this page" accordions depending on state (if they are closed) and the navigation tree will flash and readjust as the default theme JS loads which can be very jarring depending on the state and how large the navigation tree is.

The same will happen w/ loading Svelte components in regard to enhancement just after the initial page loads. To get around this visual glitch I add additional style rules to head to make body not visible and flip that state after the Svelte component bundle fully loads. This ensures that both main.js and the loaded Svelte components set state of the UI before showing which is far less distracting.

I have made the adjustment so that there is an additional noscript style rule added that will do this reversal when JS is turned off in the browser. So, if you tried things w/ Javascript disabled and you get a blank screen this is why and it is nothing to do w/ Svelte, but is the workaround I put in place to reduce visual artifacts of changing the UI state after initial page load. The DMT 0.2.0-next.5 release will have this solved for no script usage.

So the DMT loads fine without Javascript enabled. The same minimal HTML navigation tree output by the default theme is what you get.


On to your second point. Just like the bespoke loading of JS w/ the default theme "lightweight component" API my usage of Svelte is exactly the same where I just target the navigation index and search components. It just uses Svelte instead of the bespoke mechanism of the default theme. Folks that want to modify the default theme will have to understand your bespoke API regardless, so it's not a standard per se. I think it would be a minor nightmare to reproduce the UX / interactivity and accessibility support that the Svelte navigation component brings to the table with the default theme "lightweight component" API. Using Svelte doesn't mean the whole page is rendered w/ Svelte. It is surgical in implementation and the nice thing about Svelte being a "compile-time" library / framework is that a single all contained bundle is produced for loading; IE no additional runtime to load (Vue / React, etc.).

Do you know of any themes that continue to use the "lightweight component" API that you built for the default theme? My guess is that most replace those aspects with other tech or don't touch it at all.

I think a good way to evaluate the DMT is whether one feels the UX and features improve on the default theme regardless of the underlying tech.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Dec 27, 2023

I haven't kept track of what themes people have built besides making sure appropriately tagged ones show up on https://typedoc.org/guides/themes/

Some thoughts, in no particular order, I haven't played with it much on desktop yet:

  • I like the buttons to expand/collapse the nav tree, and the keyboard shortcuts docs
  • the guide lines in the nav tree are really helpful in deeply nested hierarchies
  • the search box zooms me into the page when on iOS
  • links to members within the page result in the member being partially scrolled behind the title bar (default theme tries to handle this, doesn't do it great)
  • There are a lot of boxes in the theme now, which feels rather overwhelming when looking at functions which accept callbacks, it feels kind of like TypeDoc's current styling has been half-replaced. I'm far from a UI expert, but one thing I've internalized is avoiding unnecessary change -- is there a reason to have all these extra visible layers? https://typhonjs-fvtt-lib.github.io/api-docs/types/svelte_store.StartStopNotifier.html
  • It would be nice for the animation to default to off for anyone who has their computer set to reduce motion. Personally, I almost never want motion.
  • I often can't easily see the bottom inch or so of the screen due to it being hidden behind the safari navigation bar, even when scrolling down. With the default theme, safari is able to tell you're scrolling and collapse the nav bar.
  • The custom icons are still clipped behind the search on very small screens, they should probably show up on the sidebar
  • The guide lines in the navigation are hard to see in dark mode
  • The keyboard shortcuts feel strange to me - likely due to just not being used to them... however few shortcuts I use commonly include Alt. Are these based on some common set from something? GitHub and StackOverflow both went for a go to X, gX style, TypeDoc's only current shortcut is / to open search from Less/Vim...

@typhonrt
Copy link
Author

typhonrt commented Jan 13, 2024

So the latest DMT is out 0.2.0-next.10. Quite likely it may be the final beta before the next official release. I have just released @typhonjs-typedoc/typedoc-pkg which uses the DMT, and provides a CLI / robust way to automatically generate docs from package.json with correct export name / module path replacement. typedoc-pkg is zero configuration yet also has various extended options to easily automate documentation generation for well configured packages.

The major performance win in the latest DMT release is that the SVG icons cache is stored as an external file and the per page cache is removed which saves 15KB per page which really can add up with large documentation projects (20-30%) of file space.
If I can in ~40 lines of code augment the output of the default theme then it should be relatively easy to make the default theme reference an external SVG file.


Responses to the previous post:

I definitely want to reiterate that the DMT is not in a final state in respect to potential upstreaming nor would it potentially just be adopted. It is an attempt to deliver maximum value w/ minimal structural changes. A few things that you listed would require more involved structural changes that is more appropriate in the theme replacement category than augmentation. The DMT is more of a guide of where significant updates can occur with the default theme.

I am more than willing to work on a full upstream effort to incorporate these changes in the default theme.

Some thoughts, in no particular order, I haven't played with it much on desktop yet:

  • the search box zooms me into the page when on iOS

Definitely a Safari / iOS thing. I increased the font-size to 16px which should stop this default behavior on iOS. I gather the default theme does the same as there is no font-size set.

  • links to members within the page result in the member being partially scrolled behind the title bar (default theme tries to handle this, doesn't do it great)

If you can provide a bit more information / screen shot that can help in this case. It is not clear exactly what you are referring to.

  • There are a lot of boxes in the theme now, which feels rather overwhelming when looking at functions which accept callbacks, it feels kind of like TypeDoc's current styling has been half-replaced. I'm far from a UI expert, but one thing I've internalized is avoiding unnecessary change -- is there a reason to have all these extra visible layers? https://typhonjs-fvtt-lib.github.io/api-docs/types/svelte_store.StartStopNotifier.html

Just like the navigation guide lines the additional parameter list overlaid borders / background shows depth so you can easily tell which parameter list is being viewed quickly. Granted this is a subjective change, but the grouping / borders helps quite a bit when scrolling through large files with lots of methods.

  • It would be nice for the animation to default to off for anyone who has their computer set to reduce motion. Personally, I almost never want motion.

This is now enabled. Clear local storage and you'll see that the animation setting will take the OS reduce motion setting initially.

  • I often can't easily see the bottom inch or so of the screen due to it being hidden behind the safari navigation bar, even when scrolling down. With the default theme, safari is able to tell you're scrolling and collapse the nav bar.

Yeah... The change is that the main content div is now scrollable versus the whole page. This does favor the desktop display in this case, but this change also significantly improves the left hand navigation which is a bit "wonky" w/ the default theme; not being pejorative there. You can also on mobile devices flick up / down the docs title bar to hide the mobile browser URL bar.

  • The custom icons are still clipped behind the search on very small screens, they should probably show up on the sidebar

This is one of those augmentation vs replacement issues. The main problem with the default theme is the absolute positioning of the search button and overflow mobile menu. In a more replacement oriented solution the search and mobile overflow buttons would be part of a flex box in the title bar.

If you really want to ensure folks w/ less than ~420px wide devices have no problems then a custom icon fly-out Svelte component is the best solution. IE once there is not enough space for all the icons they are condensed down into a fly-out overflow menu. Again that really would only come into play with folks reading documentation on their smartwatch.. :O

Presently the navigationLinks of the default theme completely overlap the document title when the browser width is reduced.

  • The guide lines in the navigation are hard to see in dark mode

I made the default brighter. A CSS variable is available to change it further.

  • The keyboard shortcuts feel strange to me - likely due to just not being used to them... however few shortcuts I use commonly include Alt. Are these based on some common set from something? GitHub and StackOverflow both went for a go to X, gX style, TypeDoc's only current shortcut is / to open search from Less/Vim...

Alt / Option is chosen for the activation because it doesn't conflict with browser shortcuts which use Ctrl / Meta. While some of the shortcuts are "go to X" oriented holding down Alt while opening / closing navigation folders is not. This is pretty subjective and I'd want to receive feedback from actual keyboard bound users. They exist for accessibility reasons. I'm not opposed to switching the to the gX style.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Jan 15, 2024

I have just released @typhonjs-typedoc/typedoc-pkg

Neat! The exports handling sounds like exactly what I want to include in TypeDoc someday.

The major performance win in the latest DMT release is that the SVG icons cache is stored as an external file and the per page cache is removed which saves 15KB per page which really can add up with large documentation projects (20-30%) of file space.
If I can in ~40 lines of code augment the output of the default theme then it should be relatively easy to make the default theme reference an external SVG file.

I've wanted to do this for a while, didn't realize it was so easy to do! Going to steal that right now..
Edit: Ick. Turns out this only works if serving from a web server. If loading a local file browsers block it... I really don't want to lose that, I use local files very heavily during development... but it's a pretty major savings... going to think about this one.

I gather the default theme does the same as there is no font-size set.

Surprisingly, it doesn't... frontend is a mystery to me. Potentially due to the font-size on body?

If you can provide a bit more information / screen shot that can help in this case. It is not clear exactly what you are referring to.

This appears to be another safari specific thing, I can't reproduce on desktop - visiting https://typhonjs-fvtt-lib.github.io/api-docs/classes/_runtime_util.Hashing.html#hashCode results in this:

(Icons inside the circles are consistently off center as shown in the image as well, same for the dropdown expand/collapse)

Image

This is now enabled. Clear local storage and you'll see that the animation setting will take the OS reduce motion setting initially.

👍 very nice

Just like the navigation guide lines the additional parameter list overlaid borders / background shows depth so you can easily tell which parameter list is being viewed quickly. Granted this is a subjective change, but the grouping / borders helps quite a bit when scrolling through large files with lots of methods.

Having played with it some more, I'm coming around to this change, it still feels a bit noisy when dealing with callbacks, but for the majority of members, I definitely like it.

Yeah... The change is that the main content div is now scrollable versus the whole page. This does favor the desktop display in this case, but this change also significantly improves the left hand navigation which is a bit "wonky" w/ the default theme; not being pejorative there. You can also on mobile devices flick up / down the docs title bar to hide the mobile browser URL bar.

Very unfortunate if that's a necessary change... It's fine on most pages, until I get to one where there's just enough content to hide content behind the navigation bar (ComponentProps on my phone), then the page gets quite wonky.

What wonk are you referring to in the default theme? I thought I'd brute forced my way through all the issues there.

If you really want to ensure folks w/ less than ~420px wide devices have no problems then a custom icon fly-out Svelte component is the best solution. IE once there is not enough space for all the icons they are condensed down into a fly-out overflow menu. Again that really would only come into play with folks reading documentation on their smartwatch.. :O

My non-massive phone is 375px wide according to Firefox devtools, it's not too uncommon for me to use it for a quick reference if talking to someone on Discord. Definitely agree that the default theme does no better, and in many ways worse.

I made the default brighter. A CSS variable is available to change it further.

Much easier to see 👍

Alt / Option is chosen for the activation because it doesn't conflict with browser shortcuts which use Ctrl / Meta. While some of the shortcuts are "go to X" oriented holding down Alt while opening / closing navigation folders is not. This is pretty subjective and I'd want to receive feedback from actual keyboard bound users. They exist for accessibility reasons. I'm not opposed to switching the to the gX style.

Definitely makes sense. Having played with it a bit more, I don't mind the alt based shortcuts, though it isn't really true to say that because it uses Alt, it won't conflict - Alt+T+I opens Firefox's Page Information dialog.

Esc should definitely close the search bar. I especially like Alt+C, though it makes me wonder if the "reflection summary" interface block on pages like this one should have an anchor to skip them.


Now, the elephant:

TypeDoc's default theme really hasn't changed structurally in any significant way since I started working on it ~6 years ago. The two biggest changes were moving from handlebars to JSX (which didn't actually change the output structure, just made it much easier to detect type errors) and a styling rework which made browsing on mobile much better, and fixed a lot of accessibility problems. Is it time for that to change?

You're clearly passionate about making docs better, which is awesome. If you think it is time that TypeDoc's default theme was rebuilt, and want to lead that effort, I'd be happy to add you as a contributor and provide what help I can, we'd definitely want to do this as a part of a minor release.

@typhonrt
Copy link
Author

Neat! The exports handling sounds like exactly what I want to include in TypeDoc someday.

Re: typedoc-pkg. I think the best approach for working out something like this w/ TypeDoc is to have two binaries and completely separate the logic of the package.json targeted CLI. typedoc-pkg can easily be upstreamed into TypeDoc. Granted I have a few extra features like connecting the ts-lib-docs API linking. In an ideal future scenario it would be great to make the ts-lib-docs an official resource and making it widely available / easy to hook up via TypeDoc.

The major performance win in the latest DMT release is that the SVG icons cache is stored as an external file and the per page cache is removed which saves 15KB per page which really can add up with large documentation projects (20-30%) of file space.
I've wanted to do this for a while, didn't realize it was so easy to do! Going to steal that right now.. Edit: Ick. Turns out this only works if serving from a web server. If loading a local file browsers block it... I really don't want to lose that, I use local files very heavily during development... but it's a pretty major savings... going to think about this one.

Indeed I missed that aspect during development as I use WebStorm and it runs a local web server behind the scene when you open HTML files directly. Upon investigation it indeed is a run of the mill CORS issue and there doesn't appear to be a work around for opening the docs locally without a web server when externally referencing the SVG icons. The good news is something like this can be optional and is easy to turn on / off at least w/ the DMT.

Also these days it is easy to configure a local web server. That could be incorporated into the typedoc binary itself or simple instructions to use a web server locally. I really like sirv for embedded use cases and it also has an easy to use CLI sirv-cli. This results in a NPM script: "sirv docs --host" to run a local web server.

I think it is important to consider the deployment aspects of generated documentation and prioritize production use cases over broad "it has to work everywhere no exceptions" mindset. With a flag / option to externalize the the SVG icons this allows production use cases ending up w/ ~20-30% less disk space and network bandwidth which are important cost factors especially with large documentation efforts.

This appears to be another safari specific thing, I can't reproduce on desktop - visiting https://typhonjs-fvtt-lib.github.io/api-docs/classes/_runtime_util.Hashing.html#hashCode results in this:

(Icons inside the circles are consistently off center as shown in the image as well, same for the dropdown expand/collapse)

Image

It's certainly good that you have a small form factor device to test with. The solution here is to redesign the structural layout such that there is the fly-out icon component that can reduce the icons to a single button in addition to making the search and overflow mobile menu a part of the the flex box layout added by the DMT. Also when doing this the documentation title can also use overflow ellipses such that the any fly-out icon component, search button, and mobile overflow button is always visible and if the documentation title is abbreviated via overflow that is just the situation with a small form factor device.

The present state of the DMT is still an interim solution where I'm not as heavy handed in changing the structural aspects of the absolute positioned search / mobile overflow buttons. I'll look into creating the above better solution before I launch DMT 0.2.0 in a couple of weeks.

Having played with it some more, I'm coming around to this change, it still feels a bit noisy when dealing with callbacks, but for the majority of members, I definitely like it.

👍

Yeah... The change is that the main content div is now scrollable versus the whole page. This does favor the desktop display in this case, but this change also significantly improves the left hand navigation which is a bit "wonky" w/ the default theme; not being pejorative there. You can also on mobile devices flick up / down the docs title bar to hide the mobile browser URL bar.

Very unfortunate if that's a necessary change... It's fine on most pages, until I get to one where there's just enough content to hide content behind the navigation bar (ComponentProps on my phone), then the page gets quite wonky.

What wonk are you referring to in the default theme? I thought I'd brute forced my way through all the issues there.

The DMT implementation is a subjective change to make things cleaner. I have a general idea what you are trying to work around in regard to the browser URL title / popup in the bottom left corner. I haven't used Safari much, but for instance Chrome moves this popup away when the mouse navigates toward the bottom of the screen. The work around of making the left hand navigation floating and then having extra space at the bottom after scrolling down is a work around for a particular browser that you use.

Here is what Chrome looks like in regard to moving the mouse / cursor over the URL popup at the left hand bottom of navigation. You can see that it moves this popup down / away from the cursor:

chrome-url-popup.mp4

My non-massive phone is 375px wide according to Firefox devtools, it's not too uncommon for me to use it for a quick reference if talking to someone on Discord. Definitely agree that the default theme does no better, and in many ways worse.

It's definitely worth working out the aspects commented upon from the main header toolbar to other use cases where small form factor is a consideration. Assuming everyone has modern phone is not globally ideal either. I believe that the forthcoming additional efforts mentioned above will get things there.

Definitely makes sense. Having played with it a bit more, I don't mind the alt based shortcuts, though it isn't really true to say that because it uses Alt, it won't conflict - Alt+T+I opens Firefox's Page Information dialog.

And it doesn't. Alt does open the Firefox "files" / top menubar, but doesn't preclude the use of Alt + T + I. I'm amenable to finding the best most compatible solution though it's pretty good right now.

Esc should definitely close the search bar. I especially like Alt+C, though it makes me wonder if the "reflection summary" interface block on pages like this one should have an anchor to skip them.

I don't think that is a problem as the breadcrumb header gets focus on that particular page. Also follow up tab navigation works through the interface block where they can be selected. So no particular need to skip them.

Now, the elephant:
TypeDoc's default theme really hasn't changed structurally in any significant way since I started working on it ~6 years ago. The two biggest changes were moving from handlebars to JSX (which didn't actually change the output structure, just made it much easier to detect type errors) and a styling rework which made browsing on mobile much better, and fixed a lot of accessibility problems. Is it time for that to change?

I don't think so when it comes to static HTML generation. When I mentioned structural changes I was referring to HTML structural aspects where a few more specific changes to the HTML structural output will improve solutions like the top header bar / buttons / icons. I didn't make these kinds of more involved changes to keep the changes I do make to ~300 lines of code and a bit more resilient. In the upstreaming process I'd convert the bespoke modifications that I do in the DMT to .tsx / implementation that the default theme uses.

The aspect of swapping out the bespoke scripting API ala src/lib/output/themes/default/assets/typedoc/Component.ts and such for Svelte based components is limited to just the default theme implementation. This API can always remain just in case other themes happen to use it, but likely could be removed.

You're clearly passionate about making docs better, which is awesome. If you think it is time that TypeDoc's default theme was rebuilt, and want to lead that effort, I'd be happy to add you as a contributor and provide what help I can, we'd definitely want to do this as a part of a minor release.

I'm definitely open to contributing, but also doing it in a proper / controlled fashion with lots of documentation. From a timing perspective I think targeting something like this for the end of '24 is best. This gives ample time to continue iterating on the DMT to really fine tune it and in many respects it'll be known exactly what may change and what the results will be before upstreaming. Svelte 5 is soon to be released, so I'll be updating the DMT to use it in the coming months, so this also fits well w/ an end of '24 timeline. I also have my large OSS projects that I need to get back to servicing. I spent the last two months in the TypeDoc / tooling space and developers using my other projects are waiting for a release or two.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Jan 20, 2024

I think the best approach for working out something like this w/ TypeDoc is to have two binaries and completely separate the logic of the package.json targeted CLI

Why two binaries? I was thinking that TypeDoc would use the package.json handling by default if no entry points were provided manually by the user.

The good news is something like this can be optional

True, I don't love needing an option here, might experiment with adding a script that's only loaded for file:// urls which injects it into the document and rewrites <use> elements.

I have a general idea what you are trying to work around in regard to the browser URL title / popup in the bottom left corner.

I wish this is what I was working around! The smaller nav size there is actually to keep the entire scrollbar on the screen since it doesn't start directly adjacent to the header bar

Safari on mobile is unfortunately very annoying to optimize for, the entire url bar + navigation buttons at the bottom of the screen in the image I sent earlier will collapse if the page is large enough to scroll down, there's a specific page size for which this doesn't work on the DMT.

When I mentioned structural changes I was referring to HTML structural aspects where a few more specific changes to the HTML structural output will improve solutions like the top header bar / buttons / icons.

This is exactly what I meant by structural changes, apologies, that was rather vague.

From a timing perspective I think targeting something like this for the end of '24 is best

👍 sounds good, gives me some time to try to improve the theming code to be more understandable in the meantime.

@typhonrt
Copy link
Author

typhonrt commented Jan 21, 2024

Why two binaries? I was thinking that TypeDoc would use the package.json handling by default if no entry points were provided manually by the user.

A separate CLI... Allows flexibility for changing environments; while Node / package.json based configuration is still dominant today that may not always be the case. Less user confusion. A dedicated CLI streamlines the user experience with a clear work flow. Grabbing entry points from a local package.json is what it does by default, but there are more options to expose. Like would you add additional CLI options to the main CLI for designating a mono-repo to load? What about alternate export conditions? Being able to silo documentation for a specific use case makes things easier to communicate. A separation of the codebase follows a single responsibility reduction in maintenance burdens where additional maintainers can take ownership.

There are alternate solutions like turning the TypeDoc CLI into a multi-command CLI which is better than stuffing too many competing flags / options into a single use CLI.

Re: typedoc-pkg; it features a simplified programmatic API which the CLI uses that is far simpler than what is required under the hood to manipulate the main TypeDoc programmatic API. Something like this can be facilitated w/ a sub-path export for TypeDoc.

True, I don't love needing an option here, might experiment with adding a script that's only loaded for file:// urls which injects it into the document and rewrites <use> elements.

Experimentation is good of course, but that is a pretty big commitment to deploy. I don't think it's as much of a concern though over other features / maintenance, but gathering some metrics in that regard is useful. A flag / option for enabling offline usage is easier with production deployment being the default. It should be possible to gracefully fail and show informational resources regarding a given production doc deployment when opened locally.

Indeed the main "problem" with the DMT is ESM deployment. type="module" scripts don't load locally from file:/// URLs. I also use fetch to load the search data. Granted a lot of that can be possibly be worked around, but I do think it's reasonable to consider 2024 environments and just ship capability for an easy to use web server for local viewing of docs as necessary and adopt modern dev practices. That's probably a strong opinion, but personally I lean toward adopting modern tech & practices. I'll definitely consider options as the year progresses.

I wish this is what I was working around! The smaller nav size there is actually to keep the entire scrollbar on the screen since it doesn't start directly adjacent to the header bar

Safari on mobile is unfortunately very annoying to optimize for, the entire url bar + navigation buttons at the bottom of the screen in the image I sent earlier will collapse if the page is large enough to scroll down, there's a specific page size for which this doesn't work on the DMT.

Not having the same device you do gives me a bit of a handicap in discussing this aspect. The DMT is fine on any standard / modern phone & tablet (Android tested on my side w/ recent PIxel tablet and a Pixel 4 phone). Optimizing things for your specific small form factor mobile device / OS over the desktop environment is what it is. I'd say though that it's more important to optimize the desktop experience. The DMT works well on 4K monitors vs the default theme for instance.

I'll give it a good shot in fixing up all the final tweaks I mentioned above which should alleviate most small form factor concerns. I'm not opposed to picking up a used iOS device that is similar to what you are working with...

From a timing perspective I think targeting something like this for the end of '24 is best
👍 sounds good, gives me some time to try to improve the theming code to be more understandable in the meantime.

Yeah, let's stay in touch. I'd be glad to upstream everything that makes sense as there is no particular reason to recreate the wheel.

I'll probably open another discussion thread about subtleties detected in recent TypeDoc releases. For instance the full hierarchy view crashes with ts-lib-docs for the DOM API (~2600 pages). It might be good to test that on a complex / large codebase for instance.

@typhonrt
Copy link
Author

Hiya @Gerrit0...

I have updated my TypeDoc theme (DMT) and plugins for the latest 0.26.x.

In brief ts-lib-docs is updated for all TS 5.6.2 built-in lib declarations. The DMT saw many upgrades taking into account layout changes in TypeDoc, refinement of new features added, and hardening of all of the frontend code.

I do have a lot of critical feedback on various aspects since we last chatted regarding perceived UX issues with new features as currently implemented in the mainline default theme, but also other internal API changes that affect 3rd party developers; some of the latter not even receiving a mention in release notes. I'd be glad to continue discussing these issues if you are interested as well as a possible pathway to mainline some of the DMT features in the future. While there are aspects that could be inspiring to take and implement within TypeDoc I'd really like to find out if there is a way to unify particularly my frontend dynamic work into the mainline release.


Perhaps to start some video demos of various UX refinements to new features. In particular the separation of additional Markdown documents into a unique navigation tree and synchronized document content folding between the main content area and On This Page sections. I think it is really important to separate the additional Markdown documents and leave the source code navigation tree 100% just covering the source code at hand. I currently have a large PR refactor pending for a gl-matrix proposed new release, but generated a quick peak comparison between the DMT and the default theme with this project. This will show the refinement the DMT adds to these features:
https://www.youtube.com/watch?v=MFJW6FxfvCg


This demo video shows improvement to the DMT icon links where if there is not enough space (mobile vertical orientation) they will collapse into a hamburger menu:
https://www.youtube.com/watch?v=P_ayXxwhj0s


You can view hosted generated docs with the latest DMT here with my proposed gl-matrix PR docs and my main large framework API:

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Sep 24, 2024

but also other internal API changes that affect 3rd party developers; some of the latter not even receiving a mention in release notes.

This doesn't surprise me too much, it is entirely too easy for people to end up using parts of TypeDoc which are not considered part of the public API, and with the HTML output in particular, every small change has the potential to break things in ways that can't be caught by a type checker. Documenting every change to internal functions isn't feasible... if there are specific instances you think should have been documented, I'm not opposed to including addition entries in the changelog/releases, though updating the releases is an unfortunately manual process.

I think it is really important to separate the additional Markdown documents and leave the source code navigation tree 100% just covering the source code at hand.

For projects which have top level project documents and an API package, I'm leaning towards agreeing with you. However, this does not cover all documents! That feature very intentionally allows users to include markdown documents at lower level areas, for example TypeDoc's Internationalization namespace also includes a related document. How does the DMT handle this?

Similarly, if you have documentation for more than one package included in one site, each package might have documents specific to that package.

Something I should mention is that your typedoc default theme output has the unfortunate consequence of my forgetting to set the default sort option to put documents first, perhaps not as ideal as fully separating them, but it can be better. (The default is already changed on the beta branch)

synchronized document content folding between the main content area and On This Page sections

I considered doing this when adding the folding, and decided against doing it, it seemed confusing to me at the time... having played with your update for 20 minutes or so now though, I quite like it. Definitely something that'll come to the default theme.

This demo video shows improvement to the DMT icon links where if there is not enough space (mobile vertical orientation) they will collapse into a hamburger menu:

Very nice, this is exactly the type of tweak that I have absolutely no desire to add myself, but greatly appreciate when someone else makes it happen.


Upstreaming:

There are a fairly small number of things that'd need to change for me to be happy to just adopt the DMT as TypeDoc's default, assuming that the handling of non-toplevel documents is reasonable. Parts of it have really grown on me, including the keyboard shortcuts being awesome. I also really like the inclusion of the class hierarchy in the main navigation as a link.

  1. Update the search to work properly without a http server (I know it isn't a big deal for your use case, but the one extra step gets awfully annoying for me)
  2. Fix handling of narrator when tabbing through search results, it currently doesn't properly announce them, likely due to the entries not actually being selected? Fixing this was a pain in the current default theme.
  3. (Already done?) Make sure all strings in the frontend can be localized
  4. ...that's really it.

There are a few minor things that still feel somewhat off about the DMT to me, but I'm not really opposed to, including:

  1. The Settings/On This Page boxes have a fuzzy drop shadow while nothing else does
  2. It feels unfortunate to lose the "On this page" section when on the two column layout, though I suppose on a phone people will just end up searching in most cases anyways....

Another thing you might like, I'm considering a different rendering strategy for modules in 0.27:

Re-exports have the arrows, and the first paragraph (or @summary block) of comments are included in the module page for members of a module.

image

@typhonrt
Copy link
Author

but also other internal API changes that affect 3rd party developers; some of the latter not even receiving a mention in release notes.
This doesn't surprise me too much, it is entirely too easy for people to end up using parts of TypeDoc which are not considered part of the public API

So, from previous conversation you mentioned you have no visibility into what 3rd party developers are doing and this is a general problem. The 0.25.x to 0.26.x release cycle was rather poor in this regard. You made sweeping changes to the internal API surface between the last beta.5 release and the official 0.26.0 release that likely broke every 3rd party devs plugins giving no time to correct these issues or provide meaningful feedback in advance of 0.26.x. I checked in at beta.5 and confirmed things were working and did so at that time because of the Typescript 5.5 release and we communicated about some TS 5.5 aspects to take into consideration. When I got around to updating the DMT after 0.26.x dropped things were broken in unexpected ways. Changing the event emitter API was not clearly communicated in release notes. Rather that API: perhaps use Breaking API: to more clearly communicate a change that is well breaking. While annoying it wasn't too bad to change usage for the new event emitter API.

In fact that particular commit, Typed event emitters, in between beta.5 and 0.26.0 contained way more API changes than just the event emitter. In particular the deletion of removeComponent from ChildableComponent in component.ts seems entirely superfluous serving no purpose whatsoever and taking away capabilities from 3rd party devs. This was an API surface long available and the removal is bad and limits 3rd party plugins like the DMT when it comes to disabling components from the default theme like the navigation / search index generation which the DMT handles in a different manner. The DMT can't remove those components from the default theme and now simply deletes the files generated by those components while also running DMT navigation and search index creation separately.

Something that will help in these matters is for you to make pre-release releases in GitHub for all beta releases. I monitor the release feed of TypeDoc. It's not as easy to monitor releases made "silently" through NPM. You can use the Releases section of GitHub for all beta releases and mark them as pre-releases. This will send out notifications to those subscribed to the release feed. Furthermore IMHO it makes sense to make rc / release candidate releases and mark them as unique compared to beta.x releases. This signals to 3rd parties that a new official major is incoming and allows folks to verify and provide feedback before the major goes out. There was no indication that beta.5 was the last before 0.26.0 and you made tons of sweeping changes that just dropped out of the blue in 0.26.0 breaking the entire 3rd party ecosystem without any ability for 3rd parties to provide meaningful feedback in advance.

Not condemning here. Just pointing out there are options to make beta releases and potential release candidate releases and make them trackable / visible through GitHub as pre-releases.

, and with the HTML output in particular, every small change has the potential to break things in ways that can't be caught by a type checker.

Yes, it took me 5 weeks! fulltime to update the DMT to 0.26.x. Not every 3rd party developer will put in such effort. Now granted I have to deal and improve / augment many of the layout changes you made. I also spent a lot of time hardening the frontend Svelte components and additionally adding aspects like the dynamic icon links. The DMT is somewhat of a special case as a 3rd party plugin because I am handling all cases / new features in response to changes in the default theme which is time intensive. I'm doing this in order to stay in sync and make any official mainlining of the DMT "easier" / more acceptable.

I think it is really important to separate the additional Markdown documents and leave the source code navigation tree 100% just covering the source code at hand.
For projects which have top level project documents and an API package, I'm leaning towards agreeing with you. However, this does not cover all documents! That feature very intentionally allows users to include markdown documents at lower level areas, for example TypeDoc's Internationalization namespace also includes a related document. How does the DMT handle this?

Often your changes and additions seem a little scattershot. Less is more and you'll certainly hear me repeat this at the end with your additional proposed changes below. The DMT completely handles the NavigationElement data structure and supports the @document feature. It's easy to duplicate usage of the same component and feed it a different root node. IMHO the @document feature is not fully baked. If you apply it to non-navigation index reflections (say methods in a class) it doesn't work and produces garbage output. Rather than a completely new @document why didn't you just repurpose and use @see ../markdown/somefile.md? Also every time you use @document it creates a unique version of that markdown file potentially duplicating it many times if @document is used multiple times. What I'm getting at is that you tried to do too much with this update where just adding Markdown files ideally in a separate location more than sufficed and the additional @document feature didn't receive as much attention as it should have to be solid.

Similarly, if you have documentation for more than one package included in one site, each package might have documents specific to that package.

One can use categories in the Markdown frontmatter to separate them into logical sections for different packages and the like. Now granted it would be really handy with a separate document index section like in the DMT to enable categories just for the Markdown documents and have the option to apply it to the source code index as well and not always both.

Something I should mention is that your typedoc default theme output has the unfortunate consequence of my forgetting to set the default sort option to put documents first, perhaps not as ideal as fully separating them, but it can be better. (The default is already changed on the beta branch)

IMHO having a complete separation in the GUI that doesn't alter the source navigation index particularly fabricating a module where non-exists is really important. I get that this was the "easiest" way for you to implement the new feature by just stuffing it into a single navigation tree, but it's not good UX and obscures package / source code structure. Various options for the single navigation index for categories and groups makes it much worse. The DMT handles all permutations and cleanly separates the markdown documents as well as removing the fabricated index module when it is generated.

synchronized document content folding between the main content area and On This Page sections

I considered doing this when adding the folding, and decided against doing it, it seemed confusing to me at the time... having played with your update for 20 minutes or so now though, I quite like it. Definitely something that'll come to the default theme.

Please try and implement that in the Accordion component. I completely disable to Accordion component from the default theme and take full control of all accordions / details elements.

This demo video shows improvement to the DMT icon links where if there is not enough space (mobile vertical orientation) they will collapse into a hamburger menu:

Very nice, this is exactly the type of tweak that I have absolutely no desire to add myself, but greatly appreciate when someone else makes it happen.

It's easy on my side because of Svelte and I have a larger library of utility functions / Svelte actions that make it streamlined to hookup ResizeObserver and such. Still a unique problem though in pre-calculating the image widths.

Upstreaming:
There are a fairly small number of things that'd need to change for me to be happy to just adopt the DMT as TypeDoc's default, assuming that the handling of non-toplevel documents is reasonable. Parts of it have really grown on me, including the keyboard shortcuts being awesome. I also really like the inclusion of the class hierarchy in the main navigation as a link.

I appreciate the nod. Hopefully my comments don't come across as too critical. If they seem charged it's just because I have spent ~4 months in the last ~1.75 years working explicitly on TypeDoc related efforts instead of my actual projects that really matter and are being underserved as a result with devs waiting / not happy generally. I've spent over $20k of my own funds (IE keeping the lights on / bills paid) in this time effectively in this effort. I am glad though that we might move forward with mainlining and making all that effort turn out positively.

  1. Update the search to work properly without a http server (I know it isn't a big deal for your use case, but the one extra step gets awfully annoying for me)

Just take a moment to understand that this is a side effect of using modern ES Module loading. ESM requires for increased security purposes that a web server is used. I have provided clear instructions on several easy ways to handle this situation in a wiki post.

I gather you don't use a modern IDE like WebStorm or VSCode which has built-in web servers and the docs can instantly be opened directly from the IDE. Please take note that it is super easy to start a static web server given that one already is working with Node / NPM for TypeDoc in the first place. You just need a one line script like:

{
  "scripts": {
    "sirv": "npx sirv-cli docs --host"
  }
}

npm run sirv from the command line or IDE that doesn't have a built-in web server from there and no problems. It's not groundbreaking and just how modern Javascript technology works. 20 years ago API docs might have been distributed on CD / email what have you where opening a local file was handy, but times have changed. Embracing the change results in a more sound product.

I'm strongly in the ESM first / work with modern technology camp. That doesn't mean I can't find a way to work around and generate an IIFE for the DMT, but at some point it's just a good idea to adopt modern technology.

  1. Fix handling of narrator when tabbing through search results, it currently doesn't properly announce them, likely due to the entries not actually being selected? Fixing this was a pain in the current default theme.

I'll look into it. I don't expect any complications.

  1. (Already done?) Make sure all strings in the frontend can be localized

There is no posted information on how a 3rd party plugin (or I missed it) can interface with the new i18n API. I'm sure it's possible, but I haven't taken the time yet.

  1. ...that's really it.
    There are a few minor things that still feel somewhat off about the DMT to me, but I'm not really opposed to, including:
  2. The Settings/On This Page boxes have a fuzzy drop shadow while nothing else does

Yeah, definitely on the subjective side of things. The drop shadow just visually separates the right column from the main content. I'm not married to it, but that is why I added it.

  1. It feels unfortunate to lose the "On this page" section when on the two column layout, though I suppose on a phone people will just end up searching in most cases anyways....

I was thinking of a way to keep this in and likely the solution is to add another button in the header bar when in reduced width mode. It's just too much / long of a column to put that on top of the source code index in the mobile menu which is higher priority.

Another thing you might like, I'm considering a different rendering strategy for modules in 0.27:
Re-exports have the arrows, and the first paragraph (or @summary block) of comments are included in the module page for members of a module.

Here I am definitely going to advocate for less is more. Particularly for more randomly generated content in the module pages. It is definitely not guaranteed that picking the first sentence / paragraph for any given reflection will result in quality output. It will really be an eyesore for complex projects yet may seem like it's an improvement for smaller projects. I'd gate this behind an option and have it turned off by default until feedback can come in and effectiveness evaluated for all sizes of projects. Even for gl-matrix it would result in less quality / concise output. Encouraging folks to use @module and provide a module description is far better IMHO.

I need to better understand the Re-exports have the arrows change. Do you mean the main reflection will have an arrow pointing to the re-export? If so I'd suggest not adding that. Re-exports already properly indicate the source. IMHO having the source point to the re-export would be bad UX. It would be rather awkward in the gl-matrix API docs that do use re-exports for past API compatibility. The new API doesn't need to point to the compatibility re-export package at all.

A bit long here, but yeah. Let's definitely continue the conversation. As far as mainlining the DMT goes I need to focus on my actual projects and get release out in the coming 1-2 months and can once again look at things in ~December timeframe. Generally I'd suggest working on non layout effecting features as there are many things that can be tightened up without tweaking the layout and such more for the next release. This will just make integrating the DMT easier.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Sep 25, 2024

that likely broke every 3rd party devs plugins

I can't agree with this. When making changes I suspect might be breaking I check
all plugins updated in the past 6 months or so with a quick grep. At the time I
last grabbed plugins for updating this was:

  • autotool-plugin-typedoc-0.11.4
  • docusaurus-plugin-typedoc-1.0.1
  • ig-typedoc-theme-5.0.3
  • service-client-documentation-generator-3.0.0
  • typedoc-github-wiki-theme-2.0.0
  • typedoc-gitlab-wiki-theme-2.0.0
  • typedoc-material-theme-1.0.2
  • typedoc-plugin-404-2.2.0
  • typedoc-plugin-copy-assets-1.0.11
  • typedoc-plugin-djs-links-2.2.1
  • typedoc-plugin-frontmatter-1.0.0
  • typedoc-plugin-inline-sources-1.0.2
  • typedoc-plugin-localization-3.0.4
  • typedoc-plugin-markdown-4.0.3
  • typedoc-plugin-mdn-links-3.1.27
  • typedoc-plugin-missing-exports-2.2.0
  • typedoc-plugin-plausible-1.0.2
  • typedoc-plugin-replace-text-3.3.0
  • typedoc-plugin-zod-1.1.2
  • typedoc-storybook-theme-1.2.0
  • typedoc-theme-1.0.2
  • typedoc-theme-1.2.0
  • typedoc-theme-dumi-0.1.2
  • typedoc-vitepress-theme-1.0.0

Notably, your DMT theme was not in that list as it was not at the time properly
tagged with the typedoc-plugin or typedoc-theme, so any methods which were used
by it exclusively (removeComponent) were not something I saw any use of in the
wild.

Rather that API: perhaps use Breaking API: to more clearly communicate a change that is well breaking.

All of those changes are listed under a ## Breaking Changes header, I think
the repetition of breaking on each line would be more harmful to readability
than helpful.

seems entirely superfluous serving no purpose whatsoever

(breathes) Removal of (so far as I was aware) functions which were completely
unused and not properly tested is a good thing in terms of keeping TypeDoc
reasonably maintainable. I can see how something you're using going away can be
frustrating, but it's far from purposeless. I could have preserved it and
spent another 8...16? hours to keep the event emitter API exactly the same, but
decided against it.

...releases...

This feels like an annoying amount of overhead for a hobby project that gets a
few hours of my time during the weekend... Making a final pre release build a
few days before the final release and pushing that to GitHub seems reasonable,
but GitHub doesn't seem to have a way to only follow full releases, and I'd hate
to spam people with a beta release every week... maybe that's normal in GitHub
land today? Not wanting to promise too much stability is part of why TypeDoc is
still pre-1.0.

If you apply it to non-navigation index reflections (say methods in a class) it doesn't work and produces garbage output

This ought to produce a warning today that it isn't supported there. I thought
I'd added that, but apparently did not. I'll be correcting that...

Rather than a completely new @document why didn't you just repurpose and use @see ../markdown/somefile.md?

Because it does different things! @document adds the referenced document as a
child. If you wanted to link to a document added elsewhere, then a regular
@link tag or html/markdown link to the given path will give you that.

just adding Markdown files ideally in a separate location more than sufficed

For your use cases, sure. However, I have seen projects, and have had feature
requests where documents are more strongly associated with individual
components, and it doesn't make sense to include them in a top level tree. It
would have been much simpler to just support a folder of documents.

Just take a moment to understand that this is a side effect of using modern ES Module loading.

Yes, and it is incredibly unfortunate that to use ESM and fetch one must give up
being able to just open a page in a browser. I'm not alone in thinking that this
is a desirable feature. rustup doc will open Rust's fully searchable docs
which don't require spinning up a server, git help (with the Git for Windows
distribution at least) will open up a browser with local HTML help files.
Several projects at $WORK that I didn't start also do this with docusaurus,
and not needing to spin up a server every time I want to search the docs is
really nice.

It's not that I can't set up a web server, so much as it's an extra step that
I appreciate not having to do.

There is no posted information on how a 3rd party plugin (or I missed it) can
interface with the new i18n API. I'm sure it's possible, but I haven't taken
the time yet.

https://typedoc.org/api/interfaces/Internationalization.TranslatableStrings.html
has some very light information, definitely deserves more at some point, and
some restructuring to not require digging into the interfaces to find it.

I'd gate this behind an option and have it turned off by default until feedback can come in and effectiveness evaluated for all sizes of projects.

I definitely plan to put the first paragraph "implicit summary" behind an option.

Do you mean the main reflection will have an arrow pointing to the re-export?

The opposite, re-exports in that picture are documented with the arrow pointing
to the main reflection.

Generally I'd suggest working on non layout effecting features as there are
many things that can be tightened up without tweaking the layout and such more
for the next release.

Unfortunately this somewhat conflicts with the issues I currently have worked/am
working/have planned for the next minor... #2597, #2303, #2554 are all in it,
now that the DMT is in the list of plugins that'll get grepped when I change
things, hopefully I can do a better job of noticing when something will break.

@typhonrt
Copy link
Author

that likely broke every 3rd party devs plugins
seems entirely superfluous serving no purpose whatsoever

(breathes) Removal of (so far as I was aware) functions which were completely unused and not properly tested is a good thing in terms of keeping TypeDoc reasonably maintainable. I can see how something you're using going away can be frustrating, but it's far from purposeless.

removeComponent literally served a purpose. While not common any theme that is overridden or extended can't have components removed otherwise due to how the component loading occurs. It's not a lot of code and clearly was doing the intended job. This prevents anyone from extending a theme and changing behavior / altering already loaded components from the base theme.

I could have preserved it and spent another 8...16? hours to keep the event emitter API exactly the same, but decided against it.

I'm not really commenting on keeping the event API here. That is fine because there is a replacement / comparable API. That is not the case with removeComponent.

...releases...
This feels like an annoying amount of overhead for a hobby project that gets a few hours of my time during the weekend... Making a final pre release build a few days before the final release and pushing that to GitHub seems reasonable, but GitHub doesn't seem to have a way to only follow full releases, and I'd hate to spam people with a beta release every week... maybe that's normal in GitHub land today? Not wanting to promise too much stability is part of why TypeDoc is still pre-1.0.

It is totally fine to do this as you have to explicitly follow releases to get notifications anyway. I follow Svelte 5 releases and they are on pre-release 259 with almost daily updates for the last year. Considering for TypeDoc that is ~3-6 beta releases then an RC before final that is low bandwidth for those that need to track progress. For 3rd party devs getting notifications of all releases beta through RC and final is useful.

Just take a moment to understand that this is a side effect of using modern ES Module loading.
Yes, and it is incredibly unfortunate that to use ESM and fetch one must give up being able to just open a page in a browser.
It's not that I can't set up a web server, so much as it's an extra step that I appreciate not having to do.

fetch is available / browser API. With ESM loading file:// URLs are not available for security purposes as import statements would need access to the full file system. My perspective is that it is technical debt not embracing modern tech. The 99% use case is already handled for most devs using a modern IDE w/ built-in web server. A one liner NPM script can spin up a local server otherwise easily.

It would take ~30 lines of code granted with a multi-command CLI to make typedoc open docs be a thing that could automatically spin up a local web server and open the default browser to the docs using sirv and open. There are many options to make things automatic while using modern tech for this "1%" use case.

That being said to not rock the boat I will look into an IIFE approach, but it's just technical debt / convolutes things IMHO.

I definitely plan to put the first paragraph "implicit summary" behind an option.

👍

Unfortunately this somewhat conflicts with the issues I currently have worked/am working/have planned for the next minor... #2597, #2303, #2554 are all in it.

While one is internal API change the other two seem to deal with content and not the macro layout of the docs. I'm more concerned about layout aspects changing between the main column, header, sidebars, and footer. So a relatively quiet period for those aspects would be handy.


Let's keep the conversation going hopefully 0.27.x and following up to any DMT mainlining goes smooth. I need to avoid spending weeks of full time effort leading up to any integration toward the end of the year. As it goes integration likely will be 2-4 weeks of effort in and of itself, so I'd like to keep it contained to just that.

My next window for TypeDoc efforts is likely in December.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Sep 28, 2024

I'm not really commenting on keeping the event API here. That is fine because there is a replacement / comparable API. That is not the case with removeComponent.

Sorry, I was not clear -- the event emitter change was what drove removeComponent. It was implemented through inversion of control on the emitter class.

It is totally fine to do this as you have to explicitly follow releases to get notifications anyway.

Fair enough, I'll think about how to do this in a way which isn't a pain to maintain... I really like how simple it is to make a release/beta release today, but someone has probably made decent automation for it since I set up the original system 4 years ago.

While one is internal API change the other two seem to deal with content and not the macro layout of the docs. I'm more concerned about layout aspects changing between the main column, header, sidebars, and footer. So a relatively quiet period for those aspects would be handy.

The only potential change there is #2630, which I doubt I'll work anytime soon, narrator isn't fun to wrok on.

My next window for TypeDoc efforts is likely in December.

👍 Will probably be the change for 0.28 then, I'm hoping to get 0.27 out before then

Gerrit0 added a commit that referenced this issue Sep 29, 2024
Also make the arrows work properly without JS

As discussed in #2335
@Gerrit0
Copy link
Collaborator

Gerrit0 commented Oct 4, 2024

Not going to work it now, but putting it here as it is something that I think probably would greatly improve the docs for some use cases. If a library references types declared in a dependency, sometimes that dependency might not have a generated doc site. In that case, it may make sense to generate docs for that dependency, but they really probably ought to be more visually separated from the "real" docs than just including a namespace for them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discussion on large effort changes to TypeDoc
Projects
None yet
Development

No branches or pull requests

2 participants