NNI Manager Code Quality Improvement

Long Term Project

Quality Metrics

Decoupled

• A module must list all of its coupled modules in comments.
• A module must not couple with 3+ modules, and should not couple with 2+ modules.
• Including coupled modules, a module should not depend on 5+ modules.

Term definition: Two modules are calls coupled if they are bidirectional dependent, that it is practically impossible to modify one module without modifying the other.

Strong Typed

• Weak types including any, object, and structured string are semantically forbidden.
  • If these types are used in signature, the docstring must contain detailed schema and an example.
• If a non-constrained type is passed across modules, it must have a type alias. (e.g. SearchSpace)

Clearly Named

• One concept can only have one term, except in user visible strings for compatibility.
• Directory name must have concrete meaning, or concrete explanation if it's an abstract word.
• "Utils" can only contain optional function. Anything with more than one dependent cannot be util.
• Exported classes and functions shall not use abbreviation, unless it's extremely long and has no ambiguity in any context.

Tested

• Each .ts file should have a test file.
• Apart from error handling routines, each module's unit test coverage should be 100%.
• Overall unit test coverage must be 80%+.
• If a script has a strong reason for not being covered, the reason must be written in comment.
• Unit tests should not require other modules except globals.

Documented

• Each directory must have doc in index.ts. It must clarify:
  1. What are in this directory.
  2. What can be put into this directory.
• Each script must have file-level doc. It must clarify:
  1. Why this is needed.
  2. How to use classes and functions of this module.
  3. What assumptions are made on other modules (excluding lib).
• No need to say, each exported API must be documented.
• Unit test files must be documented. The comments should contain:
  1. What behaviors are tested.
  2. How are mocked things set up.
  3. For module level output recording objects, what they hold and where the values come from.

Progress Tracker

• Infrastructure
  • start up
  • shutdown
  • logging
• REST Server
  • express server
  • API handlers
  • WebSockets
• NNI Manager
  • tuner interface
  • storage
  • business logic
• Standalone Modules
  • experiment manager
  • tensorboard
• Training Service
  • classic
  • "reusable"
  • environment services