Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extend json and yaml support #42

Closed
2 tasks done
kekavc24 opened this issue Oct 30, 2023 · 0 comments
Closed
2 tasks done

feat: extend json and yaml support #42

kekavc24 opened this issue Oct 30, 2023 · 0 comments
Assignees
Labels
adds-feature Pull request adds new feature with pending release to production. enhancement New feature or request

Comments

@kekavc24
Copy link
Owner

kekavc24 commented Oct 30, 2023

Description

Partial support for JSON was added after #28 was closed. However, key differences and similarities exist in both, with each having its strengths. It would be great to extend and support that

Requirements

  {
      "key" : [ "normal", ["nested"] ]
  }
 {
      "key" : [  "normal value", {"nested" : "map"}, "another value" ]
  }

In Yaml, a nested list would need to be linked to another key. So any list would technically be incorporated to the nearest key.

key:
    - normal value
    
     # a nested map in list in yaml
    - nested: map
      another key: value

    - another value
    
    # a nested list linked to a key
    - nested-list:
        - nested value
        - another value
@kekavc24 kekavc24 added enhancement New feature or request adds-feature Pull request adds new feature with pending release to production. labels Oct 30, 2023
@kekavc24 kekavc24 self-assigned this Oct 30, 2023
@kekavc24 kekavc24 changed the title feat: extend json support feat: extend json and yaml support Oct 30, 2023
kekavc24 added a commit that referenced this issue Nov 1, 2023
> allow for adding of maps in list
> allow for adding string/lists to map. map will be converted to list
> add support for recursive update of keys within lists
> add support for updates within lists nested in lists
@kekavc24 kekavc24 linked a pull request Nov 1, 2023 that will close this issue
7 tasks
@kekavc24 kekavc24 pinned this issue Nov 2, 2023
kekavc24 added a commit that referenced this issue Feb 4, 2024
> allow for adding of maps in list
> allow for adding string/lists to map. map will be converted to list
> add support for recursive update of keys within lists
> add support for updates within lists nested in lists
@kekavc24 kekavc24 removed a link to a pull request Feb 4, 2024
7 tasks
@kekavc24 kekavc24 mentioned this issue Feb 4, 2024
7 tasks
kekavc24 added a commit that referenced this issue Feb 4, 2024
* test: add generic types to infer type

> remove dynamic type

* feat: add recursive function to update file

> add extension on map to update nested keys
> add extension to convert file read as String to Map
> clean up code to edit yaml file

* refactor: add context to thrown errors

> add information to error to give hint to user on what to fix

* test: add tests for recursive function in 39841af

> update tests for refactored code to test via recursive function
> move tests to map extension test file
> add tests for recursively reading nested keys
> add tests for keys created when missing in file

* style: run dart format

* docs: add missing key in example

* test: update test to check for new exception

> check exception matcher one added in 7348ac0

* refactor: make argsChecker class abstract

* refactor: reduce code duplication

> reduce number of classes managing version modifiers
> update code to reflect changes

* refactor: clean-up arg checker class

> make base class abstract
> add custom overrides for subtypes
> update test to to reflect changes

* refactor: replace mixin with handler class

> consolidate file operations and reused properties to one class
> reduce number of inputs from outside the handler

* refactor: create template for handling a command

> reduce code repetition. Move common functionality to super class
> move file I/O inputs to FileHandler a42d8dd
> move core functionality to subclasses with custom implementation

* test: update file handler tests

* refactor: add extended json/yaml support #42

> allow for adding of maps in list
> allow for adding string/lists to map. map will be converted to list
> add support for recursive update of keys within lists
> add support for updates within lists nested in lists

* refactor: add recursive helper class

> add helper function added in 532bb6e to class
> update recursive read to support changed added in 532bb6e

* test: update tests to reflect 532bb6e

> add new tests for new functionality

* style: run dart format

* fix: fix issues with test

> fix bug caught where value returned is not a string but list
> update tests based on changes added in 532bb6e

* fix: fix bug caught in test

> add ignore rest args for set and bump command arg checkers
> add missing override for set command handler

* refactor: clean up code

> reduce constant conversion of file content
> add yml as valid file extension

* style: remove unused file import

* fix: fix bug saving yaml files as json

> remove direct conversion of file to dart map
> add file ouput containing the file as YamlMap from file handler
> remove dead unreferenced code
> update tests to use update file output

* refactor: clean up code

> remove multiple "continue" in loops

* feat(initial): add finder implementation

> add recursive indexer for yaml/json nodes
> add finder that matches based on keys, values or pairs provided
> add data types for indexing & data nodes found based on conditions

* style: run dart format

* fix: ensure length is same in strict order

> add extension method to check for this

* refactor: clean up code

> add hasAny method to iterable
> add minor documentation

* refactor: add support for multi-directory commands

> extend `FileHandler` to support multiple file reads

* test: update `FileHandler` tests based on e1b0105

> add test for reading file from multiple directories

* refactor: clean up code

> abstract finder functionality

* feat: add support for replacer

> add initial `Replacer` implementation
> add `UpdateMode` enum for easy control
> add support for replace and renaming keys in recursive update
> move recursive helpers to own file
> add supporting typedefs for `Replacer` & `Finder`
> clean up code to reflect `UpdateMode` enum changes

* test: update old tests to support Update Enum

* test: add tests for UpdateMode.Replace

> add tests for recursion based on UpdateMode.Replace

* refactor: clean up code

> add equality & hashcode to `NodeData` & `MatchedNodeData` classes
> add collection for equality comparison
> run dart format

* fix: fix bug caught in test

> convert fileAsYamlMap to modifiable map

* fix: fix bug caught in test

> fix typo in prompt requesting file from console

* fix: fix bug caught in test

> fix bug introduced when refactoring code (ref commit 3555e37)

* refactor: code cleaup & improvements

> remove flag from base command
> rename arg checkers to normalizers
> update name changes

* feat: add partial implementation for  new commands

> add normalizers for `find`, `rename` & `replace` commands
> add enums, typedefs & extensions for functionality

* feat: add custom `Replacer`

> add `MagicalReplacer` for values & `MagicalRenamer` for keys
> move implementations to relevant folders

* feat: add custom lightweight pair definition

> add key and value with level and indices pointing to index in list
> port old NodeData object to support functionality

* refactor(initial): extend 676ee02 to existing code

> declutter and revert changes made in e53d8a0 while adding `UpdateMode`
> remove class with static methods in favour of Dart top level functions
> add method to handle UpdateMode.replace on previously indexed map/list
> update `Replacer` & `Finder` to use pair definition (676ee02)
> extende `MatchedNodeData` to provide additional info

* feat(initial): add initial manager implementation

> add `FindManager` & `ReplacerManager` for simple aggregation
> add `TransformTracker` to managers to abstract aggregation progress

* refactor: clean up & optimize code

> simplify pair definition using Dart records
> apply changes to `NodeData` object
>  add method to return shortest key path for recursive rename
>  update replacer to use shortest key path

* fix: fix bugs caught in local test run

> switch to target if path is exhausted and is nested in list
> return modified list at "n" instead of "n+1" when recursing list

* test: add tests reverted in c4df240

> tests in question where reverted when UpdateMode.replace was removed

* refactor: extend Replacer functionality

> reduce code repetition. Return pair used for replacement

* fix: fix pair definition bug 194b9c0

> map key to value instead of calling toString on dart records added in commit

* refactor: make `MatchedNodeData` a subclass of `NodeData`

> remove "composition" allowing `MatchedNodeData` to use important methods available for `NodeData`
> add method to get "path" of node with key & value indices stripped

* feat(initial PoC): add custom printer class for aggregating info

> add utility methods for parsing and creating tree-like view
> make code ready for future extensibility

* refactor: extend custom tracker functionality

> require file index when resetting tracker for better file tracking
> add method to get count using key

* refactor: clean up manager & subclasses

> add `ConsolePrinter` as required arg d9df525
> clean up `FinderManager` &  `ReplacerManager`. Reduce bloated code which made it hard to follow code
> add file number when calling reset in tracker 909ad98
> add initial final touches that makes the managers complete and ready for tests
> customize enums & typedefs for manager use

* chore(deps): bump test from 1.24.9 to 1.25.0 (#47)

Bumps [test](https://github.com/dart-lang/test/tree/master/pkgs) from 1.24.9 to 1.25.0.
- [Release notes](https://github.com/dart-lang/test/releases)
- [Commits](https://github.com/dart-lang/test/commits/test-v1.25.0/pkgs)

---
updated-dependencies:
- dependency-name: test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: switch to class `PairType`

> replace `Key` & `Value` record with subclasses of `PairType`
> add package Equatable to ease equality comparison. Poor hashing led to test failures despite objects being same.
> refactor `NodeData` to use new pair types
> clean up code references using old `PairType` record

* test: add tests for `MagicalIndexer` class

> update extension tests to use updated `PairType` 4af6107

* refactor: switch to `Equatable`

* refactor: add reusable generic tracker class

> remove bloated narrow class that was hard to customize
> clean up code referencing old generic class

* refactor: simplify generic tracker class f97ad57

> rename appropriately to `Counter` as this reflects functionality
> remove unnecessary `MultiValue` class. Duplicate as simple `Counter` performs same functionality
> add separate counter class that maintains previous counter histories

* refactor: move granular tracking for each value to `Finder` class

> add custom counter to increment using `MatchedNodeData`
> move members to `ValueFinder` subclass.

* refactor: add method to get sum of count in `Counter`

* refactor: add capability to swap map being indexed once complete

> make map member public in `MagicalIndexer`
> move counter history functionality to `Finder` itself. Switch to `CounterWithHistory`

* refactor(initial): clean up `TranformManager` class

> reduce code complexity in favour of granularity
> add `Counter` for keeping track of count of values found in each file
> refine `FinderManager` functionality to queue files & manage Finder. Builds on 8933949 & d3091c5

* fix: fix bug where execution continues after yielding a generator

* refactor(initial): clean up `ReplacerManager`

> clean class and add changes introduced in 629cc95 and any linked commits
> switch to normal list instead of queue to store modified maps
> add docs to `FinderManager` members used in `ReplacerManager`

* refactor: improve printer aggregation

> add method to enable easy access to counter history using cursor & value
> clean code due changes introduced in 629cc95
> add better formatting of info aggregated

* test: add tests for `ValueFinder`

> move tests for `MagicalIndexer` to dedicated folder
> remove print statement in map extension tests

* fix: fixed `Counter` issues caught in tests added in 47f7fe4

> make `Origin` a required parameter as it is always passed in all methods
> prefer a DualKey when adding map entries to preserve hashcode

* fix: fixed `Finder` bugs caught in tests added in 47f7fe4

> add origin when adding entries to `MatchCounter`
> move `MatchCounter` setup to methods called by `Finder.find`
> switch to named constructors in `ValueFinder`

* refactor: add extensible tracker class

> add generic tracker class for reusability
> clean up rigid `Counter` class

* refactor: clean up manager

> split manager to uphold encapsulation
> add custom tracker for Replacer manager thanks to 6aa1486

* refactor: remove unnecessary protected method for creating keys

> remove need for implicit type when creating key

* refactor: add contextual names to replacer subclasses

* feat(WIP):  add modular Formatter in place of ConsolePrinter

> simplify formatter to simply format strings
> explore generics & modularity to ease testing
> clean up utility methods
> add custom Tracker(6aa1486) class for custom functionality

* refactor(WIP): cleanup Managers

> add custom Formatters dd8d15d
> reduce bloated code

* refactor: clean up file handler class

> make static methods a top level function
> remove unnecessary typedefs and annotations

* refactor: remove unused utility methods & classes

> convert static methods to top level function in VersionModifier object
> remove dead code and anti-pattern behavior

* refactor: clean up core functionality code

> clean up ripple effects of a560c88
> optimize existing code
> remove class with static methods in favour of top level functions

* refactor: clean up code

> add context to base generic classes & other classes
> remove dead unused code
> undo unnecessary encapsulation

* refactor: remove unnecessary mutations on targets for replacer

> remove need to indicate if targets are keys/values since each Replacer subclass does that
> remove typedefs

* test: add Replacer tests & helper

* test: update old tests to use latest code refactor

> Refer to commits -> f76f4f1, 4afea55, 6ab2c73

* test: add tests for base `Tracker` class

> rename exception helper to provide more context

* test: add tests from base `Counter` class

* test: add `ReplacerTracker` tests

* fix: fix bug where tracker overwrites first match of a path

> add method on parent Tracker class allowing to check if it contains a value
> not much of a destructive bug, but this would have resulted in wasting additional CPU time updating a path that already exists?
> replace guard clause with `map.putIfAbsent`

* test: add tests for custom tracker used by the `Formatter` class

* refactor: make tracker testable (refer 8ae19f0)

> allow for custom max tolerance injection

* test: add tests for utility methods used by `Formatter` & its subclasses

* refactor: encapsulate formatter functionality

> move formatting method to actual class instead of making it a top level method
> rename file with utility methods for formatter

* test: add tests for `Formatter` and its subclasses

* refactor: add record for storing formatted path info

> remove confusing use of DualTrackerKey
> remove generics bloat which made it hard to understand functionality

* test: update newly added test to match changes in a6067c8

> remove test for DualTrackerKey
> updated tests commit ref: b465b31, 9361917 & 8ae19f0

* refactor: clean up manager classes

> remove static members & factory constructors. Make use of late & named constructors as is.
> make file inputs a generic map instead of a concrete type
> add way of accessing modified files in replacer manager

* refactor: allow option to show progress in FileHandler

* refactor: encapsulate & consolidate core members of manager subtype

> initialize all code within constructor body in ReplacerManager
> A manager just manages the file queue. Most importantly, the FinderManager does the heavy lifting.

* test: add find & replacer manager tests

* feat(WIP): add `walk` command to `mag`

> add find subcommand for find values in yaml/json files
> add rename subcommand for renaming/swapping keys
> add replace subcommand for replacing values
> add handlers for each subcommand

* style: run dart format

* chore: manually bump to WIP dev release

* chore: update changelog for 80044e7

* style: remove reference to console printer

* style: run dart format

* fix: make targets nullable when normalizing arguments for bump subcommand

> allow targets to "prepped" if null

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@kekavc24 kekavc24 closed this as completed Feb 7, 2024
@kekavc24 kekavc24 unpinned this issue Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
adds-feature Pull request adds new feature with pending release to production. enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant