Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade the computer vision template #1085

Closed
10 of 13 tasks
Ezward opened this issue Jan 14, 2023 · 0 comments
Closed
10 of 13 tasks

Upgrade the computer vision template #1085

Ezward opened this issue Jan 14, 2023 · 0 comments
Assignees

Comments

@Ezward
Copy link
Contributor

Ezward commented Jan 14, 2023

The cv_control.py/cfg_cv_control.py template files implement a line following algorithm using opencv and traditional computer vision techniques. Currently the template only supports the raspberrypi camera and the a pwm steering/throttle drivetrain. So it does not support the various cameras and drivetrains supported by the complete.py and path_follow.py template. In addition the LineFollower part is hard-coded into the template, so it is not easy to replace.

We would like to upgrade the template;

  • use the add_camera() method in complete.py to add support for the full range of supported cameras.
  • use the add_drivetrain() method in complete.py to add support for the full range of supported drivetrains
  • use the add_controller() method in complete.py to add support for the full range of game controllers, including the web controller.
  • separate out the LineFollower part into it's own source file in the parts folder.
  • add logic that allows a computer vision part to be selected and loaded at runtime, so the user can create their own part without having to edit the LineFollower part. We do something similar with the custom joystick; if a user has created a custom joystick using the Joystick Wizard then they have a file named MyJoystick.py in their mycar application folder. If they choose "custom" as the JOYSTICK_TYPE in myconfig.py, then it will load that part and use it as the joystick.
  • add configuration for the LineFollower part; you can see the hard-coded parameters the constructor
        self.scan_y = 60   # num pixels from the top to start horiz scan
        self.scan_height = 10 # num pixels high to grab from horiz scan
        self.color_thr_low = np.asarray((0, 50, 50)) # hsv dark yellow
        self.color_thr_hi = np.asarray((50, 255, 255)) # hsv light yellow
        self.target_pixel = None # of the N slots above, which is the ideal relationship target
        self.steering = 0.0 # from -1 to 1
        self.throttle = 0.15 # from -1 to 1
        self.recording = False # Set to true if desired to save camera frames
        self.delta_th = 0.1 # how much to change throttle when off
        self.throttle_max = 0.3
        self.throttle_min = 0.15

        # These three PID constants are crucial to the way the car drives. If you are tuning them
        # start by setting the others zero and focus on first Kp, then Kd, and then Ki.
        self.pid_st = PID(Kp=-0.01, Ki=0.00, Kd=-0.001)

  • add configuration to optionally draw the algorithm's output on an output image; see LineFollower.debug_display(). We need to make the image an output parameter. If the user has chosen to update the image, then do it and return it; if not then just return the input image. Note that the current code just draws on the image then shows it in an opencv windows; it does not produce a new image and return it. it is better to create a new image and leave the original intact and then decide which image will be shown in the webui, so the user can see it in realtime.
  • support the assignable web ui buttons; allow the user to assign various functions to the web ui buttons via configuration; like we do in path_follow.py template. This is particularly important if the user is using a standard RC controller that does not have buttons. It is also useful to allow the user to remap the controls for any joystick/game controller.
    • Add functions to control various aspects/configuration of the template; like the pid parameters, the min and max throttle, etc.
  • Share more code; pull out of inline classes and make the importable
    • AiRunCondition, UserCondition, PilotCondition, DriveMode
      • Note that complete.py has PilotCondition but not UserCondition; it appears path_follow.py needs it, but not complete. We may want to combine this into a single that outputs both run_user and run_pilot conditions.
      • Note that complete.py has both AiRunCondition and PilotCondition that create redundant states; ai_running and run_pilot respectively. We probably only need one of these.
      • Note the differences between DriveMode in path_follow and complete templates; the version if complete.py applies the AI throttle multiplier, the version if path_follow does not. However it would be good if it did.
    • ToggleRecording and AiRecordingCondition;
      • ToggleRecording currently only used by path_follow.py. It looks like it is used to handle toggle recording with user-programmable buttons. This competes with the auto_record_on_throttle setting; so it auto_record_on_throttle is true, then manual toggling of mode is disabled. It looks like we should be using this in complete as well to make sure the recording condition is properly maintained.
      • AiRecordingCondition is used in complete.py and turns on recording if the ai is running if the RECORD_DURING_AI configuration is set. RECORD_DURING_AI also causes the pilot/steering and pilot/thottle to be saved in the tub. We need to make the different ways of setting the recording condition simpler (a single part that adjudicates the associated configuration settings and runtime modes would be ideal.)
      • Note that the LineFollower part as written is outputing the recording condition and is based on a hard-coded value. It should not do this; it should allow AiRecordingCondition to decide if images should be recorded during autopilot.
  • make sure tub writing supports the necessary outputs; it's very simple in the current cv_control.py; it only saves image, throttle and steering. Make sure it correctly honors the recording condition.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant