Inferring Actions from Video Demonstration using IDM

Step 1

Clone the repository

Step 2

Download the datasets from here - dataset

Keep the data and validation folder in your root directory of the project.

Step 3

Install necessary packages -

• PyTorch (Installation Instruction)
• Numpy
• Tensorboard
• os
• gc

Step 4

Install jupyter lab and run the local jupyter server by following the instructions

Step 5

Open the IDM.ipynb file. Configure the adjustable variables and then run the script.

For custom data generation

Step 1

Setup Airsim on Unreal Engine using the instructions here - Windows, Linux, macOS

Step 2

Run the airsim in computer vision mode. And run either DataCollectionPipeline.py for keyboard controlled data collection or RandomDataCollectionPipeline.py for automated data collection.