How to train on coco dataset ?

[Tingwei-Jen]

hi,
I can train on pascal voc dataset, but i don't know how to train on coco.
The format of annotations are different, one is xml another is json,
and dose the darkflow only read xml format of annotation?

Any suggestion?

[abagshaw]

@Tingwei-Jen I believe darkflow (as of now) only reads the XML format for PASCAL VOC annotations. It would be great to make darkflow's annotation parsing more versatile and able to read from some more formats, but I don't see that happening any time soon - unless someone wants to work on it 😃

For now, I think you will have to write a script to convert your annotations to the VOC format to train with darkflow.

[jubjamie]

If it helps, I created a script today that converted my own dataset format to VOC and it wasn't too hard to do in python. You only need a few parts of the VOC dataset. An example xml file of mine is this:

<?xml version="1.0"?>
<annontation>
<folder>images</folder>
<filename>10.jpg</filename>
<size>
<width>450</width>
<height>328</height>
</size>
<object>
<name>gun</name>
<bndbox>
<xmin>19</xmin>
<ymin>84</ymin>
<xmax>144</xmax>
<ymax>236</ymax>
</bndbox>
</object>
</annontation>

If you compare that to the VOC dataset you'll notice that it contains a lot of extra fluff that darkwflow doesn't need. Whilst I won't be making a conversion script (as it varies on your original dataset) I can probably help you to make yours in python. Look up ElementTree and the xml stuff in python, super helpful! Good luck!

[abagshaw]

@jubjamie Great, I'm glad you got it to work. What I had in mind for the ideal setup would be some configuration file that specifies where in the XML file the important information can be found (i.e. filename, width, height, name etc.) kind of like a regex but for XML...if you know what I mean. Then this small configuration file can be read by darkflow and used to parse any kind of training annotation data, provided it is in XML format. No need to write any conversion script to convert to a specific format of annotation.

That way, a configuration file can be created for VOC, COCO and any other annotation format someone would want to train from. Then darkflow's parsing system can remain format agnostic. Just an idea - and not sure if it'll ever get implemented but thought I would put it out there.

[jubjamie]

I kind of get what you mean. The initial format of the data wouldn't really matter as long as it ended up in the darkflow format. You don't really need much info and it's quite difficult. The tricky bit is identifying the key info in these random datasets. If it was all in XML then you could just specify the original XML field and then copy that to the darkflow XML, but that's quite easy. For ANY dataset you need to understand the original data layout.

If you are starting from scratch someone on another issue found this:
https://github.com/tzutalin/labelImg

Very interesting stuff!

[abagshaw]

I wasn't thinking it would auto-detect the data from the configuration file - although that would be cool. The configuration file (something like annotation_format.json) would specify how to traverse the XML annotation data to find the nessecary fields and then this file would be modified to read that configuration file and parse the XML annotations accordingly. Unless I'm misunderstanding something, there is no "darkflow XML" format per se - as everything gets read into the memory. Anyway, something for the future maybe.

[jubjamie]

I'm confused. What would the .json file do or contain?
I might just write a quick guide and a little python API to build these datasets at some point tomorrow or next week. It's really not very hard. I'm sure it already exists to be honest. I just don't understand your idea sorry! What would the user put in? And then would it return an XML file? Or go straight into the training thing?

[abagshaw]

The .json file would contain something along these lines

{
    "file_name": "annotation->filename",
    "width": "annotation->size->width",
    "height": "annotation->size->height"
    "label": "annotation->object->name",
    etc.
}

The above is an example for VOC showing the darkflow parser how to find the attributes it needs (i.e. how to traverse the XML annotation files) if the annotation files are in a JSON format then the same configuration style should still work to traverse the nested dictionaries. If someone wanted to use a different annotation format all they need to change is the .json file as shown above to show darkflow how to find the necessary values from their annotation format. Maybe this is a bad idea - but I thought it would make things a little more versatile between format types without having one format type (i.e. VOC) that everything needs to be converted to (makes things simpler without having to convert).

No XML file is returned - this is not a conversion script - what is currently here would be replaced by code that would read the .json configuration file, traverse the annotation files as specified by the configuration file while loading everything into memory like it already does. The parsing script is currently hardcoded for VOC (it only reads the VOC annotation format). This method would make it able to read any kind of JSON or XML annotation format provided the configuration file properly tells it how to traverse the files.

Does that make sense?

[jubjamie]

I see what you mean. I was thinking of doing it as a kind of API where you just pass the file and the mappings and it will generate the corrected files for you. I guess you could do it with a json file too that you load in as a translator. However, considering that you're images also need to be put in a certain place (and be named correctly) it might be better in the long run to have something that creates the file so that way you can share it or use it again without having to run the conversion every time you train etc.

[abagshaw]

@jubjamie Your method should work too although I think it's a little more complex. I still don't think we're quite on the same page here. I'm not proposing any method that uses conversion.

Right now darkflow parses the XML annotation files each time you try to train. Instead of parsing the XML files with the current script that can only traverse VOC annotation XML files I'm proposing that a configuration file specifies how darkflow should parse the XML files. It would not be any slower than the current method. It would be faster than your proposed method as there is no intermediate step.

In your method (if I understand it correctly) your original annotation files are parsed and then saved in VOC format - then darkflow parses the VOC files and loads the data into memory. My method removes the middle step - the original annotation files are directly parsed by darkflow into memory as specified by the json configuration file. Does this make sense? 😄

If you wanted to share your annotation files all you would have to do is share them along with your small .json configuration file that tells darkflow how to parse them. That's it.

[jubjamie]

Yes I did understand your method. I just know that it would be quicker for me to implement one that makes a file.
And you need to remember that datasets need to be easily shared across the community. Not everyone uses darkflow so it wouldn't hurt to have your dataset available in the popular VOC format.
Both solutions can be implemented separately anyway :)

[Tingwei-Jen]

@abagshaw Thanks a lot, i will find another solution to solve this problem 😄

[thevennamaneni]

I know it's an old post but you can convert those json annotations to xml using something like this:
https://github.com/tylin/coco-dpm/blob/master/coco/convert_to_pascalformat.py