BWN/XNOR SqueezeNet training

[AGhiuta]

Hi,

I've been trying to train SqueezeNet in both configurations(bwn and xnor), but I can't get past 31% (24% respectively) top-1 accuracy (I was expecting accuracies similar to alexnet). I tried something similar to the GoogLenet variant depicted in the paper (I replaced the expand layers with straightforward convolutions with kernel sizes of 3x3, so there is no branching).

Have you tried to train this model? If positive, can you, please, tell me how did you do it?

Thank you,
Alex

ping @mrastegari

[Rahim16]

Hi,

I think the problem you are facing arises from that SqueezeNet can achieve 50x reduction in model size compared to AlexNet because it[AlexNet] has a lot of redundancy and also SqueezeNet's Fire modules were fine-tuned to achieve that level of accuracy with float weights. Binarization itself however results in a reduction of redundancy in AlexNet, thus further applying filter size reduction as proposed by SqueezeNet becomes too destructive.

BR,
Rahim