Gradient:
-
Searching for A Robust Neural Architecture in Four GPU Hours | [CVPR 2019]
D-X-Y/GDAS | [Pytorch] -
ASAP: Architecture Search, Anneal and Prune | [2019/04]
-
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours | [2019/04]
dstamoulis/single-path-nas | [Tensorflow] -
Automatic Convolutional Neural Architecture Search for Image Classification Under Different Scenes | [IEEE Access 2019]
-
sharpDARTS: Faster and More Accurate Differentiable Architecture Search | [2019/03]
-
Learning Implicitly Recurrent CNNs Through Parameter Sharing | [ICLR 2019]
lolemacs/soft-sharing | [Pytorch] -
Probabilistic Neural Architecture Search | [2019/02]
-
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation | [2019/01]
-
SNAS: Stochastic Neural Architecture Search | [ICLR 2019]
-
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search | [2018/12]
-
Neural Architecture Optimization | [NIPS 2018]
renqianluo/NAO | [Tensorflow] -
DARTS: Differentiable Architecture Search | [2018/06]
quark0/darts | [Pytorch]
khanrc/pt.darts | [Pytorch]
dragen1860/DARTS-PyTorch | [Pytorch]
一个很快就能跑完的NAS工作是用可微分的方式进行搜索的DARTS: Differentiable Architecture Search[ICLR’19]，在今年被各种魔改（我猜很可能是因为开源了pytorch的代码，然后代码写得很漂亮改起来也比较容易）。DARTS和ENAS很像，也是从DAG中找子图，并且同样采用了weight sharing，主要的区别在于搜索空间和搜索算法上。搜索空间比较类似NASNet search space，搜索算法则是采用了可微分的方式。
DARTS也是搜cell然后按照一定的模式堆起来，在搜索算法上与之前的方法最大的区别在于DARTS选择某个操作的时候并不是像之前按照RNN的输出概率进行采样，而是把所有操作都按照权重(不是直接加权，先将权重softmax归一化)加起来，那这样权重就在计算图里面了，在计算验证集loss之后就可以backward计算权重的梯度，直接通过梯度下降来优化权重。搜索的过程也就是优化权重的过程，最后保留权重最大的操作就是最后搜到的结构。
权重的优化和DAG里面weight的优化类似于ENAS，也是交替进行的(在训练集上优化weight，在验证集优化权重)，不过DARTS是整个DAG进行forward-backward（这样的缺点在于内存开销会比较大）而不是像ENAS一样先采样子图。
