论文阅读 [TPAMI-2023] Cyclic Differentiable Architecture Search

论文搜索(studyai.com)

搜索论文: Cyclic Differentiable Architecture Search

搜索论文: http://www.studyai.com/search/whole-site/?q=Cyclic+Differentiable+Architecture+Search&fr=csdn

关键字(Keywords)

Computer architecture; Optimization; Search problems; Task analysis; Training; Microprocessors; Object detection; Cyclic; introspective distillation; differentiable architecture search; unified framework

机器学习

神经架构搜索

摘要(Abstract)

Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search.

可区分ARchiTecture搜索，即DARTS，在神经架构搜索中引起了极大的关注。

It tries to find the optimal architecture in a shallow search network and then measures its performance in a deep evaluation network.

它试图在浅搜索网络中找到最佳架构，然后在深度评估网络中测量其性能。

The independent optimization of the search and evaluation networks, however, leaves a room for potential improvement by allowing interaction between the two networks.

然而，搜索和评估网络的独立优化允许两个网络之间的交互，从而为潜在的改进留下了空间。

To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CDARTS.

为了解决有问题的优化问题，我们提出了新的联合优化目标和一个新的循环可微ARchiTecture搜索框架，称为CDARTS。

Considering the structure difference, CDARTS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation.

考虑到结构差异，CDARTS通过内省蒸馏在搜索和评估网络之间建立循环反馈机制。

First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized.

首先，搜索网络生成用于评估的初始架构，并优化评估网络的权重。

Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation.

第二，通过分类中的标签监督，以及通过特征提取从评估网络中正则化，进一步优化搜索网络中的架构权重。

Repeating the above cycle results in a joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network.

重复上述循环会导致搜索和评估网络的联合优化，从而使体系结构的演变能够适应最终的评估网络。

The experiments and analysis on CIFAR, ImageNet and NATS-Bench [95] demonstrate the effectiveness of the proposed approach over the state-of-the-art ones.

在CIFAR、ImageNet和NATS Bench[95]上的实验和分析表明，与最先进的方法相比，所提出的方法是有效的。

Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet.

具体而言，在DARTS搜索领域，我们在CIFAR10上获得了97.52%的前1精度，在ImageNet上获得了76.3%的前1准确性。

In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0.

在链结构搜索空间中，我们在ImageNet上实现了78.2%的前1精度，比EfficientNet-B0高1.1%。

Our code and models are publicly available at https://github.com/microsoft/Cream…

我们的代码和模型可在https://github.com/microsoft/Cream.。

作者(Authors)

[‘Hongyuan Yu’, ‘Houwen Peng’, ‘Yan Huang’, ‘Jianlong Fu’, ‘Hao Du’, ‘Liang Wang’, ‘Haibin Ling’]