模块出处
[CVPR 20] [link] [code] ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
模块名称
Efficient Channel Attention (ECA)
模块作用
通道注意力
模块结构
模块代码
import torch
import torch.nn as nn
import torch.nn.functional as Fclass ECA(nn.Module):"""Constructs a ECA module.Args:channel: Number of channels of the input feature mapk_size: Adaptive selection of kernel size"""def __init__(self, channel, k_size=3):super(ECA, self).__init__()self.avg_pool = nn.AdaptiveAvgPool2d(1)self.conv = nn.Conv1d(1, 1, kernel_size=k_size, padding=(k_size - 1) // 2, bias=False) self.sigmoid = nn.Sigmoid()def forward(self, x):# feature descriptor on the global spatial informationy = self.avg_pool(x)# Two different branches of ECA moduley = self.conv(y.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1)# Multi-scale information fusiony = self.sigmoid(y)return x * y.expand_as(x)if __name__ == '__main__':x = torch.randn([3, 256, 40, 40])rca = ECA(256)out = rca(x)print(out.shape) # 3, 256, 40, 40
原文表述
通过剖析 SENet 中的通道注意模块，我们通过经验证明避免降维对于学习通道注意很重要，适当的跨通道交互可以保持性能，同时显着降低模型复杂性。因此，我们提出了一种无需降维的局部跨通道交互策略，可以通过 1D 卷积有效实现。此外，我们开发了一种自适应选择 1D 卷积核大小的方法，确定局部跨通道交互的覆盖范围。
