一:频率限制原理
django rest framework 中频率控制基本原理基于访问次数和时间,通过计算实现,当然我们也可以自己定义频率控制方法。基本原理如下:
启用频率,DRF内部会有一个字典记录来访者的IP,以及访问时间最近几(通过配置)次的访问时间,这样确保每次列表中最后一个元素都是该用户请求的最早时间,形式如下:
{
IP1:[第三次请求时间,第二次请求时间,第一次请求时间,],
IP2:[第二次请求时间,第一次请求时间,],
.....
}
举例说明,比如我现在配置了5秒内只能访问2次,每次请求到达频率控制时候先判断请求者IP是否已经在这个请求字典中,若存在,在判断用户请求5秒内的请求次数,若次数小于等于2,则允许请求,若大于2,则超过频率,不允许请求。
关于请求频率的的算法(以5秒内最多访问两次为例):
1.首先删除掉列表里5秒之前的请求,循环判断当前请求时间和最早请求时间之差记作t1,若t1大于5则代表列表中最早的请求已经在5秒外了,删除掉,继续判断倒数第二个请求,直到t1小于5.
2.当确保请求列表中只有5秒内请求时候,接着判断其请求次数(列表长度),若长度大于2,则证明超过5秒内访问超过2次了,则不允许,否则,通过并将此次访问时间插入到列表最前面,作为最新访问时间。
二:基本使用
1:在utils目录下新建立文件,throttle.py,添加频率控制为每分钟只能访问5次
from rest_framework.throttling import SimpleRateThrottle
class VisitThrottle(SimpleRateThrottle):
"""5秒内最多访问三次"""
scope = "WD" #settings配置文件中的key,用于获取配置的频率
def get_cache_key(self, request, view):
return self.get_ident(request)
2:settings.py中配置全局频率控制
REST_FRAMEWORK = {
#频率控制配置
"DEFAULT_THROTTLE_CLASSES":['utils.throttle.VisitThrottle'], #全局配置,
"DEFAULT_THROTTLE_RATES":{
'WD':'5/m', #速率配置每分钟不能超过5次访问,WD是scope定义的值,
}
}
3:路由配置
urlpatterns = [
re_path(r'^test/$', views.TestView.as_view()),
]
4:视图类
class TestView(APIView):
authentication_classes = [MyAuth]
def get(self, request):
return Response('测试')
5:测试如图
三:源码剖析
1:dispatch()源码
def dispatch(self, request, *args, **kwargs):
"""
`.dispatch()` is pretty much the same as Django's regular dispatch,
but with extra hooks for startup, finalize, and exception handling.
"""
self.args = args
self.kwargs = kwargs
#对原始request进行加工,丰富了一些功能
#Request(
# request,
# parsers=self.get_parsers(),
# authenticators=self.get_authenticators(),
# negotiator=self.get_content_negotiator(),
# parser_context=parser_context
# )
#request(原始request,[BasicAuthentications对象,])
#获取原生request,request._request
#获取认证类的对象,request.authticators
#1.封装request
request = self.initialize_request(request, *args, **kwargs)
self.request = request
self.headers = self.default_response_headers # deprecate?
try:
self.initial(request, *args, **kwargs)
# Get the appropriate handler method
if request.method.lower() in self.http_method_names:
handler = getattr(self, request.method.lower(),
self.http_method_not_allowed)
else:
handler = self.http_method_not_allowed
response = handler(request, *args, **kwargs)
except Exception as exc:
response = self.handle_exception(exc)
self.response = self.finalize_response(request, response, *args, **kwargs)
return self.response
2.执行inital方法,initial方法中执行check_throttles则开始频率控制
def initial(self, request, *args, **kwargs):
"""
Runs anything that needs to occur prior to calling the method handler.
"""
self.format_kwarg = self.get_format_suffix(**kwargs)
# Perform content negotiation and store the accepted info on the request
neg = self.perform_content_negotiation(request)
request.accepted_renderer, request.accepted_media_type = neg
# Determine the API version, if versioning is in use.
version, scheme = self.determine_version(request, *args, **kwargs)
request.version, request.versioning_scheme = version, scheme
# Ensure that the incoming request is permitted
#2.实现认证
self.perform_authentication(request)
#3.权限判断
self.check_permissions(request)
#4.频率限制
self.check_throttles(request)
3.下面是check_throttles源码,与认证、权限一样采用列表对象方式,通过判断allow_request方法返回值判断频率是否通过
def check_throttles(self, request):
"""
Check if request should be throttled.
Raises an appropriate exception if the request is throttled.
"""
for throttle in self.get_throttles(): #循环频率控制类结果
if not throttle.allow_request(request, self): #判断其中的allow_requestf返回结果,true则频率通过,否则返回等待多少秒可以访问
self.throttled(request, throttle.wait())
4.get_throttles方法,采用列表生成式生成频率控制对象,与认证、权限一样
def get_throttles(self):
"""
Instantiates and returns the list of throttles that this view uses.
"""
return [throttle() for throttle in self.throttle_classes] #列表生成式生成控制频率对象列表
5.self.throttle_classes属性获取
class APIView(View):
# The following policies may be set at either globally, or per-view.
renderer_classes = api_settings.DEFAULT_RENDERER_CLASSES
parser_classes = api_settings.DEFAULT_PARSER_CLASSES
authentication_classes = api_settings.DEFAULT_AUTHENTICATION_CLASSES
throttle_classes = api_settings.DEFAULT_THROTTLE_CLASSES #频率控制全局配置
permission_classes = api_settings.DEFAULT_PERMISSION_CLASSES
content_negotiation_class = api_settings.DEFAULT_CONTENT_NEGOTIATION_CLASS
metadata_class = api_settings.DEFAULT_METADATA_CLASS
versioning_class = api_settings.DEFAULT_VERSIONING_CLASS
6.通过以上分析,知道了频率控制是通过判断每个类中的allow_request放法的返回值来判断频率是否通过,下面我们来看看我们所使用的SimpleRateThrottle怎么实现的,分析部分请看注解:
SimpleRateThrottle类源码:
class SimpleRateThrottle(BaseThrottle):
"""
A simple cache implementation, that only requires `.get_cache_key()`
to be overridden.
The rate (requests / seconds) is set by a `rate` attribute on the View
class. The attribute is a string of the form 'number_of_requests/period'.
Period should be one of: ('s', 'sec', 'm', 'min', 'h', 'hour', 'd', 'day')
Previous request information used for throttling is stored in the cache.
"""
cache = default_cache # 存放请求时间,类似与示例中的大字典,这里使用的是django的缓存
timer = time.time
cache_format = 'throttle_%(scope)s_%(ident)s'
scope = None
THROTTLE_RATES = api_settings.DEFAULT_THROTTLE_RATES
def __init__(self):
if not getattr(self, 'rate', None):
self.rate = self.get_rate()
self.num_requests, self.duration = self.parse_rate(self.rate)
def get_cache_key(self, request, view):
# 获取请求的key标识,必须要有否则会报错,这里可以重写,使用用户的用户名、或其他作为key,在示例中使用的get_ident方法用户获取用户IP作为key
"""
Should return a unique cache-key which can be used for throttling.
Must be overridden.
May return `None` if the request should not be throttled.
"""
raise NotImplementedError('.get_cache_key() must be overridden')
def get_rate(self): # 获取配置文件的配置速率
"""
Determine the string representation of the allowed request rate.
"""
if not getattr(self, 'scope', None): # 通过获取共有属性scope来获取配置的速率
msg = ("You must set either `.scope` or `.rate` for '%s' throttle" %
self.__class__.__name__)
raise ImproperlyConfigured(msg)
try:
return self.THROTTLE_RATES[self.scope]
except KeyError:
msg = "No default throttle rate set for '%s' scope" % self.scope
raise ImproperlyConfigured(msg)
def parse_rate(self, rate): # 格式化速率
"""
Given the request rate string, return a two tuple of:
<allowed number of requests>, <period of time in seconds>
"""
if rate is None:
return (None, None)
num, period = rate.split('/') # 分离字符串
num_requests = int(num)
duration = {'s': 1, 'm': 60, 'h': 3600, 'd': 86400}[period[0]] # 转换时间为数字,示例配置的5/m,m转为60秒
return (num_requests, duration)
def allow_request(self, request, view): # 判断请求的速率是否通过
"""
Implement the check to see if the request should be throttled.
On success calls `throttle_success`.
On failure calls `throttle_failure`.
"""
if self.rate is None:
return True
self.key = self.get_cache_key(request, view)
if self.key is None:
return True
self.history = self.cache.get(self.key, [])
self.now = self.timer()
# Drop any requests from the history which have now passed the
# throttle duration
while self.history and self.history[-1] <= self.now - self.duration: # 频率判断实现原理,已经举例进行了说明
self.history.pop()
if len(self.history) >= self.num_requests:
return self.throttle_failure()
return self.throttle_success()
def throttle_success(self): # 频率通过返回true
"""
Inserts the current request's timestamp along with the key
into the cache.
"""
self.history.insert(0, self.now)
self.cache.set(self.key, self.history, self.duration)
return True
def throttle_failure(self): # 不通过返回false
"""
Called when a request to the API has failed due to throttling.
"""
return False
def wait(self): # 返回等待时间
"""
Returns the recommended next request time in seconds.
"""
if self.history:
remaining_duration = self.duration - (self.now - self.history[-1])
else:
remaining_duration = self.duration
available_requests = self.num_requests - len(self.history) + 1
if available_requests <= 0:
return None
return remaining_duration / float(available_requests)
get_ident方法源码,该方法用于获取请求的IP:
def get_ident(self, request):
"""
Identify the machine making the request by parsing HTTP_X_FORWARDED_FOR
if present and number of proxies is > 0. If not use all of
HTTP_X_FORWARDED_FOR if it is available, if not use REMOTE_ADDR.
"""
xff = request.META.get('HTTP_X_FORWARDED_FOR')
remote_addr = request.META.get('REMOTE_ADDR')
#这里request是封装以后的requst,django原生的是request._request.META 这样也可以获取
num_proxies = api_settings.NUM_PROXIES
if num_proxies is not None:
if num_proxies == 0 or xff is None:
return remote_addr
addrs = xff.split(',')
client_addr = addrs[-min(num_proxies, len(addrs))]
return client_addr.strip()
return ''.join(xff.split()) if xff else remote_addr
四:内置频率控制类
五:自定义频率控制
自定义频率控制无非实现request_allow方法和wait方法,你可以根据实际需求来定制你的频率控制
使用方法:
继承BaseThrottle类
重写request_allow方法和wait方法,request_allow方法返回true代表通过,否则拒绝,wait返回等待的时间
from rest_framework.throttling import BaseThrottle
import time
REQUEST_RECORD = {} # 访问记录,可使用nosql数据库
class VisitThrottle(BaseThrottle):
'''60s内最多能访问5次'''
def __init__(self):
self.history = None
def allow_request(self, request, view):
# 获取用户ip (get_ident)
remote_addr = self.get_ident(request)
ctime = time.time()
if remote_addr not in REQUEST_RECORD:
REQUEST_RECORD[remote_addr] = [ctime, ] # 保持请求的时间,形式{ip:[时间,]}
return True # True表示可以访问
# 获取当前ip的历史访问记录
history = REQUEST_RECORD.get(remote_addr)
self.history = history
while history and history[-1] < ctime - 60:
# while循环确保每列表中是最新的60秒内的请求
history.pop()
# 访问记录小于5次,将本次请求插入到最前面,作为最新的请求
if len(history) < 5:
history.insert(0, ctime)
return True
def wait(self):
'''返回等待时间'''
ctime = time.time()
return 60 - (ctime - self.history[-1])
全局配置:
REST_FRAMEWORK = {
#频率控制配置
"DEFAULT_THROTTLE_CLASSES":['utils.throttle.VisitThrottle'], #全局配置,
"DEFAULT_THROTTLE_RATES":{
'WD':'5/m', #速率配置每分钟不能超过5次访问,WD是scope定义的值
}
}
局部配置在视图类中添加:
##单一视图使用
throttle_classes = [VisitThrottle,]