实验跳转目录
反向代理:反向代理也叫reverse proxy,指的是代理外网用户的请求到内部的指定web服务器,并将数据返回给用户的一种方式,这是用的比较多的一种方式。
Nginx除了可以在企业提供高性能的web服务之外,另外还可以将本身不具备的请求通过某种预定义的协议转发至其它服务器处理,不同的协议就是Nginx服务器与其他服务器进行通信的一种规范,主要在不同的场景使用以下模块实现不同的功能:
ngx_http_proxy_module: 将客户端的请求以http协议转发至指定服务器进行处理 |
---|
ngx_stream_proxy_module:将客户端的请求以tcp协议转发至指定服务器处理 |
---|
ngx_http_fastcgi_module:将客户端对php的请求以fastcgi协议转发至指定服务器助理 |
---|
ngx_http_uwsgi_module:将客户端对Python的请求以uwsgi协议转发至指定服务器处理 |
---|
实验准备:
主机及IP | 用途 |
---|---|
U8 :192.168.124.30 | NGINX代理服务器 |
U8-1 :192.168.124.31 | 后端web A Apache部署 |
U8-2 :192.168.124.32 | 后端web B Apache部署 |
U8-3 :192.168.124.33 | 后端web C Apache部署 |
部署后端Apache服务器:
[root@U8-1 ~]# apt -y install apache2
[root@U8-1 ~]# echo “U8-1 web,192.168.124.31” > /var/www/html/index.html
[root@U8-1 ~]# systemctl start apache2
[root@U8-1 ~]# ss -ntulp|grep 80
tcp LISTEN 0 511 *:80 : users:((“apache2”,pid=2644,fd=4),(“apache2”,pid=2643,fd=4),(“apache2”,pid=2641,fd=4))
[root@U8-1 ~]# curl http://192.168.124.31
U8-1 web,192.168.124.31
#在U8-2上重复以上操作
[root@U8-2 ~]# apt -y install apache2
[root@U8-2 ~]# echo “U8-2 web,192.168.124.32” > /var/www/html/index.html
[root@U8-2 ~]# systemctl start apache2
[root@U8-2 ~]# ss -ntulp|grep 80
tcp LISTEN 0 511 *:80 : users:((“apache2”,pid=13011,fd=4),(“apache2”,pid=13010,fd=4),(“apache2”,pid=13008,fd=4))
[root@U8-2 ~]# curl http://192.168.124.32
U8-1 web,192.168.124.32
Nginx http 反向代理入门:
#反向代理配置参数:
proxy_pass;
#用来设置将客户端请求转发给的后端服务器的主机,可以是主机名、IP地址:端口的方式,也可以代理到预先设置的主机群组,需要模块gx_http_upstream_module支持。
location /web {
index index.html;
proxy_pass http://192.168.124.31:80;
#不带斜线将访问的/web,等于访问后端服务器 http://192.168.124.31:80/web/index.html 即后端服务器配置的站点根目录要有web目录才可以被访问,这是一个追加/web到后端服务器 http://servername:port/WEB/INDEX.HTML 的操作
proxy_pass http://192.168.124.31:80/;
#带斜线,等于访问后端服务器的http://192.168.124.31:80/index.html 内容返回给客户端
}
#重启Nginx测试访问效果:
# curl -L http://www.dushansao.com/web/index.html
proxy_hide_header field;
#用于nginx作为反向代理的时候,在返回给客户端http响应的时候,隐藏后端服务版本相应头部的信息,可以设置在http/server或location块,
location /web {
index index.html;
proxy_pass http://192.168.124.31:80/;
proxy_hide_header ETag;
}
proxy_pass_header field;
#默认nginx在响应报文中不传递后端服务器的首部字段Date, Server, X-Pad, X-Accel等参数,如果要传递的话则要使用 proxy_pass_header field声明将后端服务器返回的值传递给客户端。
proxy_pass_request_body on | off;
#是否向后端服务器发送HTTP包体部分,可以设置在http/server或location块,默认即为开启
proxy_pass_request_headers on | off;
#是否将客户端的请求头部转发给后端服务器,可以设置在http/server或location块,默认即为开启
proxy_set_header;
#可以更改或添加客户端的请求头部信息内容并转发至后端服务器,比如在后端服务器想要获取客户端的真实IP的时候,就要更改每一个报文的头部,如下:
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-For $remote_addr;
#添加HOST到报文头部,如果客户端为NAT上网那么其值为客户端的共用的公网IP地址,常用于在日之中记录客户端的真实IP地址。
proxy_connect_timeout time;
#配置nginx服务器与后端服务器尝试建立连接的超时时间,默认为60秒,用法如下:
proxy_connect_timeout 60s;
#60s为自定义nginx与后端服务器建立连接的超时时间
proxy_read_time time;
#配置nginx服务器向后端服务器或服务器组发起read请求后,等待的超时时间,默认60s
proxy_send_time time;
#配置nginx项后端服务器或服务器组发起write请求后,等待的超时时间,默认60s
proxy_http_version 1.0;
#用于设置nginx提供代理服务的HTTP协议的版本,默认http 1.0
proxy_ignore_client_abort off;
#当客户端网络中断请求时,nginx服务器中断其对后端服务器的请求。即如果此项设置为on开启,则服务器会忽略客户端中断并一直等着代理服务执行返回,如果设置为off,则客户端中断后Nginx也会中断客户端请求并立即记录499日志,默认为off。
proxy_headers_hash_bucket_size 128;
#当配置了 proxy_hide_header和proxy_set_header的时候,用于设置nginx保存HTTP报文头的hash表的上限。
proxy_headers_hash_max_size 512;
#设置proxy_headers_hash_bucket_size的最大可用空间
server_namse_hash_bucket_size 512;
#server_name hash表申请空间大小
server_names_hash_max_szie 512;
#设置服务器名称hash表的上限大小
反向代理示例–单台web服务器:
[root@U8: ~]# cd /apps/nginx/conf/conf.d/
[root@U8: /apps/nginx/conf/conf.d]# vim pc.conf
server {
listen 80;
server_name www.dushansao.com;
location / {
proxy_pass http://192.168.124.31:80/;
}
}
[root@U8: /apps/nginx/conf/conf.d]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: /apps/nginx/conf/conf.d]# /apps/nginx/sbin/nginx -s reload
[root@U8: ~]# echo “192.168.124.30 www.dushansao.com” >> /etc/hosts
[root@U8: ~]# curl http://www.dushansao.com
U8-1 web,192.168.124.31
反向代理示例–指定location:
[root@U8: /apps/nginx/conf/conf.d]# vim pc.conf
server {
listen 80;
server_name www.dushansao.com;
location / {
index index.html index.php;
root /data/nginx/html/pc;
}
location /web {
proxy_pass http://192.168.124.32/web/;
}
}
[root@U8: ~]# cat /data/nginx/html/pc/index.html
dushansao PC web
[root@U8: ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: ~]# /apps/nginx/sbin/nginx -s reload
#等下面的后端Apache服务器部署后再进行访问
[root@U8: ~]# curl http://www.dushansao.com
dushansao PC web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.32 web
#部署后端Apache服务器:
[root@U8-2 ~]# mkdir /var/www/html/web
[root@U8-2 ~]# echo “This is 192.168.124.32 web” > /var/www/html/web/index.html
[root@U8-2 ~]# tail -f /var/log/apache2/access.log
……………………………….
192.168.124.30 - - [15/Jan/2020:17:51:08 +0800] "GET /web// HTTP/1.0" 304 143 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
192.168.124.30 - - [15/Jan/2020:17:51:08 +0800] "GET /web// HTTP/1.0" 304 143 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
192.168.124.30 - - [15/Jan/2020:17:51:08 +0800] "GET /web// HTTP/1.0" 304 143 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
反向代理示例–缓存功能:
#缓存功能默认关闭状态
proxy_cache zone | off; 默认off
#指明调用的缓存,或关闭缓存机制;Context:http, server, location
proxy_cache_key string;
#缓存中用于“键”的内容,默认值:proxy_cache_key $scheme$proxy_host$request_uri;
proxy_cache_valid [code ...] time;
#定义对特定响应码的响应内容的缓存时长,定义在http{...}中
例:
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
proxy_cache_path;
定义可用于proxy功能的缓存;Context:http
proxy_cache_path path [levels=levels] [use_temp_path=on|off] keys_zone=name:size
[inactive=time] [max_size=size] [manager_files=number] [manager_sleep=time]
[manager_threshold=time] [loader_files=number] [loader_sleep=time]
[loader_threshold=time] [purger=on|off] [purger_files=number] [purger_sleep=time]
[purger_threshold=time];
例:在http配置定义缓存信息
proxy_cache_path /var/cache/nginx/proxy_cache #定义缓存保存路径,proxy_cache会自动创建
levels=1:2:2 #定义缓存目录结构层次,1:2:2可以生成2^4x2^8x2^8=1048576个目录
keys_zone=proxycache:20m #指内存中缓存的大小,主要用于存放key和metadata(如:使用次数)
inactive=120s; #缓存有效时间
max_size=1g; #最大磁盘占用空间,磁盘存入文件内容的缓存空间最大值
#调用缓存功能,需要定义在相应的配置段,如server{...};或者location等
proxy_cache proxycache;
proxy_cache_key $request_uri;
proxy_cache_valid 200 302 301 10m; #指定的状态码返回的数据缓存多长时间
proxy_cache_valid any 1m;
proxy_cache_use_stale error http_502 http_503;
#在被代理的后端服务器出现哪种情况下,可直接使用过期的缓存响应客户端,
proxy_cache_use_stale error | timeout | invalid_header | updating | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | off ; #默认是off
proxy_cache_methods GET | HEAD | POST ...;
#对哪些客户端请求方法对应的响应进行缓存,GET和HEAD方法总是被缓存
非缓存场景压测:
#准备一个文件
[root@U8-2 ~]# ll /var/www/html/web/index.html -h
-rw-r–r-- 1 root root 23K Jan 15 20:36 /var/www/html/web/index.html
[root@U8: ~]# ab -n 2000 -c200 http://www.dushansao.com/web/index.html
………………………………..
Total transferred: 47398000 bytes
HTML transferred: 46870000 bytes
Requests per second: 750.04 [#/sec] (mean)
Time per request: 266.654 [ms] (mean)
Time per request: 1.333 [ms] (mean, across all concurrent requests)
Transfer rate: 17358.52 [Kbytes/sec] received
……………………………………..
缓存配置:
[root@U8: ~]# vim /apps/nginx/conf/nginx.conf
#将其追加至主配置文件http中
proxy_cache_path /data/nginx/proxycache levels=1:1:1 keys_zone=proxycache:20m inactive=120s max_size=1g;
include mime.types;
default_type application/octet-stream;
[root@U8: ~]# vim /apps/nginx/conf/conf.d/pc.conf
server {
listen 80;
server_name www.dushansao.com;
location / {
index index.html index.php;
root /data/nginx/html/pc;
}
location /web {
proxy_pass http://192.168.124.32/web/;
proxy_set_header clientip $remote_addr;
proxy_cache proxycache;
proxy_cache_key $request_uri;
proxy_cache_valid 200 302 301 1h;
proxy_cache_valid any 1m;
}
}
[root@U8: ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: ~]# /apps/nginx/sbin/nginx -s reload
#访问后端web并验证缓存目录
[root@U8: ~]# curl http://www.dushansao.com/web
[root@U8: ~]# ab -n 2000 -c200 http://www.dushansao.com/web/index.html
………………………………….
Total transferred: 47398000 bytes
HTML transferred: 46870000 bytes
Requests per second: 2578.78 [#/sec] (mean)
Time per request: 77.556 [ms] (mean)
Time per request: 0.388 [ms] (mean, across all concurrent requests)
Transfer rate: 59682.10 [Kbytes/sec] received
………………………………………..
[root@U8: ~]# tree /data/nginx/proxycache/
/data/nginx/proxycache/
├── 4
│ └── 2
│ └── e
│ └── 3f6b9cf331e40eed5f95fc1a7dc7fe24
└── 8
└── 0
└── c
└── 21f470a48e953a199103a3a4c064ac08
6 directories, 2 files
#查看生成的临时缓存文件
[root@U8: ~]# cd /data/nginx/proxycache/4/2/e/
[root@U8: /data/nginx/proxycache/4/2/e]# ll
total 24K
-rw------- 1 nginx nginx 24K Jan 15 20:59 3f6b9cf331e40eed5f95fc1a7dc7fe24
[root@U8: /data/nginx/proxycache/4/2/e]# cd /data/nginx/proxycache/8/0/c/
[root@U8: /data/nginx/proxycache/8/0/c]# ll
total 24K
-rw------- 1 nginx nginx 24K Jan 15 20:59 21f470a48e953a199103a3a4c064ac08
[root@U8: ~]# head -n15 /data/nginx/proxycache/8/0/c/21f470a48e953a199103a3a4c064ac08
7Z¿v#
KEY: /web
XshellXshell OK
Date: Wed, 15 Jan 2020 13:02:57 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Wed, 15 Jan 2020 12:36:24 GMT
ETag: "5b8b-59c2cf4dbc9d1"
Accept-Ranges: bytes
Content-Length: 23435
Vary: Accept-Encoding
Connection: close
Content-Type: text/html
添加头部报文信息:
nginx基于模块ngx_http_headers_module可以实现对头部报文添加指定的key与值
Syntax: add_header name value [always];
Default: —
Context: http, server, location, if in location
#添加自定义首部,如下:
add_header name value [always];
add_header X-Via $server_addr;
add_header X-Cache $upstream_cache_status;
add_header X-Accel $server_name;
#添加自定义响应信息的尾部, 1.13.2版后支持
add_trailer name value [always];
[root@U8: ~]# vim /apps/nginx/conf/conf.d/pc.conf
server {
listen 80;
server_name www.dushansao.com;
location / {
index index.html index.php;
root /data/nginx/html/pc;
}
location /web {
proxy_pass http://192.168.124.32/web/;
proxy_set_header clientip $remote_addr;
proxy_cache proxycache;
proxy_cache_key $request_uri;
proxy_cache_valid 200 302 301 1h;
proxy_cache_valid any 1m;
add_header X-Via $server_addr;
add_header X-Cache $upstream_cache_status;
add_header X-Accel $server_name;
}
}
[root@U8: ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: ~]# /apps/nginx/sbin/nginx -s reload
[root@U8: ~]# curl -I http://www.dushansao.com/web
HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Wed, 15 Jan 2020 13:27:38 GMT
Content-Type: text/html
Content-Length: 23435
Connection: keep-alive
Last-Modified: Wed, 15 Jan 2020 12:36:24 GMT
ETag: "5b8b-59c2cf4dbc9d1"
Vary: Accept-Encoding
X-Via: 192.168.124.30
X-Cache: HIT
X-Accel: www.dushansao.com
Accept-Ranges: bytes
Nginx http 反向代理高级应用:
Nginx可以将客户端的请求转发至单台后端服务器但是无法转发至特定的一组的服务器,而且不能对后端服务器提供相应的服务器状态监测,但是Nginx可以基于ngx_http_upstream_module模块提供服务器分组转发、权重分配、状态监测、调度算法等高级功能,
http upstream配置参数:
upstream name {
}
#自定义一组服务器,配置在http内;
server address [parameters];
#配置一个后端web服务器,配置在upstream内,至少要有一个server服务器配置;
#server支持的parameters如下:
weight=number #设置权重,默认为1;
max_conns=number #给当前server设置最大活动链接数,默认为0表示没有限制;
max_fails=number #对后端服务器连续监测失败多少次就标记为不可用;
fail_timeout=time #对后端服务器的单次监测超时时间,默认为10秒;
backup #设置为备份服务器,当所有服务器不可用时将重新启用次服务器;
down #标记为down状态;
resolve #当server定义的是主机名的时候,当A记录发生变化会自动应用新IP而不用重启Nginx;
hash KEY consistent;
#基于指定key做hash计算,使用consistent参数,将使用ketama一致性hash算法,适用于后端是Cache服务器(如varnish)时使用,consistent定义使用一致性hash运算,一致性hash基于取模运算;
hash $request_uri consistent;
#基于用户请求的uri做hash;
ip_hash;
#源地址hash调度方法,基于的客户端的remote_addr(源地址)做hash计算,以实现会话保持;
least_conn;
#最少连接调度算法,优先将客户端请求调度到当前连接最少的后端服务器;
反向代理示例–多台web服务器:
[root@U8: ~]# vim /apps/nginx/conf/conf.d/pc.conf
upstream webserver {
server 192.168.124.31:80 weight=1 fail_timeout=5s max_fails=3; #后端服务器状态监测
server 192.168.124.32:80 weight=1 fail_timeout=5s max_fails=3;
server 192.168.124.33:80 weight=1 fail_timeout=5s max_fails=3 backup;
}
server {
listen 80;
server_name www.dushansao.com;
location / {
index index.html index.php;
root /data/nginx/html/pc;
}
location /web {
index index.html;
proxy_pass http://webserver/web/;
}
}
[root@U8: ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: ~]# /apps/nginx/sbin/nginx -s reload
#后端web
[root@U8-1 ~]# cat /var/www/html/web/index.html
This is 192.168.124.31 web
[root@U8-2 ~]# cat /var/www/html/web/index.html
This is 192.168.124.32 web
#增加新的虚拟机U8-3 192.168.124.33
apt -y install apache2
systemctl start apache2
mkdir /var/www/html/web
echo "This is 192.168.124.33 web" > /var/www/html/web/index.html
#访问后端web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.31 web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.32 web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.31 web
#关闭后端U8-1,U8-2 apache服务
[root@U8-1 ~]# systemctl stop apache2
[root@U8-1 ~]# ss -ntulp|grep 80
[root@U8-2 ~]# systemctl stop apache2
[root@U8-2 ~]# ss -ntulp|grep 80
#测试U8-3 backup服务器可用性:
#NGINX代理服务器访问后端web
[root@U8: ~]# while true;do curl http://www.dushansao.com/web;sleep 1;done
This is 192.168.124.33 web
This is 192.168.124.33 web
This is 192.168.124.33 web
^C
反向代理示例–客户端IP透传:
[root@U8: ~]# vim /apps/nginx/conf/conf.d/pc.conf
upstream webserver {
server 192.168.124.32:80 weight=1 fail_timeout=5s max_fails=3;
server 192.168.124.33:80 weight=1 fail_timeout=5s max_fails=3 backup;
}
server {
listen 80;
server_name www.dushansao.com;
location / {
index index.html index.php;
root /data/nginx/html/pc;
}
location /web {
index index.html;
proxy_pass http://webserver/web/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; #添加客户端IP到报文头部
}
}
[root@U8: ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@U8: ~]# /apps/nginx/sbin/nginx -s reload
#后端web服务器配置;U8-3,U8-2
#找到如下一行并添加
[root@U8-3 ~]# vim /etc/apache2/apache2.conf
LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
[root@U8-3 ~]# systemctl restart apache2
#在U8-2上重复以上操作
[root@U8-2 ~]# vim /etc/apache2/apache2.conf
LogFormat “%{X-Forwarded-For}i %h %l %u %t “%r” %>s %O “%{Referer}i” “%{User-Agent}i”” combined
[root@U8-2 ~]# systemctl restart apache2
#重启apache后访问web界面并验证apache日志:
#NGINX代理服务器访问web
[root@U8: ~]# vim /apps/ncurl http://www.dushansao.com/web
This is 192.168.124.32 web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.32 web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.32 web
#后端web服务器监控日志
[root@U8-2 ~]# > /var/log/apache2/access.log
[root@U8-2 ~]# tail -f /var/log/apache2/access.log
192.168.124.30 192.168.124.30 - - [16/Jan/2020:08:51:37 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "curl/7.58.0"
192.168.124.30 192.168.124.30 - - [16/Jan/2020:08:51:43 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "curl/7.58.0"
192.168.124.30 192.168.124.30 - - [16/Jan/2020:08:52:08 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "curl/7.58.0"
…………………………………
[root@U8-3 ~]# > /var/log/apache2/access.log
[root@U8-3 ~]# tail -f /var/log/apache2/access.log
#由以上观察出U8-3的日志并无内容,因为该端被设为了backup服务端
#只有当其他后端web服务器全部无法访问时,U8-3才会生效
#停止U8-2的Apache2服务,清空并继续监控日志
[root@U8-2 ~]# systemctl stop apache2
[root@U8-2 ~]# > /var/log/apache2/access.log
[root@U8-2 ~]# tail -f /var/log/apache2/access.log
#NGINX代理服务器继续访问后端web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.33 web
[root@U8: ~]# curl http://www.dushansao.com/web
This is 192.168.124.33 web
#此时U8-2已经没有新的日志生成,而U8-3则开始启用
[root@U8-3 ~]# tail -f /var/log/apache2/access.log
192.168.124.30 192.168.124.30 - - [16/Jan/2020:08:58:30 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "curl/7.58.0"
192.168.124.30 192.168.124.30 - - [16/Jan/2020:09:01:01 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "curl/7.58.0"
192.168.124.30 192.168.124.30 - - [16/Jan/2020:09:01:02 +0800] "GET /web/ HTTP/1.0" 200 273 "-" "