一、canal数据同步(TODO 日后有了集群再做,先攒着)
cannal |
阿里巴巴的开源项目,纯java开发,基于数据库增量日志解析,体用增量数据订阅与消费,目前主要支持MySql |
原理主要基于mysql binlog技术,所以这里一定需要开启mysql的binlog写入功能 |
数据同步 |
在以往的数据存储中,我们都是通过接口的远程调用获取数据,这样耦合度较高,效率相对较低 |
比如统计注册人数,就需要远程调用会员微服务,会员表中的数据 |
而canal可以通过实时同步数据库表的方式来实现 |
比如我们统计每天注册人数,只需要将会员表同步到统计库中,本地统计即可,效率更高,耦合度低 |
开启相关用户权限(不开启据说用户远程访问不到,因为我没有集群,所以也只能听说) |
CREATE USER ‘canal’@’%’ IDENTIFIED BY ‘canal’; |
GRANT SHOW VIEW, SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON . TO ‘canal’@’%’; |
FLUSH PRIVILEGES; |
二、Spring Clous Gateway网关
我们一直在使用nginx的请求转发功能,现在这些可以用Gateway来实现 |
GATEWAY网关 |
和nginx一样,对外提供一个通用接口,然后根据路径帮我们转发 |
解决跨域 |
帮我们实现负载均衡 |
spring clous Gateway |
spring官方基于Spring 5.0、Spring Boot2.0和Project Reactor等技术开发的网关 |
目标是替代Netflix Zuul,其不仅提供统一的路由方式,并且还基于Filer链的方式提供了网关基本的功能,例如:安全、监控/埋点、限流等 |
Gateway术语 |
1)路由。路由是网关最基础的部分,路由信息有一个ID、一个目的URL、一组断言和一组Filter组成。如果断言路由为真,则说明请求的URL和配置匹配 |
2)断言。Java8中的断言函数。Spring Cloud Gateway中的断言函数输入类型是Spring5.0框架中的ServerWebExchange。Spring Cloud Gateway中的断言函数允许开发者去定义匹配来自于http request中的任何信息,比如请求头和参数等。 |
3)过滤器。一个标准的Spring webFilter。Spring cloud gateway中的filter分为两种类型的Filter,分别是Gateway Filter和Global Filter。过滤器Filter将会对请求和响应进行修改处理 |
1、环境搭建(通过配置文件配置路由实现请求转发)
1、创建模块并引入依赖(重点依赖冲突)
<!--引入自己的全局模块 需要使用统一返回结果-->
<dependency>
<groupId>com.yzpnb</groupId>
<artifactId>common_utils</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<!--nacos & spring cloud-->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<!--Spring Cloud Gateway-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
<version>2.2.3.RELEASE</version>
</dependency>
<!--gson-->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<!--feign 使用服务发现-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
<version>2.2.3.RELEASE</version>
</dependency>
因为gateway完全不兼容web依赖,所以我们不可以让spring-boot-starter-web依赖和spring-cloud-starter-gateway依赖处于同一个项目 |
我们只能修改项目的依赖,让需要使用spring-boot-starter-web的微服务单独引入 |
2、application.yml(重点:gateway的路由配置)
server:
port: 8222
spring:
application:
name: service-gateway
cloud:
nacos:
discovery:
server-addr: 127.0.0.1:8848
gateway:
discovery:
locator:
enabled: true
lower-case-service-id: true
routes:
- id: service-edu
uri: lb://service-edu
predicates:
- Path=/eduservice/**
- id: SERVICE-UCENTER
uri: lb://SERVICE-UCENTER
predicates:
- Path=/ucenter/**
- id: service-oss
uri: lb://service-oss
predicates:
- Path=/ossservice/**
注意配置文件中uri:lb://这里跟的是你在nacos中的注册名 |
3、启动类
4、启动nacos并测试
2、跨域
package com.yzpnb.gateway.config;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.gateway.discovery.DiscoveryClientRouteDefinitionLocator;
import org.springframework.cloud.gateway.discovery.DiscoveryLocatorProperties;
import org.springframework.cloud.gateway.route.RouteDefinitionLocator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.codec.support.DefaultServerCodecConfigurer;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.reactive.CorsUtils;
import org.springframework.web.cors.reactive.CorsWebFilter;
import org.springframework.web.cors.reactive.UrlBasedCorsConfigurationSource;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebFilter;
import org.springframework.web.server.WebFilterChain;
import org.springframework.web.util.pattern.PathPatternParser;
import reactor.core.publisher.Mono;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.reactive.CorsWebFilter;
import org.springframework.web.cors.reactive.UrlBasedCorsConfigurationSource;
import org.springframework.web.util.pattern.PathPatternParser;
@Configuration
public class CorsConfig {
@Bean
public CorsWebFilter corsFilter() {
CorsConfiguration config = new CorsConfiguration();
config.addAllowedMethod("*");
config.addAllowedOrigin("*");
config.addAllowedHeader("*");
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(new PathPatternParser());
source.registerCorsConfiguration("/**", config);
return new CorsWebFilter(source);
}
}
3、全局过滤器(全局Filter)
package com.yzpnb.gateway.filter;
import com.google.gson.JsonObject;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.stereotype.Component;
import org.springframework.util.AntPathMatcher;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;
import java.nio.charset.StandardCharsets;
import java.util.List;
@Component
public class AuthGlobalFilter implements GlobalFilter, Ordered {
private AntPathMatcher antPathMatcher = new AntPathMatcher();
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
String path = request.getURI().getPath();
if(antPathMatcher.match("/**/inner/**", path)) {
ServerHttpResponse response = exchange.getResponse();//不让其得到响应
return out(response);//报错
}
return chain.filter(exchange);
}
@Override
public int getOrder() {
return 0;
}
private Mono<Void> out(ServerHttpResponse response) {
JsonObject message = new JsonObject();
message.addProperty("success", false);
message.addProperty("code", 28004);
message.addProperty("data", "鉴权失败");
byte[] bits = message.toString().getBytes(StandardCharsets.UTF_8);
DataBuffer buffer = response.bufferFactory().wrap(bits);
//response.setStatusCode(HttpStatus.UNAUTHORIZED);
//指定编码,否则在浏览器中会中文乱码
response.getHeaders().add("Content-Type", "application/json;charset=UTF-8");
return response.writeWith(Mono.just(buffer));
}
}
4、全局异常(原生异常不太友好,我们通过覆盖,提高观赏度)
package com.yzpnb.gateway.handler;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.autoconfigure.web.ResourceProperties;
import org.springframework.boot.autoconfigure.web.ServerProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.web.reactive.error.ErrorAttributes;
import org.springframework.boot.web.reactive.error.ErrorWebExceptionHandler;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.web.reactive.result.view.ViewResolver;
import java.util.Collections;
import java.util.List;
@Configuration
@EnableConfigurationProperties({ServerProperties.class, ResourceProperties.class})
public class ErrorHandlerConfig {
private final ServerProperties serverProperties;
private final ApplicationContext applicationContext;
private final ResourceProperties resourceProperties;
private final List<ViewResolver> viewResolvers;
private final ServerCodecConfigurer serverCodecConfigurer;
public ErrorHandlerConfig(ServerProperties serverProperties,
ResourceProperties resourceProperties,
ObjectProvider<List<ViewResolver>> viewResolversProvider,
ServerCodecConfigurer serverCodecConfigurer,
ApplicationContext applicationContext) {
this.serverProperties = serverProperties;
this.applicationContext = applicationContext;
this.resourceProperties = resourceProperties;
this.viewResolvers = viewResolversProvider.getIfAvailable(Collections::emptyList);
this.serverCodecConfigurer = serverCodecConfigurer;
}
@Bean
@Order(Ordered.HIGHEST_PRECEDENCE)
public ErrorWebExceptionHandler errorWebExceptionHandler(ErrorAttributes errorAttributes) {
JsonExceptionHandler exceptionHandler = new JsonExceptionHandler(
errorAttributes,
this.resourceProperties,
this.serverProperties.getError(),
this.applicationContext);
exceptionHandler.setViewResolvers(this.viewResolvers);
exceptionHandler.setMessageWriters(this.serverCodecConfigurer.getWriters());
exceptionHandler.setMessageReaders(this.serverCodecConfigurer.getReaders());
return exceptionHandler;
}
}
package com.yzpnb.gateway.handler;
import org.springframework.boot.autoconfigure.web.ErrorProperties;
import org.springframework.boot.autoconfigure.web.ResourceProperties;
import org.springframework.boot.autoconfigure.web.reactive.error.DefaultErrorWebExceptionHandler;
import org.springframework.boot.web.reactive.error.ErrorAttributes;
import org.springframework.context.ApplicationContext;
import org.springframework.http.HttpStatus;
import org.springframework.web.reactive.function.server.*;
import java.util.HashMap;
import java.util.Map;
public class JsonExceptionHandler extends DefaultErrorWebExceptionHandler {
public JsonExceptionHandler(ErrorAttributes errorAttributes, ResourceProperties resourceProperties,
ErrorProperties errorProperties, ApplicationContext applicationContext) {
super(errorAttributes, resourceProperties, errorProperties, applicationContext);
}
@Override
protected Map<String, Object> getErrorAttributes(ServerRequest request, boolean includeStackTrace) {
Map<String, Object> map = new HashMap<>();
map.put("success", false);
map.put("code", 20005);
map.put("message", "网关失败");
map.put("data", null);
return map;
}
@Override
protected RouterFunction<ServerResponse> getRoutingFunction(ErrorAttributes errorAttributes) {
return RouterFunctions.route(RequestPredicates.all(), this::renderErrorResponse);
}
@Override
protected int getHttpStatus(Map<String, Object> errorAttributes) {
return 200;
}
}
5、完善,以及更改前后端配置
1、根据微服务在nacos中注册名将所有路由配置好
2、两种跨域二选一
3、前端请求端口修改为网关端口8222