1.搭建一台代理服务器
2.HttpClient方式如下:
- 引入包:
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.9</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.11</version>
</dependency>
- main方法如下,可自己封装:
public static void main(String[] args) throws IOException {
//创建Httpget实例 ,http://2018.ip138.com/ic.asp为该网址返回对应的ip
//以下为要访问的网址
HttpGet httpGet = new HttpGet("http://www.baidu.com");
//代理IP设置,代理 ip查询地址:https://www.xicidaili.com/
HttpHost httoHost = new HttpHost("代理服务器ip",端口号);
// 设置认证
CredentialsProvider provider = new BasicCredentialsProvider();
provider.setCredentials(new AuthScope(httoHost), new UsernamePasswordCredentials("username", "password"));
CloseableHttpClient httpClient = HttpClients.custom().setDefaultCredentialsProvider(provider).build();
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(10000)//设置连接超时时间,单位毫秒
.setSocketTimeout(10000)//设置读取超时时间,单位毫秒
.setProxy(httoHost)//设置代理
.build();
httpGet.setConfig(requestConfig);
//设置Http报文头信息
httpGet.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0");
CloseableHttpResponse response = httpClient.execute(httpGet); // 执行http get请求
if (response != null){
HttpEntity entity = response.getEntity(); //获取返回实体
if (entity != null){
System.out.println("网页内容为:");
System.out.println(EntityUtils.toString(entity,"gbk"));
}
}
if (response != null){
response.close();
}
if (httpClient != null){
httpClient.close();
}
}
- restTemplate方式设置代理
@Bean
RestTemplate createRestTemplate() throws Exception {
final String username = "xxxx";
final String password = "xxxx";
final String proxyUrl = "代理服务器ip或域名";
final int port = 端口号;
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
new AuthScope(proxyUrl, port),
new UsernamePasswordCredentials(username, password));
HttpHost myProxy = new HttpHost(proxyUrl, port);
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
clientBuilder.setProxy(myProxy).setDefaultCredentialsProvider(credsProvider).disableCookieManagement();
HttpClient httpClient = clientBuilder.build();
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setHttpClient(httpClient);
return new RestTemplate(factory);
}
- URLConnection方式使用代理网络访问
public static void main(String[] args) throws IOException { // 设置代理服务器 InetSocketAddress addr = new InetSocketAddress("代理服务器ip",8080); Proxy proxy = new Proxy(Proxy.Type.HTTP, addr); // http 代理 //配置帐号密码信息 Authenticator authenticator = new Authenticator() { @Override protected PasswordAuthentication getPasswordAuthentication() { return new PasswordAuthentication("userName","password".toCharArray()); } }; Authenticator.setDefault(authenticator); URLConnection connection = new URL("http://www.baidu.com").openConnection(proxy); InputStream inputStream = connection.getInputStream(); ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(); int i; while ((i = inputStream.read()) != -1){ byteArrayOutputStream.write(i); } String s = byteArrayOutputStream.toString(); System.out.println("..."); }
PS:本文参考了其他多篇文章进行的总结,已经过测试,肯定能跑通,如果你的代理服务器搭建好了的话,写篇博客,怕忘了