一.首先，下载5个Struts2核心jar包：
commons-logging-1.1.1.jar
freemarker-2.3.15.jar
ognl-2.7.3.jar
struts2-core-2.1.8.1.jar
xwork-core-2.1.6.jar
二.然后创建Web Project项目，创建完后的项目结构如下
三，引入jar包
1、复制下载好的5个Struts2核心jar包，粘贴到项目的lib目录下
2、右键点击项目，分别点击【Build Path】à【Configure Build Path…】à【Libraries】à【Add JARs…】，选中项目的lib下的5个Struts2核心jar包，点击确定，把包引入到项目中，点击【OK】即可。
四。修改web.xml文件 ，加入这2句话
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
<filter>
<filter-name>struts2</filter-name>
<filter-class>
org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter
</filter-class>
</filter>
<filter-mapping>
<filter-name>struts2</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-app>
5.例子:自己写一个jsp页面，struts2可以用实体类，也可以不用，用ajax的data传值就行
<%@ page contentType="text/html;charset=UTF-8" language="java"
import="java.util.*" pageEncoding="UTF-8"%>
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<title>首页</title>
<style>
.clear {
clear: both;
}
</style>
<jsp:include page="header.jsp"></jsp:include>
</head>
<body>
<form method="post">
<div class="container" style="width: 100%;">
<div id="mai2" style="height: 50px; width: 100%; margin-top: 15px;"
class="btn-group btn-group-justified">
<a class="btn btn-info" style="width: 10%">频道入口测试</a> <a
href="jsp/templatetools.jsp" class="btn btn-info">返回</a>
</div>
<div class="panel panel-default">
<div class="panel-body" style="width: 99%;">
<table>
<tr>
<td><span style="color: red;">*</span>网站名称：</td>
<td><input type="text" class="form-control" id="site"
name="site" />
</td>
<td> <span style="color: red;">*</span>频道名称：</td>
<td> <select style="width: 150px" id="category" name="category">
<option value="26">国内</option>
<option value="30">国际</option>
<option value="34">财经</option>
<option value="37">地产</option>
<option value="58">能源</option>
<option value="59">教育</option>
<option value="60">科技</option>
<option value="61">环境</option>
<option value="62">汽车</option>
<option value="63">交通</option>
<option value="64">钢铁</option>
<option value="65">文化</option>
<option value="66">旅游</option>
<option value="263">食品</option>
<option value="307">农业</option>
<option value="314">体育</option>
<option value="825">军事</option>
<option value="827">宗教</option>
<option value="829">搜索</option>
<option value="830">论坛</option>
<option value="831">医疗</option>
</select>
</td>
<td> <span style="color: red;">*</span>模板编辑者：</td>
<td><input type="text" class="form-control" id="editor"
name="editor" />
</td>
<td align="right" height="50px"> 网页编码：</td>
<td><select id="encode" name="encode">
<option value="UTF-8">UTF-8</option>
<option value="gbk">GBK</option>
<option value="gb">GB2312</option>
<option value="iso">ISO-8859-1</option>
</select>
</td>
</tr>
</table> <table>
<tr>
<td align="right"><span style="color: red;">*</span>入口地址：</td>
<td><input type="text" class="form-control" size="50%"
id="base" name="base"/></td>
<td><input type="button" value="下载源码" onclick="getsource();" id="dowsour"/>
</td>
<td><input type="button" value="查询" onclick="Inquiry();" />
</td>
<td align="right"><span style="color: red;">*</span>链接匹配URL：</td>
<td><input type="text" class="form-control" size="47%"
id="regurl" name="regurl" /></td>
</tr>
</table>
<table>
<tr>
<td><span style="color: red;">*</span>块抽取：</td>
<td><select style="height: 27px" id="containerkey1"
name="containerkey1">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" size="43%" class="form-control"
id="containervalue1" name="containervalue1" />
</td>
<td><select style="height: 27px" id="containerkey2"
name="containerkey2">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" size="43%" class="form-control"
id="containervalue2" name="containervalue2" />
</td>
<td><input type="button" value="测试" onclick="getcontainer();" />
</td>
</tr>
<tr>
<td><span style="color: red;">*</span>链接抽取：</td>
<td><select style="height: 27px" id="urlkey1" name="urlkey1">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" class="form-control" id="urlvalue1"
name="urlvalue1" />
</td>
<td><select style="height: 27px" id="urlkey2" name="urlkey2">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" class="form-control" id="urlvalue2"
name="urlvalue2" />
</td>
<td><input type="button" value="测试" onclick="geturl();" />
</td>
</tr>
<tr>
<td>分页匹配规则：</td>
<td><select style="height: 27px" id="detailedkey1"
name="detailedkey1">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" class="form-control"
id="detailedvalue1" name="detailedvalue1" />
</td>
<td><select style="height: 27px" id="detailedkey2"
name="detailedkey2">
<option value="dom">DOM</option>
<option value="reg">正则表达式</option>
<option value="xpath">Xpath</option>
<option value="http">httpwatch</option>
<option value="value">常量</option>
</select></td>
<td><input type="text" class="form-control"
id="detailedvalue2" name="detailedvalue2" />
</td>
<td><input type="button" value="测试" onclick="getdetailed();" />
</td>
</tr>
</table>
<table>
<tr>
<td align="left">源码：</td>
<td align="left" style="padding-left: 20px">结果：</td>
</tr>
<tr>
<td align="right"><textarea
style="border:solid 1px #f00;border-color: #dddddd;" rows="15%"
cols="60%" id="downloadSource" name="downloadSource" readonly="readonly"
class="form-control"></textarea></td>
<td style="padding-left: 20px"><textarea
style="border:solid 1px #f00;border-color: #dddddd;" rows="15%"
cols="60%" id="crawlDomin" name="crawlDomin"
class="form-control"></textarea>
</td>
</tr>
</table>
<div class="form-group">
<div class="col-lg-10 col-lg-offset-2">
<button type="button" class="btn btn-primary" style="margin-left: 300px;"
onclick="getcateTemplate(0);" id="total">总测试</button>
<button type="button" class="btn btn-primary" onclick="getcateTemplate(1);">添加</button>
<button type="reset" class="btn btn-primary">重填</button>
</div>
</div>
</div>
</div>
</div>
<input type="hidden" name="sour" id="sour" >
<jsp:include page="/jsp/footer.jsp"></jsp:include>
</form> </body>
里面会嵌入js方法，data值 前边是action的属性，后面是页面的属性
function getsource() {
//获得入口地址
var base = document.getElementById("base").value;
if (base == null || base == "") {
alert("入口地址不能为空！");
return;
}
$("#dowsour").attr("disabled","disabled");//这是点击按钮的时候按钮变灰，不能在点击
$.ajax({
type : "post",
url : "getsource.action",
data : {
"base" : base
},
dataType : "json",
success : function(msg) {
var rawHtml = msg.rawHtml;
document.getElementById("downloadSource").value = rawHtml;
document.getElementById("sour").value = rawHtml;
$("#dowsour").removeAttr("disabled");//恢复点击的状态
}
});
}
function getcontainer() {
//获取块抽取key值和value值
var containerkey1 = document.getElementById("containerkey1").value;
var containervalue1 = document.getElementById("containervalue1").value;
var containerkey2 = document.getElementById("containerkey2").value;
var containervalue2 = document.getElementById("containervalue2").value;
var sour=document.getElementById("sour").value;
if ((containervalue1 == null || containervalue1 == "")
&& (containervalue2 == null || containervalue2 == "")) {
alert("块抽取不能为空！");
return;
}
if(sour==null||sour==""){
alert("请先下载源码");
return;
}
$.ajax({
type : "post",
url : "getcontainer.action",
data : {
"containerkey1" : containerkey1,
"containervalue1" : containervalue1,
"containerkey2" : containerkey2,
"containervalue2" : containervalue2,
"sour":sour
},
dataType : "json",
success : function(msg) {
var data = msg.data;
var resData = "块抽取测试:\n";
for ( var i = 0; i < data.length; i++) {
if (data[i].firstSourceText != "") {
resData += data[i].firstSourceText
+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
} else {
resData += data[i].firstSourceText
+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
}
}
var end = resData.lastIndexOf("\n");
resData = resData.substring(0, end);
document.getElementById("crawlDomin").value = resData;
}
});
}
function geturl() {
//获取链接key值和value值
var urlkey1 = document.getElementById("urlkey1").value;
var urlkey2 = document.getElementById("urlkey2").value;
var urlvalue1 = document.getElementById("urlvalue1").value;
var urlvalue2 = document.getElementById("urlvalue2").value;
var containerkey1 = document.getElementById("containerkey1").value;
var containerkey2 = document.getElementById("containerkey2").value;
var containervalue1 = document.getElementById("containervalue1").value;
var containervalue2 = document.getElementById("containervalue2").value;
var sour=document.getElementById("sour").value;
if ((urlvalue1 == null || urlvalue1 == "")
&& (urlvalue2 == null || urlvalue2 == "")) {
alert("抽取链接不能为空！");
return;
}
if ((containervalue1 == null || containervalue1 == "")
&& (containervalue2 == null || containervalue2 == "")) {
alert("请填写块抽取");
return;
}
if(sour==null||sour==""){
alert("请先下载源码");
return;
}
$.ajax({
type : "post",
url : "getcontainer.action",
data : {
"urlkey1" : urlkey1,
"urlkey2" : urlkey2,
"urlvalue1" : urlvalue1,
"urlvalue2" : urlvalue2,
"containervalue1":containervalue1,
"containervalue2":containervalue2,
"containerkey1":containerkey1,
"containerkey2":containerkey2,
"sour":sour
},
dataType : "json",
success : function(msg) {
var data = msg.data;
var resData = "链接抽取:\n";
for ( var i = 0; i < data.length; i++) {
if (data[i]!= "") {
resData += data[i]+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
} else {
resData += data[i]+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
}
}
var end = resData.lastIndexOf("\n");
resData = resData.substring(0, end);
document.getElementById("crawlDomin").value = resData;
}
});
}
function getdetailed() {
//获取分页匹配规则
var detailedkey1 = document.getElementById("detailedkey1").value;
var detailedvalue1 = document.getElementById("detailedvalue1").value;
var detailedkey2 = document.getElementById("detailedkey2").value;
var detailedvalue2 = document.getElementById("detailedvalue2").value;
var sour=document.getElementById("sour").value;
if ((detailedvalue1 == null || detailedvalue1 == "")
&& (detailedvalue2 == null || detailedvalue2 == "")) {
alert("分页匹配规则不能为空！");
return;
}
if(sour==null||sour==""){
alert("请先下载源码");
return;
}
$.ajax({
type : "post",
url : "getcontainer.action",
data : {
"detailedkey1" : detailedkey1,
"detailedvalue1" : detailedvalue1,
"detailedkey2" : detailedkey2,
"detailedvalue2" : detailedvalue2,
"sour":sour
},
dataType : "json",
success : function(msg) {
var data = msg.data;
var resData = "分页匹配规则:\n";
for ( var i = 0; i < data.length; i++) {
if (data[i]!= "") {
resData += data[i]+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
} else {
resData += data[i]+ "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
}
}
var end = resData.lastIndexOf("\n");
resData = resData.substring(0, end);
document.getElementById("crawlDomin").value = resData;
}
});
}
function getcateTemplate(obj) {
//获取网站名称
var site = document.getElementById("site").value;
if(site==null||site==""){
alert("网站名称不能为空");
return;
}
//获取频道名称
var category = document.getElementById("category").value;
if(category==null||category==""){
alert("二级频道不能为空");
return;
}
//获取模板编辑者
var editor = document.getElementById("editor").value;
if(editor==null||editor==""){
alert("模板编辑不能为空");
return;
}
//获取编码
var encode = document.getElementById("encode").value;
//获取入口地址
var base = document.getElementById("base").value;
if(base==null||base==""){
alert("入口地址不能为空");
return;
}
//获取块抽取key值和value值
var containerkey1 = document.getElementById("containerkey1").value;
var containervalue1 = document.getElementById("containervalue1").value;
var containerkey2 = document.getElementById("containerkey2").value;
var containervalue2 = document.getElementById("containervalue2").value;
if ((containervalue1 == null || containervalue1 == "")
&& (containervalue2 == null || containervalue2 == "")) {
alert("块抽取不能为空");
return;
}
//获取入口地址匹配的url
var regurl = document.getElementById("regurl").value;
if(regurl==null||regurl==""){
alert("链接匹配url不能为空");
return;
}
//抽取链接匹配
var urlkey1 = document.getElementById("urlkey1").value;
var urlkey2 = document.getElementById("urlkey2").value;
var urlvalue1 = document.getElementById("urlvalue1").value;
var urlvalue2 = document.getElementById("urlvalue2").value;
if ((urlvalue1 == null || urlvalue1 == "")
&& (urlvalue2 == null || urlvalue2 == "")) {
alert("链接匹配不能为空");
return;
}
//获取分页匹配规则
var detailedkey1 = document.getElementById("detailedkey1").value;
var detailedvalue1 = document.getElementById("detailedvalue1").value;
var detailedkey2 = document.getElementById("detailedkey2").value;
var detailedvalue2 = document.getElementById("detailedvalue2").value;
if(obj==0){
$("#total").attr("disabled","disabled");
$.ajax({
type : "post",
url : "categoryTemplate.action",
data : {
"site" : site,
"category" : category,
"editor" : editor,
"encode" : encode,
"base" : base,
"containerkey1" : containerkey1,
"containervalue1" : containervalue1,
"containervalue2" : containervalue2,
"containerkey2" : containerkey2,
"regurl" : regurl,
"urlkey2" : urlkey2,
"urlkey1" : urlkey1,
"urlvalue1" : urlvalue1,
"urlvalue2" : urlvalue2,
"detailedkey1" : detailedkey1,
"detailedvalue1" : detailedvalue1,
"detailedkey2" : detailedkey2,
"detailedvalue2" : detailedvalue2
},
dataType : "json",
success : function(msg) {
var data = msg.data;
var resData = "全部测试:\n";
for ( var i = 0; i < data.length; i++) {
if (data[i].url == "") {
resData += "链接抽取：\n" + null;
} else {
resData += "链接抽取：\n" + data[i].url;
}
resData += "\n- - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n";
}
var end = resData.lastIndexOf("\n");
resData = resData.substring(0, end);
document.getElementById("crawlDomin").value = resData;
$("#total").removeAttr("disabled");
}
});
}
if(obj==1){
$.ajax({
type : "post",
url : "addbase.action",
data : {
"site" : site,
"category" : category,
"editor" : editor,
"encode" : encode,
"base" : base,
"containerkey1" : containerkey1,
"containervalue1" : containervalue1,
"containervalue2" : containervalue2,
"containerkey2" : containerkey2,
"regurl" : regurl,
"urlkey2" : urlkey2,
"urlkey1" : urlkey1,
"urlvalue1" : urlvalue1,
"urlvalue2" : urlvalue2,
"detailedkey1" : detailedkey1,
"detailedvalue1" : detailedvalue1,
"detailedkey2" : detailedkey2,
"detailedvalue2" : detailedvalue2
},
dataType : "json",
success : function(msg) {
var code=msg.code;
if(code==200){
alert("添加成功");
}else if(code==1){
alert("匹配地址错误，添加失败！");
}else{
alert("添加失败");
}
}
});
}
}
function Inquiry(){
//获得入口地址
var base = document.getElementById("base").value;
if (base == null || base == "") {
alert("入口地址不能为空！");
return;
}
$.ajax({
type : "post",
url : "query.action",
data : {
"base" : base
},
dataType : "json",
success : function(msg) {
var flag=msg.flag;
if(flag==1){
alert("网站已经存在");
qingkong();
//获取网站名称
var site=msg.template.fields.site.rule;
for(var i=0;i<site.length;i++){
var re=site[i];
var start=re.indexOf(":");
var value=re.substring(start+1);
document.getElementById("site").value =value;
}
//获取编码
var encode=msg.template.option.encode;
document.getElementById("encode").value =encode;
//获取频道名称
var category=msg.template.fields.category.rule;
for(var i=0;i<category.length;i++){
var re=category[i];
var start=re.indexOf(":");
var value=re.substring(start+1);
document.getElementById("category").value =value;
}
//获取模板编辑者
var editor=msg.template.fields.editor.rule;
for(var i=0;i<editor.length;i++){
var re=editor[i];
var start=re.indexOf(":");
var value=re.substring(start+1);
document.getElementById("editor").value =value;
}
//获取链接匹配
var regurl=msg.regurl;
document.getElementById("regurl").value =regurl;
//获取链接Url
var url=msg.template.fields.url.rule;
for( var i = 0; i < url.length; i++){
var re=url[i];
var start=re.indexOf(":");
var key=re.substring(0,start);
var value=re.substring(start+1);
var keyId="urlkey"+(i+1);
var valId="urlvalue"+(i+1);
document.getElementById(keyId).value =key;
document.getElementById(valId).value =value;
}
//获取块
var container=msg.template.container.rule;
for( var i = 0; i < container.length; i++){
var re=container[i];
var start=re.indexOf(":");
var key=re.substring(0,start);
var value=re.substring(start+1);
var keyId="containerkey"+(i+1);
var valId="containervalue"+(i+1);
document.getElementById(keyId).value =key;
document.getElementById(valId).value =value;
}
//获取分页匹配
var rules = msg.template.follow_rules["#1"].rule;
for( var i = 0; i < rules.length; i++){
var re=rules[i];
var start=re.indexOf(":");
var key=re.substring(0,start);
var value=re.substring(start+1);
var keyId="detailedkey"+(i+1);
var valId="detailedvalue"+(i+1);
document.getElementById(keyId).value =key;
document.getElementById(valId).value =value;
}
}else {
alert("网站不存在");
qingkong();
}
}
});
}
function qingkong(){
document.getElementById("site").value ="";
document.getElementById("category").value ="26";
document.getElementById("editor").value ="";
document.getElementById("regurl").value ="";
document.getElementById("detailedkey1").value ="dom";
document.getElementById("detailedkey2").value ="dom";
document.getElementById("detailedvalue1").value ="";
document.getElementById("detailedvalue2").value ="";
document.getElementById("containerkey1").value ="dom";
document.getElementById("containerkey2").value ="dom";
document.getElementById("containervalue1").value ="";
document.getElementById("containervalue2").value ="";
document.getElementById("urlkey1").value ="dom";
document.getElementById("urlkey2").value ="dom";
document.getElementById("urlvalue1").value ="";
document.getElementById("urlvalue2").value ="";
document.getElementById("encode").value ="UTF-8";
}
在创建action,这里面的字段就相当于实体类里的属性值了
package com.geedata.temp.action; import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.struts2.ServletActionContext;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.geedata.temp.server.Template;
import com.opensymphony.xwork2.ActionSupport; public class TemplateAction extends ActionSupport { private static final long serialVersionUID = 1L;
private String regurl = "";// url链接匹配
private String detailedkey1 = "";// 详细页，分页规则
private String detailedkey2 = "";
private String detailedvalue1 = "";
private String detailedvalue2 = "";
private String base = "";// 入口地址
private String keywords = "";// 关键词
private String beginPage;// 开始页数
private String factor;// 每页条数
private String pages;// 采集页数
private String urlencode = "";// 关键字编码格式
private String encode = "";// 页面采集编码格式
private boolean enablelist;// 是否采集列表页
private boolean enablePagination;// 是否对分页进行采集
private boolean enableBlock;// 是否配置快内容
private String method = "";// 传输方式
private String containerkey1 = "";// 块抽取
private String containerkey2 = "";
private String containervalue1 = "";
private String containervalue2 = "";
private String titlekey1 = "";// 标题
private String titlekey2 = "";
private String titlevalue1 = "";
private String titlevalue2 = "";
private String urlkey1 = "";// 获取链接，链接抽取
private String urlkey2 = "";
private String urlvalue1 = "";
private String urlvalue2 = "";
private String sourcekey1 = "";// 来源
private String sourcekey2 = "";
private String sourcevalue1 = "";
private String sourcevalue2 = "";
private String browsekey1 = "";// 浏览数
private String browsekey2 = "";
private String browsevalue1 = "";
private String browsevalue2 = "";
private String commentkey1 = "";// 评论数
private String commentkey2 = "";
private String commentvalue1 = "";
private String commentvalue2 = "";
private String authorkey1 = "";// 作者
private String authorkey2 = "";
private String authorvalue1 = "";
private String authorvalue2 = "";
private String contentkey1 = "";// 内容
private String contentkey2 = "";
private String contentvalue1 = "";
private String contentvalue2 = "";
private String ptimekey1 = "";// 发布时间
private String ptimekey2 = "";
private String ptimevalue1 = "";
private String ptimevalue2 = "";
private String summarykey1 = "";// 摘要
private String summarykey2 = "";
private String summaryvalue1 = "";
private String summaryvalue2 = "";
private String type = "";// 网站类别
private String editor = "";// 编辑者
private String site = "";// 网站名称
private String category = "";// 频道
private String results = "";// 结果
private String downloadSource = "";// 下载源码
private String sour="";// 源码集
private String icp="";// icp备案号
private String reprintedkey1="";//转载数
private String reprintedkey2="";
private String reprintedvalue1="";
private String reprintedvalue2="";
private String otherkey1="";//其它
private String otherkey2="";
private String othervalue1="";
private String othervalue2="";
private String filter="";//过滤 /**
* 查询方法
*
* @throws IOException
*/
public String query() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
HttpServletRequest request = ServletActionContext.getRequest();
response.setContentType("text/json;charset=gb2312");
if (!"".equals(base)) {
Template tem = new Template();
JSONObject json = tem.checkTemplate(request, base);
if (json != null) {
json.put("flag", 1);
String data = json.toString();
response.getWriter().write(data);
} else {
response.getWriter().write("{\"flag\":\"0\"}");
}
} return null;
} /**
* 获取关键词源码
*
* @throws IOException
*/
public String getkeysource() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
Template tem = new Template();
if("".equals(factor)||factor.equals(null)){
factor="";
}
String temp = "{\"urls\": {\"params\": {\"keywords\": \"" + keywords
+ "\",\"beginPage\": \"" + beginPage + "\"," + "\"factor\": \""
+ factor + "\",\"pages\": \"1\",\"urlencode\": \"utf-8\"}}}";
JSONObject getReturnData = tem.downloadRawhHtml(temp, base.trim());
String data = getReturnData.toString();
response.getWriter().write(data);
return null; } /**
* 获取内容页和频道源码
*
* @throws IOException
*/
public String getsource() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
Template tem = new Template();
JSONObject getReturnData = tem.downloadRawhHtml("",
base.replace("\\", "\\\\").replace("\"", "\\\""));
String data = getReturnData.toString();
response.getWriter().write(data); return null;
} /**
* 获取所有单点测试
*
* @throws IOException
*/
public String getcontainer() throws IOException {
JSONObject json = new JSONObject();
String containerrules = "";
String containerrules1 = "";
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
JSONArray jsonArray = new JSONArray();
JSONArray jsonArray2 = new JSONArray();
/**
* 判断块抽取
*/
if (!"".equals(containervalue1) && !"".equals(containervalue2)) {
containerrules = containerkey1 + ":" + containervalue1;
containerrules1 = containerkey2 + ":" + containervalue2;
jsonArray2.add(containerrules);
jsonArray2.add(containerrules1);
} else if (!"".equals(containervalue1) && "".equals(containervalue2)) {
containerrules = containerkey1 + ":" + containervalue1;
jsonArray2.add(containerrules);
} else if ("".equals(containervalue1) && !"".equals(containervalue2)) {
containerrules = containerkey2 + ":" + containervalue2;
jsonArray2.add(containerrules);
} jsonArray = point(urlkey1, urlvalue1, urlkey2, urlvalue2, jsonArray);// 链接
jsonArray = point(detailedkey1, detailedvalue1, detailedkey2,
detailedvalue2, jsonArray);// 分页
jsonArray = point(titlekey1, titlevalue1, titlekey2, titlevalue2,
jsonArray);// 标题
jsonArray = point(authorkey1, authorvalue1, authorkey2, authorvalue2,
jsonArray);// 作者
jsonArray = point(sourcekey1, sourcevalue1, sourcekey2, sourcevalue2,
jsonArray);// 来源
jsonArray = point(browsekey1, browsevalue1, browsekey2, browsevalue2,
jsonArray);// 浏览数
jsonArray = point(commentkey1, commentvalue1, commentkey2,
commentvalue2, jsonArray);// 评论数
jsonArray = point(reprintedkey1, reprintedvalue1, reprintedkey2,
reprintedvalue2, jsonArray);// 转载数
jsonArray = point(ptimekey1, ptimevalue1, ptimekey2, ptimevalue2,
jsonArray);// 发布时间
jsonArray = point(contentkey1, contentvalue1, contentkey2,
contentvalue2, jsonArray);// 判断原文内容
jsonArray = point(summarykey1, summaryvalue1, summarykey2,
summaryvalue2, jsonArray);// 原文摘要
jsonArray = point(otherkey1, othervalue1, otherkey2,
othervalue2, jsonArray);//其它 json.put("rawHtml", sour);
json.put("container", jsonArray2);
json.put("rule", jsonArray);
String data2 = json.toString();
Template tem = new Template();
JSONObject getReturnData = tem.displayForSingle(data2);
String data = getReturnData.toString();
response.getWriter().write(data);
return null;
} /**
* 封装单点测试值
*/ public JSONArray point(String key1, String value1, String key2,
String value2, JSONArray jsonArray) {
String rules = "";
String rules1 = "";
if (!"".equals(value1) && !"".equals(value2)) {
rules1 = key1 + ":" + value1;
rules = key2 + ":" + value2;
jsonArray.add(rules1);
jsonArray.add(rules);
} else if (!"".equals(value1) && "".equals(value2)) {
rules = key1 + ":" + value1;
jsonArray.add(rules);
} else if ("".equals(value1) && !"".equals(value2)) {
rules = key2 + ":" + value2;
jsonArray.add(rules);
}
return jsonArray; } /**
* 把频道添加到数据库
*
* @throws IOException
*/
public String addbase() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
if (!regularGroup(regurl, base)) {
response.getWriter().write("{\"code\":\"1\"}");
return null;
}
HttpServletRequest request = ServletActionContext.getRequest();
String templist = packcategory();
Template add = new Template();
JSONObject json = add.persistTemplate(request, templist);
String data = json.toString();
response.getWriter().write(data);
return null;
} /**
* 把内容添加到数据库
*
* @throws IOException
*/
public String adddetail() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
boolean flag=regularGroup(regurl, base);
if (flag==false) {
response.getWriter().write("{\"code\":\"1\"}");
return null;
}
HttpServletRequest request = ServletActionContext.getRequest(); String templist = packdetail();
Template add = new Template();
JSONObject json = add.persistTemplate(request, templist);
String data = json.toString();
response.getWriter().write(data);
return null;
} /**
* 把关键字模板添加到数据库
*
* @throws IOException
*/
public String addkwlist() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
boolean flag=regularGroup(regurl, base);
if (flag==false) {
response.getWriter().write("{\"code\":\"1\"}");
return null;
}
HttpServletRequest request = ServletActionContext.getRequest();
String templist = packkwlist();
Template add = new Template();
JSONObject json = add.persistTemplate(request, templist);
String data = json.toString();
response.getWriter().write(data);
return null;
} /**
* 把json模板添加到数据库
*
* @throws IOException
*/
public String addjson() throws IOException{
HttpServletResponse response = ServletActionContext.getResponse();
response.setContentType("text/json;charset=gb2312");
boolean flag=regularGroup(regurl, base);
if (flag==false) {
response.getWriter().write("{\"code\":\"1\"}");
return null;
}
HttpServletRequest request = ServletActionContext.getRequest();
String templist = jsonTemplate();
System.out.println(templist);
Template add = new Template();
JSONObject json = add.persistTemplate(request, templist);
String data = json.toString();
response.getWriter().write(data);
return null;
}
/**
* 获取频道入口的总测试
*
* @return
* @throws IOException
*/
public String categoryTemplate() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
HttpServletRequest request = ServletActionContext.getRequest();
response.setContentType("text/json;charset=gb2312");
String templist = packcategory();
Template tem = new Template();
JSONObject getReturnData = tem.displayCrawl(templist, base, request);
String data = getReturnData.toString();
response.getWriter().write(data);
return null;
} /**
* 获取关键词总测试
*
* @return
* @throws IOException
*/
public String kwlist() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
HttpServletRequest request = ServletActionContext.getRequest();
response.setContentType("text/json;charset=gb2312");
String templist = packkwlist();
Template tem = new Template();
JSONObject getReturnData = tem.displayCrawl(templist, base, request);
String data = getReturnData.toString();
response.getWriter().write(data);
return null;
} /**
* 获取内容页模板总测试
*
* @return
* @throws IOException
*/
public String detail() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
HttpServletRequest request = ServletActionContext.getRequest();
response.setContentType("text/json;charset=gb2312");
String templist = packdetail();
Template tem = new Template();
JSONObject getReturnData = tem.displayCrawl(templist, base, request);
String data = getReturnData.toString();
response.getWriter().write(data);// 这里面传的是json字符串
return null;
} /**
* 获取json模板总测试
*
* @return
* @throws IOException
*/
public String jsontem() throws IOException {
HttpServletResponse response = ServletActionContext.getResponse();
HttpServletRequest request = ServletActionContext.getRequest();
response.setContentType("text/json;charset=gb2312");
String templist = jsonTemplate();
System.out.println(templist);
Template tem = new Template();
JSONObject getReturnData = tem.displayCrawl(templist, base, request);
String data = getReturnData.toString();
response.getWriter().write(data);// 这里面传的是json字符串
return null;
} /**
* 封装关键字总模板
*/
public String packkwlist() {
String containerrules = "";
String rules1 = "";
String urlrules = "";
String titlerules = "";
String autorules = "";
String sourrules = "";
String browrules = "";
String comrules = "";
String ptimerules = "";
String sumrules = "";
String otherrule=""; // 判断分页匹配
rules1 = detailedrules(detailedkey1, detailedkey2, detailedvalue1,
detailedvalue2);
// 获得块抽取
containerrules = containerules(containerkey1, containerkey2,
containervalue1, containervalue2);
// 判断标题
titlerules = rules(titlekey1, titlekey2, titlevalue1, titlevalue2); // 判断作者
autorules = rules(authorkey1, authorkey2, authorvalue1, authorvalue2);
// 判断来源
sourrules = rules(sourcekey1, sourcekey2, sourcevalue1, sourcevalue2);
// 判断浏览数
browrules = denumber(browsekey1, browsekey2, browsevalue1,
browsevalue2, "${browse}");
// 判断评论数
comrules = denumber(commentkey1, commentkey2, commentvalue1,
commentvalue2, "${comment}");
// 判断发布时间
ptimerules = rules(ptimekey1, ptimekey2, ptimevalue1, ptimevalue2);
// 判断原文摘要
sumrules = rules(summarykey1, summarykey2, summaryvalue1, summaryvalue2);
// 判断链接抽取
urlrules = denumber(urlkey1, urlkey2, urlvalue1, urlvalue2, "${url}");
//判断其它抽取
otherrule=rules(otherkey1, otherkey2, othervalue1, othervalue2); regurl = regurl.replace("\\", "\\\\").replace("\"", "\\\"");
keywords = keywords.replace("\\", "\\\\").replace("\"", "\\\"");
base = base.replace("\\", "\\\\").replace("\"", "\\\"");
if("".equals(factor)||factor.equals(null)){
factor="";
}
String rules = "{\"regurl\": \"" + regurl + "\",\"template\": {";
String rules2 = "\"urls\": {\"base\": \""
+ base
+ "\",\"params\": {\"keywords\": \""
+ keywords
+ "\","
+ "\"beginPage\": \""
+ beginPage
+ "\",\"factor\": \""
+ factor
+ "\",\"pages\": \""
+ pages
+ "\","
+ "\"urlencode\": \"utf-8\",\"enable\": \"true\"},\"method\": \"GET\"},"; String con = "\"container\": {" + containerrules + "},";
String fields = "\"title\": {\"name\": \"title\"," + titlerules
+ "},\"source\": {\"name\": \"source\"," + sourrules + "},"
+ "\"ptime\": {\"name\": \"ptime\"," + ptimerules
+ "},\"summary\": {\"name\": \"summary\"," + sumrules + "},"
+ "\"browse\": {\"name\": \"browse\"," + browrules
+ "},\"comment\": {\"name\": \"comment\"," + comrules + "},"
+ "\"author\": {\"name\": \"author\"," + autorules
+ "},\"url\": {\"name\": \"url\"," + urlrules + "}," +
"\"other\": {\"name\": \"other\"," + otherrule + "},"; String fields3 = "\"type\": {\"name\": \"type\",\"rule\": [\"value:"
+ type
+ "\"]},\"editor\": {\"name\": \"editor\",\"rule\": [\"value:"
+ editor + "\"]},"
+ "\"site\": {\"name\": \"site\",\"rule\": [\"value:" + site
+ "\"]}";
String fields2 = "\"fields\": {" + fields + fields3 + "},"; String pipeline = "\"pipeline\": {\"type\": \"mongo\",\"params\": {\"host\": \"192.168.3.92\",\"port\": 27017,\"mydb\": \"zjj\",\"mytable\": \"alibaba\","
+ "\"user\": \"test\",\"pwd\": \"000000\"}},";
String pipeline2 = "\"request_headers\": {\"Accept\": \"text/html,application/xhtml+xml,application/xml;q=0.9, */*;q=0.8\","
+ "\"Accept-Language\": \"zh_cn\"},\"concurrent_requests\": \"1\","
+ "\"download_delay\": {\"enable_random\": \"false\",\"download_timeout\": \"1000\"},"
+ "\"add_proxy\": \"true\",\"version\": \"2\",\"encode\": \""
+ encode + "\",\"template_type\": \"kwlist\""; String pipeline3 = "\"option\": {" + pipeline2 + "} }}";
String templist = rules + rules1 + rules2 + con + fields2 + pipeline
+ pipeline3; return templist;
} /**
* 封装内容模板
*/
public String packdetail() {
String rules1 = "";
String titlerules = "";
String autorules = "";
String sourrules = "";
String browrules = "";
String comrules = "";
String ptimerules = "";
String contenrules = "";
String sumrules = "";
String reprules=""; // 判断分页匹配
rules1 = detailedrules(detailedkey1, detailedkey2, detailedvalue1,
detailedvalue2);
// 判断标题
titlerules = rules(titlekey1, titlekey2, titlevalue1, titlevalue2);
// 判断作者
autorules = rules(authorkey1, authorkey2, authorvalue1, authorvalue2);
// 判断来源
sourrules = rules(sourcekey1, sourcekey2, sourcevalue1, sourcevalue2);
// 判断浏览数
browrules = denumber(browsekey1, browsekey2, browsevalue1,
browsevalue2, "${browse}");
// 判断评论数
comrules = denumber(commentkey1, commentkey2, commentvalue1,
commentvalue2, "${comment}");
//判断转载数
reprules=denumber(reprintedkey1, reprintedkey2, reprintedvalue1,
reprintedvalue2, "${reprinted}");
// 判断发布时间
ptimerules = rules(ptimekey1, ptimekey2, ptimevalue1, ptimevalue2);
// 判断原文内容
contenrules = rules(contentkey1, contentkey2, contentvalue1,
contentvalue2);
// 判断原文摘要
sumrules = rules(summarykey1, summarykey2, summaryvalue1, summaryvalue2); regurl = regurl.replace("\\", "\\\\").replace("\"", "\\\"");
type = type.replace("\\", "\\\\").replace("\"", "\\\"");
editor = editor.replace("\\", "\\\\").replace("\"", "\\\"");
site = site.replace("\\", "\\\\").replace("\"", "\\\"");
category = category.replace("\\", "\\\\").replace("\"", "\\\""); String rules = "{\"regurl\": \"" + regurl + "\",\"template\": {";
String url = "\"urls\": {\"base\": \"" + base + "\"},";
String fields = "\"title\": {\"name\": \"title\","
+ titlerules
+ "},\"url\": {\"name\": \"url\",\"rule\": [\"value:${url}\"]},"
+ "\"source\": {\"name\": \"source\"," + sourrules
+ "},\"ptime\": {\"name\": \"ptime\"," + ptimerules
+ "},\"content\": {\"name\": \"content\"," + contenrules + "},"
+ "\"browse\": {\"name\": \"browse\"," + browrules
+ "},\"comment\": {\"name\": \"comment\"," + comrules
+ "},\"reprinted\": {\"name\": \"reprinted\"," + reprules
+ "},\"author\": {\"name\": \"author\"," + autorules
+ "},\"summary\": {\"name\": \"summary\"," + sumrules + "},"; String fields3 = "\"type\": {\"name\": \"type\",\"rule\": [\"value:"
+ type
+ "\"]},\"icp\": {\"name\": \"icp\",\"rule\": [\"value:"
+ icp
+ "\"]},\"editor\": {\"name\": \"editor\",\"rule\": [\"value:"
+ editor
+ "\"]},"
+ "\"site\": {\"name\": \"site\",\"rule\": [\"value:"
+ site
+ "\"]},\"category\": {\"name\": \"category\",\"rule\": [\"value:"
+ category + "\"]}";
String fields2 = "\"fields\": {" + fields + fields3 + "},"; String pipeline = "\"pipeline\": {\"type\": \"mongo\",\"params\": {\"host\": \"192.168.3.92\",\"port\": 27017,\"mydb\": \"zjj\",\"mytable\": \"alibaba\","
+ "\"user\": \"test\",\"pwd\": \"000000\"}},";
String pipeline2 = "\"request_headers\": {\"Accept\": \"text/html,application/xhtml+xml,application/xml;q=0.9, */*;q=0.8\","
+ "\"Accept-Language\": \"zh_cn\"},\"concurrent_requests\": \"1\","
+ "\"download_delay\": {\"enable_random\": \"false\",\"download_timeout\": \"1000\"},"
+ "\"add_proxy\": \"true\",\"version\": \"2\",\"encode\": \""
+ encode + "\",\"template_type\": \"detail\""; String pipeline3 = "\"option\": {" + pipeline2 + "} }}";
String templist = rules + url + rules1 + fields2 + pipeline + pipeline3;
return templist;
} /**
* 封装频道模板
*/
public String packcategory() {
String rules1 = "";
String containerrules = "";
String urlrules = ""; // 判断分页匹配规则
rules1 = detailedrules(detailedkey1, detailedkey2, detailedvalue1,
detailedvalue2);
// 判断块抽取
containerrules = containerules(containerkey1, containerkey2,
containervalue1, containervalue2);
// 判断链接抽取
urlrules = denumber(urlkey1, urlkey2, urlvalue1, urlvalue2, "${url}"); regurl = regurl.replace("\\", "\\\\").replace("\"", "\\\"");
base = base.replace("\\", "\\\\").replace("\"", "\\\"");
site = site.replace("\\", "\\\\").replace("\"", "\\\"");
editor = editor.replace("\\", "\\\\").replace("\"", "\\\"");
category = category.replace("\\", "\\\\").replace("\"", "\\\""); String rules = "{\"regurl\": \"" + regurl + "\",\"template\": {";
String rules2 = "\"urls\": {\"base\": \"" + base
+ "\",\"method\": \"GET\"},";
String con = "\"container\": {" + containerrules + "},";
String fields = "\"url\": {\"name\": \"url\"," + urlrules
+ "},\"editor\": {\"name\": \"editor\","
+ "\"rule\": [\"value:" + editor
+ "\"]},\"site\": {\"name\": \"site\",\"rule\": [\"value:"
+ site + "\"]},"
+ "\"category\": {\"name\": \"category\",\"rule\": [\"value:"
+ category + "\"]}";
String fields2 = "\"fields\": {" + fields + "},";
String pipeline = "\"pipeline\": {\"type\": \"mongo\",\"params\": {\"host\": \"192.168.3.92\",\"port\": 27017,\"mydb\": \"zjj\",\"mytable\": \"alibaba\","
+ "\"user\": \"test\",\"pwd\": \"000000\"}},";
String pipeline2 = "\"request_headers\": {\"Accept\": \"text/html,application/xhtml+xml,application/xml;q=0.9, */*;q=0.8\","
+ "\"Accept-Language\": \"zh_cn\"},\"concurrent_requests\": \"1\","
+ "\"download_delay\": {\"enable_random\": \"false\",\"download_timeout\": \"1000\"},"
+ "\"add_proxy\": \"true\",\"version\": \"2\",\"encode\": \""
+ encode + "\",\"template_type\": \"category\""; String pipeline3 = "\"option\": {" + pipeline2 + "} }}";
String templist = rules + rules2 + rules1 + con + fields2 + pipeline
+ pipeline3;
return templist;
} /**
* 封装json模板
*/
public String jsonTemplate(){
String rules1 = "";
String filterrule="";
String temrule="";
String titlerules = "";
String autorules = "";
String sourrules = "";
String browrules = "";
String comrules = "";
String ptimerules = "";
String contenrules = "";
String sumrules = "";
String reprules="";
String urlrules="";
String typerules="";
String editorrules="";
String siterule="";
String caterule="";
String icprule=""; titlerules=rule(titlevalue1);//获取标题
autorules=rule(authorvalue1);//获取作者
urlrules=rule(urlvalue1);//获取链接
sourrules=rule(sourcevalue1);//获取来源
ptimerules=rule(ptimevalue1);//获取发布时间
browrules=rule(browsevalue1);//获取浏览数
comrules=rule(commentvalue1);//获取评论数
typerules=jsonvalue(type);//获取网站类型
editorrules=jsonvalue(editor);//获取模板编辑者
siterule=jsonvalue(site);//获取网站名称
caterule=jsonvalue(category);//获取频道名称
icprule=jsonvalue(icp);//获取icp备案号
reprules=rule(reprintedvalue1);//获取转载数
contenrules=rule(contentvalue1);//获取内容
sumrules=rule(summaryvalue1);//获取摘要
rules1=containerules(containervalue1);//获取块抽取
filterrule=containerules(filter);//获取过滤 String regurlrule="{\"regurl\": \""+regurl+"\",\"template\": {";
if("".equals(keywords)){
temrule="\"urls\": {\"base\": \""+base+"\",\"method\": \"GET\"},";
}else{
temrule="\"urls\": {\"base\": \""+base+"\",\"params\": {\"keywords\": \""+keywords+"\"," +
"\"beginPage\": \""+beginPage+"\",\"factor\": \""+factor+"\",\"pages\": \""+pages+"\",\"urlencode\": \"utf-8\"}," +
"\"method\": \"GET\"},";
} String fit="\"filter\": {\"name\": \"filter\","+filterrule+"},";
String contain="\"container\": {\"name\": \"container\","+rules1+"},"; String fields="\"fields\": {\"title\": {\"name\": \"title\","+titlerules+"},\"url\": {\"name\": \"url\","+urlrules+"}," +
"\"source\": {\"name\": \"source\","+sourrules+"},\"ptime\": {\"name\": \"ptime\","+ptimerules+"}," +
"\"browse\": {\"name\": \"browse\","+browrules+"},\"comment\": {\"name\": \"comment\","+comrules+"}," +
"\"author\": {\"name\": \"author\","+autorules+"},\"type\": {\"name\": \"type\","+typerules+"}," +
"\"editor\": {\"name\":\"editor\","+editorrules+"},\"site\": {\"name\": \"site\","+siterule+"}," +
"\"category\": {\"name\": \"category\","+caterule+"},\"icp\": {\"name\": \"icp\","+icprule+"" +
"},\"reprinted\": {\"name\": \"reprinted\","+reprules+"},\"content\": {\"name\": \"content\","+contenrules+"}," +
"\"summary\": {\"name\": \"summary\","+sumrules+"}},"; String pipeline="\"pipeline\": {\"type\": \"mongo\",\"params\": {\"host\": \"192.168.3.92\",\"port\": 27017," +
"\"mydb\": \"zjj\",\"mytable\": \"alibaba\",\"user\": \"test\",\"pwd\": \"000000\"}},";
String option="\"option\": {\"request_headers\": {\"Accept\": \"text/html,application/xhtml+xml,application/xml;q=0.9, */*;q=0.8\"," +
"\"Accept-Language\": \"zh_cn\"},\"concurrent_requests\": \"1\"," +
"\"download_delay\": {\"enable_random\": \"false\",\"download_timeout\": \"1000\"}," +
"\"add_proxy\": \"true\",\"version\": \"2\",\"encode\": \""+encode+"\",\"template_type\": \"json\"}}}";
String temlist=regurlrule+temrule+fit+contain+fields+pipeline+option;
return temlist;
} /**
* 封装json模板
*
**/
public String rule(String value){
String jsonrule="";
if("".equals(value)){
jsonrule="\"rule\": []";
}else{
jsonrule="\"rule\": [\"json:"+value+"\"]";
}
return jsonrule;
} /**
* 封装json的value
*/
public String jsonvalue(String value){
String jsonrule="";
if("".equals(value)){
jsonrule="\"rule\": []";
}else{
jsonrule="\"rule\": [\"value:"+value+"\"]";
}
return jsonrule;
}
/**
* 封装块抽取，传入4个值
*
* @param key1
* @param key2
* @param value1
* @param value2
* @return
*/
public String containerules(String key1, String key2, String value1,
String value2) {
String containerrules="";
value1 = value1.replace("\\", "\\\\").replace("\"", "\\\"");
value2 = value2.replace("\\", "\\\\").replace("\"", "\\\"");
if ("".equals(value1) && "".equals(value2)) {
containerrules = "\"name\": \"container\",\"rule\": [],\"enable\": \""
+ enableBlock + "\"";
} else if (!"".equals(value1) && "".equals(value2)) {
enableBlock = true;
containerrules = "\"name\": \"container\",\"rule\": [\"" + key1
+ ":" + value1 + "\"],\"enable\": \"" + enableBlock + "\"";
} else if ("".equals(value1) && !"".equals(value2)) {
enableBlock = true;
containerrules = "\"name\": \"container\",\"rule\": [\"" + key2
+ ":" + value2 + "\"],\"enable\": \"" + enableBlock + "\"";
} else {
enableBlock = true;
containerrules = "\"name\": \"container\",\"rule\": [\"" + key1
+ ":" + value1 + "\",\"" + key2 + ":" + value2
+ "\"],\"enable\": \"" + enableBlock + "\"";
}
return containerrules;
}
/**
* 封装块抽取
* @param value
* @return
*/
public String containerules(String value) {
String containerrules="";
value = value.replace("\\", "\\\\").replace("\"", "\\\""); if ("".equals(value)) {
enableBlock = false;
containerrules = "\"rule\": [],\"enable\": \""
+ enableBlock + "\"";
} else {
enableBlock = true;
containerrules = "\"rule\": [\"json:"+value+"\"],\"enable\": \""+enableBlock+"\""; }
return containerrules;
} /**
* 封装单点测试判断
*
* @return
*/
public String rules(String key1, String key2, String value1, String value2) {
String sumrules="";
value1 = value1.replace("\\", "\\\\").replace("\"", "\\\"");
value2 = value2.replace("\\", "\\\\").replace("\"", "\\\"");
if ("".equals(value1) && "".equals(value2)) {
sumrules = "\"rule\": []";
} else if (!"".equals(value1) && "".equals(value2)) {
sumrules = "\"rule\": [\"" + key1 + ":" + value1 + "\"]";
} else if ("".equals(value1) && !"".equals(value2)) {
sumrules = "\"rule\": [\"" + key2 + ":" + value2 + "\"]";
} else {
sumrules = "\"rule\": [\"" + key1 + ":" + value1 + "\",\"" + key2
+ ":" + value2 + "\"]";
}
return sumrules;
} /**
* 封装分页单点测试
*
* @return
*/ public String detailedrules(String key1, String key2, String value1,
String value2) {
String rules1="";
value1 = value1.replace("\\", "\\\\").replace("\"", "\\\"");
value2 = value2.replace("\\", "\\\\").replace("\"", "\\\"");
if ("".equals(value1) && "".equals(value2)) {
rules1 = "\"follow_rules\": {\"#1\": {\"rule\": [],\"enable\":\""
+ enablelist + "\"}},";
} else if (!"".equals(value1) && "".equals(value2)) {
enablelist = true;
rules1 = "\"follow_rules\": {\"#1\": {\"rule\": [\"" + key1 + ":"
+ value1 + "\"],\"enable\":\"" + enablelist + "\"}},";
} else if ("".equals(value1) && !"".equals(value2)) {
enablelist = true;
rules1 = "\"follow_rules\": {\"#1\": {\"rule\": [\"" + key2 + ":"
+ value2 + "\"],\"enable\":\"" + enablelist + "\"}},";
} else {
enablelist = true;
rules1 = "\"follow_rules\": {\"#1\": {\"rule\": [\"" + key1 + ":"
+ value1 + "\",\"" + key2 + ":" + value2
+ "\"],\"enable\":\"" + enablelist + "\"}},";
}
return rules1;
} /**
* 封装默认值
* @param key1
* @param key2
* @param value1
* @param value2
* @param number 没有值传入的时候，填写个默认值
* @return
*/
public String denumber(String key1, String key2, String value1,
String value2, String number) {
String comrules="";
value1 = value1.replace("\\", "\\\\").replace("\"", "\\\"");
value2 = value2.replace("\\", "\\\\").replace("\"", "\\\"");
if ("".equals(value1) && "".equals(value2)) {
comrules = "\"rule\": [\"value:" + number + "\"]";
} else if (!"".equals(value1) && "".equals(value2)) {
comrules = "\"rule\": [\"" + key1 + ":" + value1 + "\"]";
} else if ("".equals(value1) && !"".equals(value2)) {
comrules = "\"rule\": [\"" + key2 + ":" + value2 + "\"]";
} else {
comrules = "\"rule\": [\"" + key1 + ":" + value1 + "\",\"" + key2
+ ":" + value2 + "\"]";
}
return comrules;
} /**
* 正则表达式的匹配
* @param pattern 正则表达式
* @param matcher 源码
* @return
*/
public boolean regularGroup(String pattern, String matcher) {
Pattern p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE);
Matcher m = p.matcher(matcher);
if (m.find()) { // 如果读到
return true;// 返回捕获的数据
} else {
return false; // 否则返回一个空字符串
}
} public String getRegurl() {
return regurl;
} public void setRegurl(String regurl) {
this.regurl = regurl;
} public String getDetailedkey1() {
return detailedkey1;
} public void setDetailedkey1(String detailedkey1) {
this.detailedkey1 = detailedkey1;
} public String getDetailedkey2() {
return detailedkey2;
} public void setDetailedkey2(String detailedkey2) {
this.detailedkey2 = detailedkey2;
} public String getDetailedvalue1() {
return detailedvalue1;
} public void setDetailedvalue1(String detailedvalue1) {
this.detailedvalue1 = detailedvalue1;
} public String getDetailedvalue2() {
return detailedvalue2;
} public void setDetailedvalue2(String detailedvalue2) {
this.detailedvalue2 = detailedvalue2;
} public String getBase() {
return base;
} public void setBase(String base) {
this.base = base;
} public String getKeywords() {
return keywords;
} public void setKeywords(String keywords) {
this.keywords = keywords;
} public String getBeginPage() {
return beginPage;
} public void setBeginPage(String beginPage) {
this.beginPage = beginPage;
} public String getFactor() {
return factor;
} public void setFactor(String factor) {
this.factor = factor;
} public String getPages() {
return pages;
} public void setPages(String pages) {
this.pages = pages;
} public String getUrlencode() {
return urlencode;
} public void setUrlencode(String urlencode) {
this.urlencode = urlencode;
} public String getEncode() {
return encode;
} public void setEncode(String encode) {
this.encode = encode;
} public boolean isEnablelist() {
return enablelist;
} public void setEnablelist(boolean enablelist) {
this.enablelist = enablelist;
} public boolean isEnablePagination() {
return enablePagination;
} public void setEnablePagination(boolean enablePagination) {
this.enablePagination = enablePagination;
} public boolean isEnableBlock() {
return enableBlock;
} public void setEnableBlock(boolean enableBlock) {
this.enableBlock = enableBlock;
} public String getMethod() {
return method;
} public void setMethod(String method) {
this.method = method;
} public String getContainerkey1() {
return containerkey1;
} public void setContainerkey1(String containerkey1) {
this.containerkey1 = containerkey1;
} public String getContainerkey2() {
return containerkey2;
} public void setContainerkey2(String containerkey2) {
this.containerkey2 = containerkey2;
} public String getContainervalue1() {
return containervalue1;
} public void setContainervalue1(String containervalue1) {
this.containervalue1 = containervalue1;
} public String getContainervalue2() {
return containervalue2;
} public void setContainervalue2(String containervalue2) {
this.containervalue2 = containervalue2;
} public String getTitlekey1() {
return titlekey1;
} public void setTitlekey1(String titlekey1) {
this.titlekey1 = titlekey1;
} public String getTitlekey2() {
return titlekey2;
} public void setTitlekey2(String titlekey2) {
this.titlekey2 = titlekey2;
} public String getTitlevalue1() {
return titlevalue1;
} public void setTitlevalue1(String titlevalue1) {
this.titlevalue1 = titlevalue1;
} public String getTitlevalue2() {
return titlevalue2;
} public void setTitlevalue2(String titlevalue2) {
this.titlevalue2 = titlevalue2;
} public String getUrlkey1() {
return urlkey1;
} public void setUrlkey1(String urlkey1) {
this.urlkey1 = urlkey1;
} public String getUrlkey2() {
return urlkey2;
} public void setUrlkey2(String urlkey2) {
this.urlkey2 = urlkey2;
} public String getUrlvalue1() {
return urlvalue1;
} public void setUrlvalue1(String urlvalue1) {
this.urlvalue1 = urlvalue1;
} public String getUrlvalue2() {
return urlvalue2;
} public void setUrlvalue2(String urlvalue2) {
this.urlvalue2 = urlvalue2;
} public String getSourcekey1() {
return sourcekey1;
} public void setSourcekey1(String sourcekey1) {
this.sourcekey1 = sourcekey1;
} public String getSourcekey2() {
return sourcekey2;
} public void setSourcekey2(String sourcekey2) {
this.sourcekey2 = sourcekey2;
} public String getSourcevalue1() {
return sourcevalue1;
} public void setSourcevalue1(String sourcevalue1) {
this.sourcevalue1 = sourcevalue1;
} public String getSourcevalue2() {
return sourcevalue2;
} public void setSourcevalue2(String sourcevalue2) {
this.sourcevalue2 = sourcevalue2;
} public String getBrowsekey1() {
return browsekey1;
} public void setBrowsekey1(String browsekey1) {
this.browsekey1 = browsekey1;
} public String getBrowsekey2() {
return browsekey2;
} public void setBrowsekey2(String browsekey2) {
this.browsekey2 = browsekey2;
} public String getBrowsevalue1() {
return browsevalue1;
} public void setBrowsevalue1(String browsevalue1) {
this.browsevalue1 = browsevalue1;
} public String getBrowsevalue2() {
return browsevalue2;
} public void setBrowsevalue2(String browsevalue2) {
this.browsevalue2 = browsevalue2;
} public String getCommentkey1() {
return commentkey1;
} public void setCommentkey1(String commentkey1) {
this.commentkey1 = commentkey1;
} public String getCommentkey2() {
return commentkey2;
} public void setCommentkey2(String commentkey2) {
this.commentkey2 = commentkey2;
} public String getCommentvalue1() {
return commentvalue1;
} public void setCommentvalue1(String commentvalue1) {
this.commentvalue1 = commentvalue1;
} public String getCommentvalue2() {
return commentvalue2;
} public void setCommentvalue2(String commentvalue2) {
this.commentvalue2 = commentvalue2;
} public String getAuthorkey1() {
return authorkey1;
} public void setAuthorkey1(String authorkey1) {
this.authorkey1 = authorkey1;
} public String getAuthorkey2() {
return authorkey2;
} public void setAuthorkey2(String authorkey2) {
this.authorkey2 = authorkey2;
} public String getAuthorvalue1() {
return authorvalue1;
} public void setAuthorvalue1(String authorvalue1) {
this.authorvalue1 = authorvalue1;
} public String getAuthorvalue2() {
return authorvalue2;
} public void setAuthorvalue2(String authorvalue2) {
this.authorvalue2 = authorvalue2;
} public String getContentkey1() {
return contentkey1;
} public void setContentkey1(String contentkey1) {
this.contentkey1 = contentkey1;
} public String getContentkey2() {
return contentkey2;
} public void setContentkey2(String contentkey2) {
this.contentkey2 = contentkey2;
} public String getContentvalue1() {
return contentvalue1;
} public void setContentvalue1(String contentvalue1) {
this.contentvalue1 = contentvalue1;
} public String getContentvalue2() {
return contentvalue2;
} public void setContentvalue2(String contentvalue2) {
this.contentvalue2 = contentvalue2;
} public String getPtimekey1() {
return ptimekey1;
} public void setPtimekey1(String ptimekey1) {
this.ptimekey1 = ptimekey1;
} public String getPtimekey2() {
return ptimekey2;
} public void setPtimekey2(String ptimekey2) {
this.ptimekey2 = ptimekey2;
} public String getPtimevalue1() {
return ptimevalue1;
} public void setPtimevalue1(String ptimevalue1) {
this.ptimevalue1 = ptimevalue1;
} public String getPtimevalue2() {
return ptimevalue2;
} public void setPtimevalue2(String ptimevalue2) {
this.ptimevalue2 = ptimevalue2;
} public String getSummarykey1() {
return summarykey1;
} public void setSummarykey1(String summarykey1) {
this.summarykey1 = summarykey1;
} public String getSummarykey2() {
return summarykey2;
} public void setSummarykey2(String summarykey2) {
this.summarykey2 = summarykey2;
} public String getSummaryvalue1() {
return summaryvalue1;
} public void setSummaryvalue1(String summaryvalue1) {
this.summaryvalue1 = summaryvalue1;
} public String getSummaryvalue2() {
return summaryvalue2;
} public void setSummaryvalue2(String summaryvalue2) {
this.summaryvalue2 = summaryvalue2;
} public String getType() {
return type;
} public void setType(String type) {
this.type = type;
} public String getEditor() {
return editor;
} public void setEditor(String editor) {
this.editor = editor;
} public String getSite() {
return site;
} public void setSite(String site) {
this.site = site;
} public String getCategory() {
return category;
} public void setCategory(String category) {
this.category = category;
} public String getResults() {
return results;
} public void setResults(String results) {
this.results = results;
} public String getDownloadSource() {
return downloadSource;
} public void setDownloadSource(String downloadSource) {
this.downloadSource = downloadSource;
} public String getSour() {
return sour;
} public void setSour(String sour) {
this.sour = sour;
} public String getIcp() {
return icp;
} public void setIcp(String icp) {
this.icp = icp;
} public String getReprintedkey1() {
return reprintedkey1;
} public void setReprintedkey1(String reprintedkey1) {
this.reprintedkey1 = reprintedkey1;
} public String getReprintedkey2() {
return reprintedkey2;
} public void setReprintedkey2(String reprintedkey2) {
this.reprintedkey2 = reprintedkey2;
} public String getReprintedvalue1() {
return reprintedvalue1;
} public void setReprintedvalue1(String reprintedvalue1) {
this.reprintedvalue1 = reprintedvalue1;
} public String getReprintedvalue2() {
return reprintedvalue2;
} public void setReprintedvalue2(String reprintedvalue2) {
this.reprintedvalue2 = reprintedvalue2;
} public String getOtherkey1() {
return otherkey1;
} public void setOtherkey1(String otherkey1) {
this.otherkey1 = otherkey1;
} public String getOtherkey2() {
return otherkey2;
} public void setOtherkey2(String otherkey2) {
this.otherkey2 = otherkey2;
} public String getOthervalue1() {
return othervalue1;
} public void setOthervalue1(String othervalue1) {
this.othervalue1 = othervalue1;
} public String getOthervalue2() {
return othervalue2;
} public void setOthervalue2(String othervalue2) {
this.othervalue2 = othervalue2;
} public String getFilter() {
return filter;
} public void setFilter(String filter) {
this.filter = filter;
} public static void main(String[] args) throws UnsupportedEncodingException {
String name="中石油";
String url=URLEncoder.encode(name, "GBK");
System.out.println(url); }
}
然后还要配置struts.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE struts PUBLIC "-//Apache Software Foundation//DTD Struts Configuration 2.1//EN" "http://struts.apache.org/dtds/struts-2.1.dtd">
<struts>
<package name="strutsPackage" extends="struts-default" namespace="/">
<!-- -->
<action name="categoryTemplate" class="com.geedata.temp.action.TemplateAction" method="categoryTemplate">
</action>
<action name="detail" class="com.geedata.temp.action.TemplateAction" method="detail">
</action>
<action name="kwlist" class="com.geedata.temp.action.TemplateAction" method="kwlist">
</action>
<action name="getsource" class="com.geedata.temp.action.TemplateAction" method="getsource">
</action>
<!-- 单点测试方法 -->
<action name="getcontainer" class="com.geedata.temp.action.TemplateAction" method="getcontainer">
</action>
<action name="addbase" class="com.geedata.temp.action.TemplateAction" method="addbase">
</action>
<action name="adddetail" class="com.geedata.temp.action.TemplateAction" method="adddetail">
</action>
<action name="packdetail" class="com.geedata.temp.action.TemplateAction" method="packdetail">
</action>
<action name="addkwlist" class="com.geedata.temp.action.TemplateAction" method="addkwlist">
</action>
<action name="getkeysource" class="com.geedata.temp.action.TemplateAction" method="getkeysource">
</action>
<!-- 查询方法 -->
<action name="query" class="com.geedata.temp.action.TemplateAction" method="query">
</action>
<!--获取json模板总测试 -->
<action name="jsontem" class="com.geedata.temp.action.TemplateAction" method="jsontem">
</action>
<!--获取json模板添加 -->
<action name="addjson" class="com.geedata.temp.action.TemplateAction" method="addjson">
</action> </package> </struts>
因为我这返回的都是null,也不需要跳转页面，所以不需要成功或者失败的跳转，如果想要成功或者失败
<action name="getContainer" class="com.geedata.temp.action.TemplateAction" method="getContainer">
<result name="success">jsp/contenttemplate.jsp</result>
<result name="login">login.jsp</result>
</action>
//删除记录回调函数，默认调用刷新记录方法，
function deleteCallBack(){
reloadGrid("grid");
}