常用的Lucene查询代码如下所示，该代码的作用是将path路径下的所有索引信息返回
public String matchAll(String path) {
try {
Directory directory = FSDirectory.open(new File(path));
DirectoryReader reader = DirectoryReader.open(directory);
IndexSearcher searcher = new IndexSearcher(reader);
MatchAllDocsQuery query = new MatchAllDocsQuery();
ScoreDoc[] hits = searcher.search(query, null, Integer.MAX_VALUE).scoreDocs;
StringBuffer buffer = new StringBuffer();
for (int i = 0; i < hits.length; i++) {
Document hitDocument = searcher.doc(hits[i].doc);
// System.out.println(hitDocument.get("key")
// + "......"+hitDocument.get("value"));
buffer.append(hitDocument.get("key")+";"+hitDocument.get("value")+"|");
}
return buffer.toString();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
但是当该文件夹下索引的数目比较巨大，那么在执行以下代码的时候，则会出现java.lang.OutOfMemoryError: Java heap space的提示
ScoreDoc[] hits = searcher.search(query, null, Integer.MAX_VALUE).scoreDocs;
这时候，我们可以考虑使用分页技术，比如以前大约1亿条数据，我们可以将其分成100个100W的页，每次对100W条索引数据进行处理，这样就可以避免上述情况的发生。在Lucene 中，我们使用searchAfter的方法实现上述功能。它的官方API介绍如下所示：
public TopDocs searchAfter(ScoreDoc after,
Query query,
int n)
throws IOException
Finds the top
n hits for query where all results are after a previous result (after).
By passing the bottom result from a previous page as after, this method can be used for efficient 'deep-paging' across potentially large result sets.
private String transToContent(IndexSearcher searcher,TopDocs topDocs) throws IOException {
ScoreDoc[] scoreDocs = topDocs.scoreDocs;
StringBuffer sb = new StringBuffer();
for(int i=0; i<scoreDocs.length; i++) {
Document doc = searcher.doc(scoreDocs[i].doc);
sb.append(doc.get("key")+";"+doc.get("value")+"|");
}
return sb.toString();
}
private void matchAll(String path) {
try {
Directory directory = FSDirectory.open(new File(path));
DirectoryReader reader = DirectoryReader.open(directory);
IndexSearcher searcher = new IndexSearcher(reader);
ScoreDoc after = null;
TopDocs topDocs = searcher.searchAfter(after, new MatchAllDocsQuery(), Preference.PAGE_SIZE);
int curPage = 1;
while(topDocs.scoreDocs.length > 0) {
System.out.println("Current Page:"+ (curPage++) );
System.out.println(transToContent(searcher, topDocs));
after = topDocs.scoreDocs[topDocs.scoreDocs.length -1];
topDocs = searcher.searchAfter(after, new MatchAllDocsQuery(), Preference.PAGE_SIZE);
}
} catch (IOException e) {
e.printStackTrace();
}
}