A persistent, network resilient, full text search library for the browser and Node.js
https://github.com/fergiemcdowall/norch
https://github.com/fergiemcdowall/search-index
使用的是levelDB存储索引,不过目前没有明白,对于倒排索引,其是否适合?类似思路还有:
https://github.com/patrickfrey/strus
Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to store its data. Currently there exists an implementation based on the google LevelDB library. http://www.project-strus.net
用法:https://www.codeproject.com/articles/1059766/building-a-search-engine-with-python-tornado-and-s
go写的,评分比较高的,也是levelDB实现底层存储:
https://github.com/blevesearch/bleve 功能强大 支持facet等
https://github.com/huichen/wukong 2000+star 貌似使用的boltBD存储 支持了分布式 源码分析在https://ayende.com/blog/171745/code-reading-wukong-full-text-search-engine 可以看到其使用的内存 持久化支持kv DB
https://github.com/basho/riak_search/ 应该使用的是riak(kv存储)中sets来存储倒排索引 不过是erlang语言写的让人很忧伤啊
https://github.com/victorparmar/zsearch 也是levelDB做的,看起来很牛叉的样子,Low Data fragmentation and good random write performance by using levelDB Log Structured Merge Trees. High performance query speed by using CompressedBitmap to store DocumentIds in an InvertedIndex interface provided by a simple libEvent2 http server.
https://github.com/daviddengcn/gcse 貌似也是levelDB
https://github.com/eugeneware/fulltext-engine
使用mongodb做倒排存储的:
https://github.com/c-bata/gosearch 本质是底层addToSet 见这个即可知 https://github.com/c-bata/gosearch/blob/master/models/index.go
https://github.com/hemslo/poky-engine A simple search engine in python using Tornado, Scrapy, Redis and MongoDB
360的:
linkin的:
https://github.com/linkedin/indextank-engine 比较强大 支持facet等 使用内存和文件两种方式做索引 有时间可以好好研究下 底层文件应该支持压缩
https://github.com/gigablast/open-source-search-engine http://www.gigablast.com/使用的搜索引擎 代码是c++写的不过看起来稍微有点凌乱 也支持索引持久化到数据库
分布式:
https://github.com/izenecloud/sf1r-lite 使用nginx做负载均衡,底层倒排貌似是用hadoop做的 架构图 https://raw.githubusercontent.com/izenecloud/sf1r/master/docs/source/images/sf1r.png
值得一看的:
https://github.com/aaalgo/donkey
https://github.com/groonga/groonga Groonga is an open-source fulltext search engine and column store. It lets you write high-performance applications that requires fulltext search.列存储?
ruby的:
https://github.com/mrkamel/search_cop 使用了SQL数据库,支持SQL语句+全文搜索 Search engine like fulltext query support for ActiveRecord
使用lucene做全文搜索的:
go search网站的搜索引擎:(http://go-search.org/search?q=hello)
https://github.com/daviddengcn/gcse 用的是 https://github.com/daviddengcn/go-index 做索引 有时间可以研究下后者
使用原始文件做倒排的:
https://github.com/bradleypeabody/fulltext Pure-Go full text indexer and search library
lucene的go移植版:
https://github.com/philipsoutham/golucy
https://github.com/ipfs-search/ipfs-search 使用ES5做搜索
用sqlite存倒排索引:
https://github.com/gansidui/gose
https://github.com/rsesek/usda-ndb 搜食品成分的
https://github.com/devict/magopie 搜BT种子的
https://github.com/yieldbot/ferret Ferret is a search engine that unifies search results from Github, Slack, Trello and more
https://github.com/ndmitchell/hoogle haskell写的
https://github.com/BitFunnel/BitFunnel
https://github.com/Maxime2/dataparksearch
其他:
https://github.com/KunBetter/GridSearch real-time grid search engine 网格搜索引擎 不知道原理
https://github.com/kanatohodets/carbonsearch search engine for graphite metrics
https://github.com/carrot2/carrot2 可能用到了ES Solr
https://github.com/reyesr/fullproof 使用webSQL或者浏览器DB来存倒排索引的JS搜索引擎
https://github.com/nolanlawson/pouchdb-quick-search 使用小型数据库的离线搜索例如phonegap、app等
https://github.com/legendary001/SearchEngine 使用hadoop+lucene的搜索引擎
不过按照我的观点看,搜索引擎本质上是针对各个field的特定搜索word的列存储。所以其底层实现用tokuDB线性树结构应该更合适,日志的话搜索使用时间序列存储更合适。
本文转自张昺华-sky博客园博客,原文链接:http://www.cnblogs.com/bonelee/p/6265853.html,如需转载请自行联系原作者