Lucene

Lucene

Lucene
Lucene logo
Developer(s) Apache Software Foundation
Stable release 5.3.1 / September 24, 2015 (2015-09-24)
Development status Active
Written in Java
Operating system Cross-platform
Type Search and index
License Apache License 2.0
Website .org.apachelucene

Apache Lucene is a free and open-source information retrieval software library, originally written in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License.

Lucene has been ported to other programming languages including Delphi, Perl, C#, C++, Python, Ruby, and PHP.[1]

Contents

  • History 1
  • Features and common use 2
  • Lucene-based projects 3
  • Users 4
  • See also 5
  • References 6
  • Bibliography 7
  • External links 8

History

Jakarta family of open-source Java products in September 2001 and became its own top-level Apache project in February 2005.

Lucene formerly included a number of sub-projects, such as Lucene.NET, Mahout, Tika and Nutch. Lucene.NET, Mahout, Nutch, and Tika are now independent top-level projects.

In March 2010, the Apache Solr search server joined as a Lucene sub project, merging the developer communities.

Version 4.0 was released on October 12, 2012.[3]

The latest version of Lucene is 5.3.1 which was released on September 24th, 2015.[3]

Features and common use

While suitable for any application that requires full text indexing and searching capability, Lucene has been widely recognized[4][5] for its utility in the implementation of Internet search engines and local, single-site searching.

At the core of Lucene's logical architecture is the idea of a document containing fields of text. This flexibility allows Lucene's API to be independent of the file format. Text from PDFs, HTML, Microsoft Word, Mind Maps, and OpenDocument documents, as well as many others (except images), can all be indexed as long as their textual information can be extracted.[6]

Lucene-based projects

Lucene itself is just an indexing and search library and does not contain crawling and HTML parsing functionality. However, several projects extend Lucene's capability:

  • Apache Nutch — provides web crawling and HTML parsing
  • Apache Solr — an enterprise search server[7]
  • Elasticsearch — an enterprise search server[7][8]
  • Compass — the predecessor to Elasticsearch[9]
  • DocFetcher — a multiplatform desktop search application
  • Lucene.NET — a port of Lucene written in C# and targeted at .NET Framework users. There are currently two variations of the software, differing in Generics support and a few bug fixes.
  • Swiftype - an enterprise search startup based on Lucene[10]
  • Ferret — a search library for Ruby (programming language) inspired by Lucene. There is also a Ruby on Rails plugin called acts_as_ferret.[11][12] Ferret utilizes Poshlib.
  • Kinosearch — a search engine written in Perl and C[13] and a loose port of Lucene.[14] The Socialtext wiki software uses this search engine,[13] and so does the MojoMojo wiki.[15] It is also used by the Human Metabolome Database (HMDB)[16] and the Toxin and Toxin-Target Database (T3DB).[17]
  • Apache Lucy is a successor project of both KinoSearch and Ferret, being jointly developed by the authors of these and having bindings in both Perl and Ruby.[18][19]
  • Luke — A Java-based GUI for Lucene which allows you to display and modify indexes.[20]

Users

For a list of companies that use Lucene (rather than extend), see Lucene's "Powered By" page.[21] As an example, Twitter is using Lucene for its real time search.[22]

See also

References

  1. ^ "LuceneImplementations". apache.org. Retrieved 23 September 2015. 
  2. ^ "Better Search with Apache Lucene and Solr" (PDF). 19 November 2007. 
  3. ^ a b "Apache Lucene - Welcome to Apache Lucene". apache.org. Retrieved 23 September 2015. 
  4. ^ McCandless, Michael; Hatcher, Erik; Gospodnetić, Otis (2010). Lucene in Action, Second Edition. Manning. p. 8.  
  5. ^ GNU/Linux Semantic Storage System
  6. ^ Perner, Petra (2007). Machine Learning and Data Mining in Pattern Recognition: 5th International Conference. Springer. p. 387.  
  7. ^ a b "What are the main differences between ElasticSearch, Apache Solr and SolrCloud? - Quora". quora.com. Retrieved 23 September 2015. 
  8. ^ "Elasticsearch: RESTful, Distributed Search & Analytics - Elastic". elastic.co. Retrieved 23 September 2015. 
  9. ^ "The Future of Compass & Elasticsearch". the dude abides (in en-US). Retrieved 2015-10-14. 
  10. ^ Riley, Matt (May 9, 2012). "What is the technology stack behind Swiftype? - Quora". Quora. Retrieved 3 October 2014. 
  11. ^ https://github.com/jkraemer/ferret Ferret-Github repository
  12. ^ http://www.jkraemer.net/projects/acts_as_ferret
  13. ^ a b Natividad, Angela. "Socialtext Updates Search, Goes Kino". CMS Wire. Retrieved 2011-05-31. 
  14. ^ Marvin Humphrey. "KinoSearch - Search engine library. - metacpan.org". p3rl.org. Retrieved 23 September 2015. 
  15. ^ Diment, Kieren; Trout, Matt S (2009). "Catalyst Cookbook". The Definitive Guide to Catalyst.  
  16. ^ "HMDB: a knowledgebase for the human metabolome".  
  17. ^ "T3DB: a comprehensively annotated database of common toxins and their targets". Nucleic Acids Res. 38 (Database issue): D781–6. January 2010.  
  18. ^ Michael McCandless; Erik Hatcher; Otis Gospodnetić (2010). Lucene in Action (2 ed.).  
  19. ^ "Apache Lucy". apache.org. Retrieved 23 September 2015. 
  20. ^ "DmitryKey/luke". GitHub. Retrieved 2015-10-14. 
  21. ^ "PoweredBy". apache.org. Retrieved 23 September 2015. 
  22. ^ MG Siegler. "Twitter Quietly Launched A New Search Backend Weeks Ago". TechCrunch. AOL. Retrieved 23 September 2015. 

Bibliography

  • Gospodnetic, Otis; Erik Hatcher; Michael McCandless (28 June 2009). Lucene in Action (2nd ed.).  
  • Gospodnetic, Otis; Erik Hatcher (1 December 2004). Lucene in Action (1st ed.).  

External links

  • Official website
  • Lucene.NET
  • List of Lucene Ports (or Implementations) in Other Languages on the Apache wiki
  • Introductory article with Java code for search (This page is on Internet Archive) on WorldHeritage data
  • An article with explanation and example implementation of Lucene.Net
  • Apache Lucene popular APIs in GitHub