Business Weekly - Cambridge, UK

Lemur Consulting, a venture started in 2001 by former employees of Muscat is spearheading a new project which aims to topple Google as the king of internet search; while their former employers, search doyens Dr Martin Porter and John Snyder have broken cover on their fifth search engine, Grapeshot.

Given the stranglehold that Google has on internet search and Cambridge company Autonomy on enterprise search, bravado is a perhaps necessary element of the two companies' marketing armoury, but the award for grandest design goes to Lemur.

Lemur Consulting, started in 2001 by Charlie Hull, Tom Mortimer and Richard Boulton is part of a consortium that has been awarded a DTI grant of £1.5m to build a next generation web search engine, with the ultimate intention of toppling Google from its lofty perch.

According to Lemur MD, Charlie Hull, the consortium is “still very much at the beginning of what can be done” but the project will draw upon expertise in brain and neuron research and also probabilistic information retrieval.

Early web search engines, such as AltaVista, simply presented a list of documents which contained the words given in the query. Second generation engines, for example Google, added other relevance criteria such as page rank when returning a list of web pages.

Now Lemur is helping to develop the third generation of web search engines based on deeper criteria for defining the relevance of the results returned.

The partners will concentrate on the concept of intention as it applies to search, taking into account both web publishers and users. For example, is the page designed purely as an information resource, or is it designed to sell you something? Is the person making the query interested in researching, or buying?

Understandably Hull is not keen to elaborate a great deal further on the approach that the partners will take, but he revealed that they would also concentrate on image search, a problem he says, that “still has not been remotely cracked.”

“It is perhaps egotistical to say that we can compete with a company like Google,” said Hull.

“But it wasn't too long ago that people were holding up Altavista as a shining example in the internet industry. Five years later, nobody had heard of them. Things change and things get better.”

The lifetime of the project is two years, after which time, Hull hopes that sufficiently high quality results will have been achieved to warrant the formation of a company to exploit it.

Hull said: “We have a number of exciting projects on the go at the moment, but we are hoping that the DTI-funded project will be a key driver of our expansion.”

Much of Lemur's work is based around Flax and Xapian, both open source software. The open source business model is already winning market share from the proprietary or closed source camp in a number of key segments of the software market and Hull believes “the time is ripe” for an open source search engine.

The company hopes to emulate the success of Mozilla and OpenOffice in the web browser and office software spaces respectively; as well as local proponents XenSource, RealVNC and RedHat.

Open source software is made available under licence to anyone to study, change, and improve the software, and to redistribute it in modified or unmodified form. The strong community of active users that open source software often generates can contribute bug fixes if the product is developer-oriented or provide feedback on desired features.

Companies such as Lemur generate revenues by selling their intimate knowledge of the software to large corporates that need a specialist implementation of it.

“With something as complicated as search, it doesn't make sense for it to be produced in a closed environment. There is a lot to be said for the widespread, independent verification and assessment that open source software receives.”

Cambridge University scientist and one of the founding fathers of search, Dr Martin Porter along with John Snyder have in recent weeks lifted their Grapeshot venture out of stealth mode having successfully road tested the technology with IBM and “large billion-dollar revenue US customers.”

The re-emergence of the Batman and Robin of search after a six year hiatus will create a great deal of interest within the industry, not least because Dr Porter's maths has been integrated into Google and Microsoft's search engines.

The pair started Muscat in 1992 commercialising Dr Porter's University research on the MUSeum CATaloguing system. Muscat powered the first European internet search engine called Euroferret and provided search engines for the main BBC website, Reuters, Nokia, Shell and other large corporations. The technology was sold in 1997 for $15m to the Dialog Corporation (now part of the Thomson Financial publishing group).

Snyder, who has a long-running, although unlitigated beef with Autonomy founder, Mike Lynch about the provenance of Lynch's technology has the enterprise search market leader firmly in his sights with the new offering, if Grapeshot's website is anything to go by: “Back in 1994, at the start of Internet 1.0 era, it was actually the Grapeshot founders who first showed the Autonomy founders not only the power of the emerging world wide web back then, but also how Bayesian techniques could unlock the search through unstructured text (but that is a long story...),” it states.

Grapeshot is being licensed as an OEM technology for software application designers to build into their own software. According to the company, third party evaluation has indicated it is very fast – processing queries across millions of documents in 4 milliseconds, on standard home computing hardware.

Among the technology's USPs are ‘multiple-word querying,’ whereby whole documents (400 plus words) or excerpts from emails or browser pages

can be used as the search input. Google has a limit of 12 words.

It also provides dynamic word suggestion - suggesting useful related terms – and variable word-weights.

Anyone wanting to see Grapeshot's technology in action can go to http://search.dooodle.com to try out its news and blog search engine, designed by Business Weekly's own web developer, Black Pig.